extended · article · 2000

MONITORING SOURCES OF VARIABILITY WITHIN THE TEST OF SPOKEN ENGLISH ASSESSMENT SYSTEM

Carol M. Myford; Edward W. Wolfe; George Engelhard Jr.

ABSTRACT The purposes of this study were to examine four sources of variability within the Test of Spoken English (TSE ® ) assessment system, to quantify ranges of variability for each source, to determine the extent to which these sources affect examinee performance, and to highlight aspects of the assessment system that might suggest a need for change. Data obtained from the February and April 1997 TSE scoring sessions were analyzed using Facets (Linacre, 1999a). The analysis showed that, for each of the two TSE administrations, the test usefully separated examinees into eight statistically distinct proficiency levels. The examinee proficiency measures were found to be trustworthy in terms of their precision and stability. It is important to note, though, that the standard error of measurement varies across the score distribution, particularly in the tails of the distribution. The item...