extended · article · 2012

A Many-Facet Rasch Measurement of Differential Rater Severity/Leniency in Three Types of Assessment

Farahman Farrokhi; Rajab Esfandiari; Edward Schaefer; George Engelhard Jr.

Rater effects in performance testing is an area in which much new research is needed (Myford, personal communication, 23 February, 2010). While previous studies of bias or interaction effect as a component of rater effect have employed experienced teachers as raters (e.g., Schaefer, 2008), the present study uses many-facet Rasch measurement (MFRM) to investigate differential rater effect or rater severity or leniency among three rater types: self-assessor, peer-assessor, and teacher assessor. Essays written in English by 188 Iranian English majors at two state-run universities in Iran were rated both by the students themselves as self-assessors and peer-assessors and by teachers using a 6-point analytic rating scale. MFRM revealed differing patterns of severity and leniency among the three assessment types. For example, self-assessors and teacher assessors showed the opposite pattern of ...