extended · article · 2018

Multilevel Rasch modeling of two-tier multiple choice test: A case study using Lawson’s classroom test of scientific reasoning

Yang Xiao; Jing Han; Kathleen Koenig; Jianwen Xiong; Lei Bao; John M. Linacre

Assessment instruments composed of two-tier multiple choice (TTMC) items are widely used in science education as an effective method to evaluate students' sophisticated understanding. In practice, however, there are often concerns regarding the common scoring methods of TTMC items, which include pair scoring and individual scoring schemes. The pair-scoring method is effective in suppressing "false positives" at the cost of missing possible middle states of progression of student understanding. On the other hand, the individual scoring method captures an undistinguished middle level but is prone to rewarding guessing, which leads to "false positives". In addition, this middle level does not discriminate the progression between knowing the result and explaining the reason, which limits the capacity of drawing meaningful implications from the assessment outcomes. To address the concerns wit...