extended · article · 2015

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Gavin W. Fulmer; Hye‐Eun Chu; David F. Treagust; Knut Neumann; Trevor G. Bond

Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. But are the knowledge and reasoning involved in these tiers really distinguishable? Are the tiers equally challenging for students? The answers to these questions influence how we use and interpret TTMC instruments. We apply the Rasch measurement model on TTMC items to see if the items are distinguishable according to different traits (represented by the tier), or according to different content sub-topics within the instrument, or to both content and tier. Two TTMC data sets are analyzed: data from Singapore and Korea on the Light Propagation Diagnostic Instrument (LPDI), data from the United States on the Classroom Test of Scientific Reasoning (CTSR). Findings for LPDI show that tier-2 reasoning items are more difficult than t...