Test Scaling and Value-Added Measurement - WCER 测试结垢与附加值的测量wcer.pptVIP

Test Scaling and Value-Added Measurement Dale Ballou Vanderbilt University April, 2008 VA assessment requires that student achievement be measured on an interval scale: 1 unit of achievement represents the same amount of learning at all points on the scale. Scales that do not have this property: Number right Percentile ranks NCE (normal curve equivalents) IRT “true scores” Scales that may have this property IRT ability trait (“scale score”) Item Response Theory Models One-parameter logistic model Pij = [1 + exp(-D(?i-?j))]-1, Pij is the probability examinee i answers item j correctly ?i is examinee i ability ?j is item j difficulty Two- and Three-Parameter Logistic IRT Models Pij = [1 + exp(-?j(?i-?j))]-1 Pij = cj + (1-cj)[1 + exp(-?j(?i-?j))]-1 ?j is an item discrimination parameter cj is a guessing parameter IRT Isoprobability Contours (1-parameter model) Linear, parallel isoprobability curves are the basis for the claim that ability is measured on an interval scale. The increase in difficulty from item P to item Q offsets the increase in ability from examinee A to B, from B to C, and from C to D. In this respect, AB = BC = CD, etc. Moreover, the same relations hold for any pair of items. Do achievement test data conform to this model? Pij and isoprobability contours aren’t given. Data are typically binary responses. Testable hypotheses can be derived from this structure, but power is low. The model doesn’t fit the data when guessing affects Pij . Or when difficulty and ability are multidimensional. “Data” are selected to conform to the model → ability may be too narrowly defined. Implications: It seems unwise to take claims that ability is measured on an interval scale at face value. We should look at the scales. CTB/McGraw-Hill CTBS Math (1981) CTB/McGraw-Hill Terra Nova, Mean Gain From Previous Grade (Mississippi, 2001) Northwest Evaluation Association, Fall, 2005, Reading Northwest Evaluation Association, Fall, 2005, Math App