A comparison of kernel equating and item response theory true score equating

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] This study compares the accuracy of NEAT-design IRT true score equating (TIRT) and kernel equating (KE) in different conditions. The relative accuracy of three methods (Stocking-Lord transformation based TIRT (SL TIRT), kernel chain equipercentile equating (KE CE) and kernel post-stratification equating (KE PSE)) are examined in specific and general conditions. Simulated data are used for the analysis. 60 different specific conditions are generated and the relative performances of the methods in each condition are compared using ANOVA. Then the error of each equating line is summarized with RMSD. 180 RMSD values in 60 conditions are categorized into 20 general conditions. RMSD values in each level of the general condition are compared with RMSD. All findings have led to the conclusion that KE CE is the superior equating method for all general and specific conditions. Compared to KE PSE and SL TIRT, it produces more accurate and stable results. Closer examination shows how KE CE is consistently less affected by various adverse conditions, how SL TIRT is more affected, and how KE PSE is drastically affected. Hence, among the three methods, KE CE is preferred over KE PSE and SL TIRT, while SL TIRT is preferred over KE PSE.