Assessing multilevel construct validity with latent variable modeling : validating school-level inferences in college base

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] The purpose of the current study is to examine multilevel construct validity of College BASE (Basic Academic Subjects Examination) science test. In the state of Missouri, College BASE is a required test for admission to teacher education programs and students' test performance is one of the accreditation criteria for teacher training programs. Through multilevel latent variable modeling, this study examined 1) the multilevel factor structure of College BASE science test through a series multilevel latent modeling techniques, 2) the intraclass correlation (ICC) of the College BASE science test scores, 3) the reliability of the College BASE science test at each level of analysis, and 4) the concurrent validity of the College BASE science test in the multilevel model. The results showed strong evidences to support that the College BASE has a consistent one-factor structure at the within (student) and between (school) levels when estimated simultaneously. Thus, it was appropriate to label the primary factor at student level as student science achievement and the one at the school level as school science achievement. The patterns of the intraclass correlations indicated that more variation was accounted for by the within level (students) and very few variation was accounted for by the between level (schools). This suggested that school characteristics had a consistent effect on students' science achievement scores. The results at both levels also provided evidence of good reliability and concurrent validity of the measurement. Implications and limitations of the study were discussed.