Detecting and explaining differential item functioning on the social, academic, and emotional behavior risk screener

Abstract

Universal screening of social-emotional and behavioral (SEB) risk with teacher completed brief behavioral rating scales (BBRS) is one of the primary methods for identifying SEB risk in students. These measures should function similarly across races, ethnicities, and genders. However, there is limited research to support measurement invariance in universal screening for SEB risk. Therefore, the current study sought to expand upon the existing research on measurement invariance. The Emotional Behavior (EB) subscale of the Social, Academic, and Emotional Behavior Risk Screener-Teacher Rating Scale (SAEBRS-TRS) was examined. Measurement invariance was examined through differential item functioning (DIF) within item response theory (IRT). A unidimensional graded response model was fit to the data and indicated that effect sizes of DIF ranged from small to large for Black students compared to all non-Black students (Cohen's d = -0.11 to -0.87) and negligible to medium for White students compared to all non-White students (Cohen's d = -0.01 to 0.54). Effect sizes for Hispanic students and students with multiple races and ethnicities were small to negligible. Positively worded items and males had larger DIF effect sizes. Next, the Item Response Questionnaire (IRQ) was developed from information processes theory to compare the process teachers go through when completing questions on the EB subscale with the median absolute effect sizes. A micro-macro multilevel model was fit to the data and indicated that the IRQ was not a significant predictor of effect sizes. However, teachers' rank ordering of subjectivity of the EB subscale items were significantly negatively correlated with effect sizes. Limitations of the current study, implications for practice, and directions for future research are discussed.