Design considerations in three-level regression discontinuity studies
No Thumbnail Available
Authors
Meeting name
Sponsors
Date
Journal Title
Format
Thesis
Subject
Abstract
In education, sample characteristics can be complex due to the nested structure of students, teachers, classrooms, schools, and districts. In the past, not many considerations were given to such complex sampling schemes in statistical power analysis. More recently in the past two decades, however, education scholars have developed tools to conduct statistical power analysis in randomized experiments (RE) and regression discontinuity (RD) studies considering complex sampling schemes. The purpose of this study is threefold: (i) to derive formulas for various three-level RD studies where discontinuity resides at level 1 and to validate formulas using Monte Carlo (MC) simulations, (ii) to explore consequences of ignoring an intermediate- (e.g., classroom/teacher) or top-level (e.g., school/district) when designing such studies, and (iii) to provide a general framework for calculating optimal sample sizes under budget and sample size constraints when treatment and control units are associated with certain costs (equal or unequal). Derived formulas are consistent with the current literature and uses parameters commonly reported in the education studies. MC simulation results confirm validity of the formulas. On the one hand, ignoring an intermediate-level result in under-powered studies and is not recommended. An intermediate-level may be ignored had the variance of the outcome between level 2 units been small. On the contrary, ignoring top-level result in over-powered studies and is not recommended. In this case, Type I errors are severely inflated, therefore, a researcher is more likely to detect a treatment effect when in reality there is not. Finally, the general framework for constrained optimal sample allocation allows calculation of sample sizes under budget and sample size constraints when treatment and control units are associated with certain cost (equal or unequal). When cost associated with each unit depend on the treatment membership, the proportion of units in treatment condition (P) can also be optimized in multilevel RE studies.
Table of Contents
DOI
PubMed ID
Degree
Ph. D.
Thesis Department
Rights
OpenAccess.
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License.