Predicting predictive accuracy :
performance of fixed-weight decision models compared to ordinary least squares regression
Abstract
[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] It is has been established that alternative weighting schemes to Ordinary Least Squares (OLS) regression can produce accurate predictions in both within-sample and out-of-sample tests of predictive accuracy. Fixed-weight models, often inspired by simple models of decision making known as heuristics, can perform about as well as benchmark estimation processes like OLS, and in some cases even be more accurate. We link recent results on the estimation accuracy (as within-sample parameter estimate accuracy) of simple fixed-weight models to the body of literature demonstrating these same models' predictive accuracy in out-of-sample prediction. Using constrained linear estimation, we determine whether these recent results outlining the conditions necessary for a particular choice of fixed-weighting scheme to outperform OLS in estimation accuracy are indicative of the model's performance in cross-validation predictive accuracy (Davis-Stober, Dana & Budescu, 2010a,b; Davis-Stober, 2011). By way of a simulation study, we explore the predictive accuracy of fixed-weighting schemes inspired by models of decision making. This fills the gap between estimation accuracy and predictive accuracy results shown in the literature, providing researchers with a theory-driven technique for prospectively identifying when they ought to use an alternative to OLS to achieve higher predictive accuracy. Models predicted by previous work outlining the modeling environment conditions (sample size, model predictability, error variance) and bounds for the region of population parameter weights favoring a fixed-weight model over OLS can provide an accurate determination of when a fixed-weight model will outperform OLS in predicting new data.
Degree
M.A.
Thesis Department
Rights
Access is limited to the University of Missouri - Columbia.