Reference analysis of non-regular models and nonparametric Bayes modeling of large data

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Bayesian analysis is a principled approach, which makes inference about the parameter, by combining the information gained from the data and the prior belief about the parameter. There's no convergence on the choice of priors, and often different motivations for prior lead to different areas of study in Bayesian statistics. This work is motivated by two such choices, namely: reference priors and nonparametric priors. Reference priors arise out of the need to specify priors in a non-subjective manner, i.e. objective manner. Reference priors maximize the amount of information gained from the data about the parameter, in information theoretical sense. The appeal of reference priors lies in the fact that it has nice frequentist properties even for small sample size and often avoids marginalization paradoxes in Bayesian analysis. However, reference prior algorithms are typically available when the posterior is asymptotically normal and Fisher's information matrix is well-defined. In statistical parlance, such models are called regular case or regular model. Recently, Berger et al. (2009) [1] proposed a general expression of reference prior for single continuous parameter model, which is applicable for both regular and non-regular case. Motivated by Berger et al. (2009) [1], we explore reference prior methodology for a general model. Specifically, we derive expression of reference prior for single continuous parameter truncated exponential family and a general expression of conditional reference prior for multi group continuous parameter model. Furthermore, we demonstrate the usefulness of our work by deriving reference priors for models which have no known existing reference priors. We also extend Datta and Ghosh (1996) [2]'s invariance result for reference prior of regular model to general model. Nonparametric priors arise out of the need to specify priors over a large support.