Bayesian analysis for detecting differentially expressed genes from RNA-seq data

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] This dissertation introduces hmmSeq, a model-based hierarchical Bayesian technique for detecting differentially expressed genes from RNA-seq data. Our novel hmmSeq methodology uses hidden Markov models to account for potential co-expression of neighboring genes. In addition, hmmSeq employs an integrated approach to studies with technical or biological replicates, automatically adjusting for any extra-Poisson variability. Moreover, for cases when paired data are available, hmmSeq includes a paired structure between treatments that incorporates subject-specific effects. To perform parameter estimation for the hmmSeq model, we develop an efficient Markov chain Monte Carlo algorithm. Further, we develop a procedure for detection of differentially expressed genes that automatically controls false discovery rate. A simulation study shows that the hmmSeq methodology performs better than competitors in terms of receiver operating characteristic curves. Finally, the analyses of three publicly available RNA-Seq datasets demonstrate the power and flexibility of the hmmSeq methodology. This dissertation also introduces an empirical Bayesian approach to detect differentially expressed genes in time course RNA-seq experiments. The proposed Bayesian method identifies major variation in gene expression profile by Bayesian principal component regression. The expression data are normalized for each gene, and the high dimentionality of time course data is first reduced by principal component analysis. The proposed model assumes a mixture distribution of expression parameters for differentially and nondifferentially expressed genes, borrows strength by sharing same variance across multiple subjects for each single gene, as well as shares information across genes by assuming gene-wise probabilities of being differentially expressed from the common beta prior distribution.