Deep learning based nuclei detection for quantitative histopathology image analysis

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Quantitative analysis of histopathology images is important for both clinical purposes (e.g. to reduce/eliminate inter- and intra-observer variations in diagnosis) and research purposes (e.g. to understand the biological mechanisms of the disease process). Quantification and study of spatial and morphological patterns of cells in images of histopathological specimens are of particular importance, since they provide useful information for evaluating cancer progression and prognosis. Accurate detection of nuclei is the first step towards that end, but offers challenges due to large variations in size, shape, density, and batch variations. This thesis proposed two deep learning frameworks to detect nuclei in images of Hematoxylin and Eosin (H&E) stained tissue specimens. Both frameworks learn multi-scale features through sequence of convolution and pooling layers. The first framework formulates the nucleus detection problem as a discrete classification problem and uses convolutional neural networks (CNN) to classify image patches as nucleus versus background. The second framework formulates the problem as a continuous regression problem and builds a fully convolutional regression network to learn a continuous mapping from image patches centered around nucleus centroids to nuclear distance maps. The trained network produces an equivalent of probability density functions of centroids whose local maxima locate individual nuclei even within a cluster of multiple nuclei. The proposed networks are trained on a publicly available breast cancer dataset and are tested on the same dataset, and two additional datasets (colorectal adenocarcinoma and human bone marrow) without further re-training. Experimental results show superior performance compared to state-of-the-art methods. The detection results from proposed networks are further processed with spatial pattern analysis methods to quantitatively describe spatial organization of nuclei within the processed tissue samples.