Algorithms for Color Normalization and Segmentation of Liver Cancer Histopathology Images

Abstract

With the advent of Computer Assisted Diagnosis (CAD), accuracy of cancer detection from histopathology images is significantly increased. However, color variation in CAD system is inevitable due to variability of stain concentration and manual tissue sectioning. Small variation in color may lead to misclassification of cancer cells. Therefore, color normalization is the first step of Computer Assisted Diagnosis (CAD), in order to reduce the inter-variability of background color among a set of source images. In this thesis, first a novel color normalization method is proposed for Hematoxylin and Eosin (H and E) stained histopathology images. Conventional Reinhard algorithm is modified in our proposed method by incorporating fuzzy logic. Moreover, mathematically it is proved that our proposed method satisfies all three hypotheses of color normalization. Furthermore, several quality metrics are estimated locally for evaluating the performance of various color normalization methods. Experimental result reveals that our proposed method has outperformed all other benchmark methods. The second step of CAD is nuclei segmentation which is the most significant step since it enables the classification task computationally efficient and simple. However, automatic nuclei detection is fraught with problems due to highly textured nuclei boundary and various size and shapes of nuclei present in histopathology images. In this thesis, a novel edge detection technique is proposed for segmenting the nuclei regions in liver cancer Hematoxylin and Eosin (H and E) stained histopathology images, based on the notion of computing local standard deviation value. Moreover, the edge-detected image is converted into a binary image by using local Otsu thresholding and thereafter, it is refined by an adaptive morphological filter. The experimental result indicates that proposed segmentation method overcomes the limitations of existing unsupervised methods and subsequently its performance is also comparable with deep neural models. To the best of our knowledge, our proposed method is the only unsupervised method iii which achieves nuclei detection accuracy closest to 1 (0.9516). Furthermore, two more quality metrics are computed in order to measure the performance of nuclei segmentation methods quantitatively. The mean value of quality metrics reveals that our proposed segmentation method outperforms other existing methods both qualitatively and quantitatively.