Biomedical (immunohistochemical) images of breast cancer are analyzed in this paper. Related work on automatic image segmentation is reviewed. The authors analyzed the architectures of convolutional neural networks of the U-net type for automatic segmentation of immunohistochemical images. Examples of neural network architectures that make it possible to receive more accurate and better image segmentation are given. A modified neural network architecture for segmentation of immunohistochemical images is developed. Computer experiments were carried out according to different numbers of epochs and iterations. ROC-curves are constructed to assess the quality of segmentation of known and modified network architectures of the U-net type.
Cancer ranks as the second or third cause of human mortality. Early diagnosis is the only way to prevent and timely treat cancer.
With the increasing growth of computer power, it has become possible to use modern information technologies such as artificial intelligence to diagnose diseases. Methods and tools of artificial intelligence are widely used in medicine. Traditional methods, such as knowledge engineering, are applied to present expertise in various fields of medicine. With the development of INTERNET technologies, a new field emerged, that is, telemedicine. Telemedicine makes it possible to attract experts from different countries remotely [1].
Breast cancer ranks as the first cause of women mortality in the world. Pathomorphological diagnosis is the main method of research and treatment.
The following biomedical images are used in oncology for diagnosis: cytological, histological and immunohistochemical. Therefore, computer vision methods and algorithms are used to process these images [2-4].
Computer vision methods and algorithms are used at different stages of image processing: preprocessing, segmentation, classification, etc. Segmentation is one of the key stages in the processing of biomedical images. The main purpose of segmentation is to clearly identify the nuclei of tissue cells in the image.
Neural networks are widely used in medicine [4]. Recently, U-net technology has been used increasingly, which makes it possible to train neural networks for a specific type of images. First, this technology was designed for use in medicine. Therefore, the development and training of a neural network for segmentation of immunohistochemical images is an urgent task.
The aim of this work is to conduct a comparative analysis of the quality of immunohistochemical image segmentation using existing and modified architectures of U-net type networks.
U-net technology is considered to be a modern approach to image segmentation in medicine according to the analyzed literature sources. The segmentation phase is very important, as it makes it possible to prepare the image for processing at a high level of computer vision, in particular classification and diagnosis. The objectives of this work are as follows.
1. Analyze immunohistochemical images of breast cancer. 2. Develop a modification of the neural network architecture of U-net type.
3. Compare different architectures of U-net type networks for segmentation of immunohistochemical images.
4. Analyze computer experiments.
Immunohistochemical examenations clarify the diagnosis, which is made on the basis of histological studies. The specified diagnosis is made with use of biomarkers. There is a microscopic description of immunohistochemical images of breast cancer [23].
A tumor was found in the breast tissue, the morphological structure of which corresponds to moderately differentiated invasive ductal carcinoma (G2) (Figure 1). Immunohistochemical images were obtained on the basis of immunohistochemical studies.
Estrogen receptor α (DAKO, clone EP1) - positive reaction in 59% of tumor cells (PS = 4), moderate intensity (IS = 2). TS = 4 + 2 = 6 - positive result (Figure 2).
Progesterone receptor (DAKO, clone PgR 636) - specific color is not deteсted. TS = 0 - negative result (Figure 3).
Oncoprotein c-erbB-2 / neu (HER-2 / neu) (DAKO, polyclonal) - specific color is not detected, 0 points - a negative result (Figure 4).
Ki-67 antigen (DAKO, clone MIB-1) -positive reaction of significant intensity in 44% of tumor cells (Figure 5).
Moderately differentiated (G2) invasive ductal carcinoma of the breast (code ICD-O code : 8500/3).According to the results of immunohistochemical staining of luminal type B tumor, HER-2 / neu negative: • estrogen (ER +) - sensitive (59% of cells ++); • progesterone (PR-) - negative; • HER-2 / neu - negative (0 points); • 44% of cells are positive (+++) for the marker of proliferative activity of Ki-67.
In [24] the authors present a generalized approach to diagnosis based on immunohistochemical images. One of the important stages of this approach is the segmentation stage for determining the relative area and average brightness of cell nuclei.
As it is known, U-net is a convolutional neural network, which was designed mainly for the segmentation of biomedical images. This architecture is based on a sequence of convolutional layers. The architecture of the U-network is characterized by certain features, in particular in the transmission of information in the downlink and uplink. Unlike other topologies of convolutional networks, this topology does not make use of fully connected layers, but only convolutional ones.
The basic architecture of the U-net network is shown in Figure 6. The structure of the U-net is described in more detail in [25].
This architecture consists of two main parts: narrowing (left) and expansion (right). The narrowing path is a typical convolutional neural network architecture and consists of a convolution operation and a RELu function. This reduces the dimensionality of the image. Each step in the expansion phase consists of operations that increase the discretization of the property map. The neural network is trained by the method of stochastic gradient descent, based on input immunohistochemical images and segmentation maps.
In the process of U-net learning it is necessary to determine the degree of similarity between the generated segmented image and the segmented person. To do this, the Dice coefficient is used.
A generalized approach to segmentation of immunohistochemical images is shown in Figure 7.
Learning process includes the following stages:
2. Convert an image to a grayscale image. 3. Adjust the structure and parameters of the neural network. 4. The learning process. 5. Save learning outcomes in a separate file.
Examples of U-nets used for segmentation of immunohistochemical images are shown in Figure 8. In this case, examples of the encoder architecture of the basic U-net neural network (a) and the modified neural network architecture (b) are given.
Unlike convolutional neural network for classification of images, the U-net type network does not have enough flexibility to modify the architecture. The U-net type network consists of descending and ascending parts that are interconnected. The modified architecture, which is shown in Figure 3 (b), has an additional convolution layer of 32 x 32 pixels. This modification was made to increase the accuracy of segmentation of immunohistochemical images by complicating the neural network architecture.
Conv 1024X1024 Conv 512X512 Conv 256X256 Conv 128X128
Conv 64X64
Conv 1024X1024 Conv 512X512 Conv 256X256 Conv 128X128 Conv 64X64 Conv 32X32
ROC curves are constructed to assess the quality of segmentation. Quantitative interpretation of ROC is done due to AUC indicator — the area bounded by the ROC curve and the axis of the share of false positive classifications. To determine the accuracy of the classification, the pixel value (black / white) on the resulting image and the corresponding value on the image processed by the expert are used.
Table 1 shows the results of the segmentation accuracy assessment for the U-net type network architecture (Type A).
These U-net network architectures are implemented on the basis of Python programming language, Tensorflow and Keras libraries using the Linux Mint operating system.
The image dataset consists of 150 immunohistochemical images, divided in the proportion of 70% (training sample) / 30% (test sample).
The comparison shows that the best result was obtained using the architecture of a neural network of type B with the following parameters: the number of iterations – 200, the number of epochs – 2. 8. Conclusions
1. The analysis of relevant work is carried out. It is shown that automatic segmentation is a relevant task. Automatic segmentation algorithms are analyzed; their advantages and disadvantages are highlighted. The necessity of using convolutional neural networks of U-net type is substantiated.
2. The histological image of breast cancer is analyzed and the diagnosis is specified on the basis of the analysis of immunohistochemical images.
3. The typical U-net architecture is analyzed and its modification is developed. U-net architectures are compared on the basis of immunohistochemical images.
4. Based on computer experiments, it was found that the best result was obtained using the architecture of the neural network of type B with the following parameters: the number of iterations – 200, the number of epochs – 2 and the accuracy of segmentation is 85%.
The results of automatic segmentation are used for preliminary diagnosis of cancer.
Informatics & Data-Driven Medicine (IDDM 2019), Lviv, Ukraine, 11-13 November 2019.
ceurws.org/Vol-2488/paper2.pdf
[4] Ivan Izonin, Roman Tkachenko, Ivanna Dronyuk, Pavlo Tkachenko, Michal Gregus, Mariia
Rashkevych. Predictive modeling based on small data in clinical medicine: RBF-based additive
input-doubling method[J]. Mathematical Biosciences and Engineering, 2021, 18(
Medical image analysis. 58, 101547 (2019). doi: 10.1016/j.media.2019.101547
[19] F. D. Khameneh, S. Razavi, & M. Kamasak, Automated segmentation of cell membranes to
evaluate HER2 status in whole slide images using a modified deep learning network, Computers
in biology and medicine. 110 (2019) 164–174. doi: 10.1016/j.compbiomed.2019.05.020
[20] F. Kromp et al., Evaluation of Deep Learning Architectures for Complex Immunofluorescence
Nuclear Image Segmentation, in: Proceedings of the IEEE Transactions on Medical Imaging. 40(7)
(July 2021) 1934–1949. doi: 10.1109/TMI.2021.3069558.
[21] L. B. Mahanta, E. Hussain, N. Das, L. Kakoti, & M. Chowdhury, IHC-Net: A fully convolutional
neural network for automated nuclear segmentation and ensemble classification for Allred scoring
in breast pathology, Applied Soft Computing. 103, 107136. (2021).
doi:10.1016/j.asoc.2021.107136
[22] A. Lagree, M. Mohebpour, N. Meti, K. Saednia, F. I. Lu, E. Slodkowska, W. T. Tran, A review
and comparison of breast tumor cell nuclei segmentation performances using deep convolutional
neural networks, Scientific Reports. 11(