Cancer ranks as the second or third cause of human mortality. Early diagnosis is the only way to prevent and timely treat cancer.
With the increasing growth of computer power, it has become possible to use modern information technologies such as artificial intelligence to diagnose diseases. Methods and tools of artificial intelligence are widely used in medicine. Traditional methods, such as knowledge engineering, are applied to present expertise in various fields of medicine. With the development of INTERNET technologies, a new field emerged, that is, telemedicine. Telemedicine makes it possible to attract experts from different countries remotely [1]. Breast cancer ranks as the first cause of women mortality in the world. Pathomorphological diagnosis is the main method of research and treatment.
The following biomedical images are used in oncology for diagnosis: cytological, histological and immunohistochemical. Therefore, computer vision methods and algorithms are used to process these images [2][3][4].
Computer vision methods and algorithms are used at different stages of image processing: preprocessing, segmentation, classification, etc. Segmentation is one of the key stages in the processing of biomedical images. The main purpose of segmentation is to clearly identify the nuclei of tissue cells in the image.
Neural networks are widely used in medicine [4]. Recently, U-net technology has been used increasingly, which makes it possible to train neural networks for a specific type of images. First, this technology was designed for use in medicine. Therefore, the development and training of a neural network for segmentation of immunohistochemical images is an urgent task.
The aim of this work is to conduct a comparative analysis of the quality of immunohistochemical image segmentation using existing and modified architectures of U-net type networks.
Let us review some new articles on automatic image segmentation.
Manual and automatic segmentation is investagated, its advantages and disadvantages are highlighted in paper [5]. The study was conducted with the involvement of experts in the field of histology and pathology. The method of automatic segmentation is based on the threshold color. The developed method makes it possible to segment positive and negative cells based on the Ki67 biomarker with an average accuracy of 90%.
Article [6] presents a new segmentation and nuclei counting method that can automatically count nuclei using a modified single-pass super pixel segmentation method. This method was tested on a large sample of immunohistochemical images. Experiments showed sufficient accuracy and speed.
The authors in paper [7] described the automatic analysis technique of images of oral cancer tissue areas stained with immunoglobulin P53. Tissue images are segmented by the entropy threshold, and cluster cells are found by the watershed method applied selectively. For this purpose, the color indices of each nucleus and the method of reference vectors are used. This is followed by a classification assessment.
Article [8] presents a method of segmentation and classification of medical images based on a hierarchical transformation and a cascade model of nonlinear mappings. The authors offered to use the proposed method for classifying cell nuclei.
Mostly, all cancer cells are studied by diagnosticians, who are undoubtedly highly qualified specialists, but such an assessment is a subjective one. An attempt was made in paper [9] to solve the problem of subjectivity of assessment on the example of immunohistochemical images based on the biomarker KI-67. The result is an integrated, universal method of automatic cancer diagnosis based on immunohistochemical images.
The basis for assessing breast cancer is the tumor's response to a particular type of protein. An effective algorithm for automatic detection of carcinoma cells was presented in article [10]. This formed the basis of automatic segmentation and classification of cancer using biomarkers.
The combination of histological and immunohistochemical images allows one to make an accurate diagnosis. Immunohistochemical images based on biomarkers were classified using neural networks DCNN and VGG. New machine learning strategies were applied to train these networks in [11].
The methods of automatic detection of cancer cells are analyzed in paper [12]. The developed algorithm selected sets of images that contain ganglion cells. These images were then examined and evaluated by five pathologists.
The authors of paper [13] developed accurate diagnosis methods of lymphoma using machine learning methods.
Semi-automatic histopathological analysis of adenocarcinoma tumor cells is presented in article [14].
Article [15] presented the developed U-net convolutional neural network for segmentation of nuclei on a public data set. A recurrent residual U-network (R2U-Net) was proposed, which demonstrated high performance on different sets of medical images (retinal blood vessels, skin cancer, and lung segmentation) in segmentation tasks.
Scientists developed and trained U-net to segment immunohistochemical images previously coloured. This network was then applied to the raw slides to create templates [16].
The proposed approach is based on CNN and applies for pixel-by-pixel segmentation of information areas. This became possible with the parallel use of GPU processors [17].
The results presented in the article can be beneficial for clinical use. The authors tested the effectiveness of four methods based on deep learning. Testing was performed on different areas of the slide image: normal tissues, with clusters of immune cells and artifacts [18].
F. Mahmood, R. Chen, D. Borders and others proposed to supplement the existing data sets of medical images using CGAN. Competitive U-Net with spectral normalization was then used to increase learning stability. This paired network not only generates new images, but also looks for the optimal loss function.
Paper [19] presents machine learning methods for effective segmentation, classification and quantification of breast cancer images. It is shown that the presented algorithm is better than the superpixel classification, which is based on CNN.
Paper [20] compares the efficiency of segmentation of several architectures (U-Net, Mask R-CNN, Cellpose, U-Net ResNet) and traditional segmentation algorithms on the example of fluorescent images of cell nuclei.
A new method of segmentation was developed in paper [21], based on a deep learning network. It is aimed at accurate selection of the stained area of the nuclei of breast tissue images. Morphological post-processing of segmented micro-objects splits overlapping nuclei. To improve the results of a separate classifier, an ensemble method is used, which integrates the solutions of three models of machine learning for the final assessment of cancer.
The researchers developed a tandem of an artificial neural network and U-Net for recognising and generating interference. In article, the developed network achieves better results compared to the semantic segmentation of deep neural networks.
The authors of this paper analyzed segmentation methods on different types of neural networks. For conducting research, deep convolutional networks -U-Net, Mask R-CNN and the developed network (GB U-Net) are compared [22].
Thus, based on the analysis of literature review, the following approaches to the segmentation of biomedical images are identified:
1. Traditional methods of segmentation.
2. Application of CNN and deep learning.
3. The use of U-Net type networks.
U-net technology is considered to be a modern approach to image segmentation in medicine according to the analyzed literature sources. The segmentation phase is very important, as it makes it possible to prepare the image for processing at a high level of computer vision, in particular classification and diagnosis. The objectives of this work are as follows.
Immunohistochemical examenations clarify the diagnosis, which is made on the basis of histological studies. The specified diagnosis is made with use of biomarkers. There is a microscopic description of immunohistochemical images of breast cancer [23].
A tumor was found in the breast tissue, the morphological structure of which corresponds to moderately differentiated invasive ductal carcinoma (G2) (Figure 1). • estrogen (ER +) -sensitive (59% of cells ++);
• progesterone (PR-) -negative;
• HER-2 / neu -negative (0 points); • 44% of cells are positive (+++) for the marker of proliferative activity of Ki-67.
In [24] the authors present a generalized approach to diagnosis based on immunohistochemical images. One of the important stages of this approach is the segmentation stage for determining the relative area and average brightness of cell nuclei.
As it is known, U-net is a convolutional neural network, which was designed mainly for the segmentation of biomedical images. This architecture is based on a sequence of convolutional layers.
The architecture of the U-network is characterized by certain features, in particular in the transmission of information in the downlink and uplink. Unlike other topologies of convolutional networks, this topology does not make use of fully connected layers, but only convolutional ones.
The basic architecture of the U-net network is shown in Figure 6. The structure of the U-net is described in more detail in [25]. This architecture consists of two main parts: narrowing (left) and expansion (right). The narrowing path is a typical convolutional neural network architecture and consists of a convolution operation and a RELu function. This reduces the dimensionality of the image. Each step in the expansion phase consists of operations that increase the discretization of the property map. The neural network is trained by the method of stochastic gradient descent, based on input immunohistochemical images and segmentation maps.
In the process of U-net learning it is necessary to determine the degree of similarity between the generated segmented image and the segmented person. To do this, the Dice coefficient is used.
A generalized approach to segmentation of immunohistochemical images is shown in Figure 7.
Learning process includes the following stages:
1. Download images into memory for training sampling.
2. Convert an image to a grayscale image.
3. Adjust the structure and parameters of the neural network. 4. The learning process. 5. Save learning outcomes in a separate file.
The testing phase is as follows:
1. Download the test sample.
2. Download a file with a trained network.
3. The segmentation process.
Examples of U-nets used for segmentation of immunohistochemical images are shown in
The database of images was used for computer experiments [26].
ROC curves are constructed to assess the quality of segmentation. Quantitative interpretation of ROC is done due to AUC indicatorthe area bounded by the ROC curve and the axis of the share of false positive classifications. To determine the accuracy of the classification, the value (black / white) on the resulting image and the corresponding value on the image processed by the expert are used.
Table 1 shows the results of the segmentation accuracy assessment for the U-net type network architecture (Type A). Table 2 shows the results of the segmentation accuracy assessment for the U-net type network architecture (Type B). These U-net network architectures are implemented on the basis of Python programming language, Tensorflow and Keras libraries using the Linux Mint operating system.
The image dataset consists of 150 immunohistochemical images, divided in the proportion of 70% (training sample) / 30% (test sample).
The comparison shows that the best result was obtained using the architecture of a neural network of type B with the following parameters: the number of iterations -200, the number of epochs -2.