In the article, the authors investigated modern automated microscopy systems used in oncology and analyzed recent methods of image processing at different levels of computer vision. The researchers described a class of biomedical images, such as cytological images of breast precancerous conditions. Literature review has shown that the actual problem is biomedical image processing. The adaptive method of biomedical pre-processing and automated segmentation method has been developed. The method of automated selection of segmentation algorithms based on the use of Gromov-Frechet metric, Gromov-Hausdorff metric, and knowledge base has been developed. Convolution neural network method for histological and cytological image classification based on a combination of convolutional and sub-sampling layers and their input parameters has been developed. Computer experiments have been carried out at each level of computer vision in automated microscopy systems. The generalized structure and components of the intelligent automated microscopy system for histological and cytological image processing have been developed.
The problem of improving diagnosis and choosing the best tactics for treatment of
breast tumors is still one of the most relevant in modern surgical practices [
Various benign tumors of the mammary gland, usually not pose a threat to
malignancy. However, specialists often face with the significant difficulties of differential
diagnosis of benign and malignant tumors of the mammary gland at the stage of primary
patient’s contact with health care facilities or while qualified and specialized medical
care is providing. For many years, the generally accepted view is that the only real way
to reduce morbidity and mortality frombreast cancer successfully is to improve the
quality of early diagnosis [
Histological study is a method of detailed study of the human body in order to detect
the pathological process in the tissue. The cytological study allows identifying the
pathological changes of cells on the early stages of development, since the main object
of study is small cellular structures, such as, nucleus, cytoplasm, mitochondria, as well
as determination of the nuclear-cytoplasmic ratio, which is a very important indicator.
The result of the cytological study is cytological images, and in histological study it is
histological images. Different methods of differential diagnosis of benign neoplasms
and breast cancer are used today. Morphometric methods where the number of
parenchymal cells, fibroblasts and fibrocytes localized periarteriolarly are the most popular
[
However, in spite of all diversity of methodological approaches in diagnosis and prediction of the development of breast cancer, the further development of cancer may occur often after the performed surgical interventions and treatment for verified benign processes. Thus, the search for new effective schemes for the prevention of adverse clinical manifestation in patients with benign breast pathology is still actual. 2
Histological and cytological images usually have low quality [
While carrying out a cytological research, the analysis of the shape of the
investigated micro-object is of great importance [
The main algorithms for micro-object edge detection are the following ones: snake algorithm, Canny algorithm, filtering algorithm based on Sobel, Laplacian, and Prewitt operators, etc. [9]. Their defining feature is highlighting the sharp brightness differences found in the border region. As a result, a set of unconnected areas appear. To obtain continuous contours, an additional processing is required.
The following algorithms for contour detection are known: threshold segmentation, clustering, watershed algorithms, block segmentation, etc. According to these algorithms, pixels are grouped into homogeneous regions on the basis of a certain homogeneity criterion. The result of their work is a set of homogeneous areas. To obtain the description of the object contour, it is necessary to apply the backward contour tracing [10].
Today, artificial neural networks are used as current image classifiers. In particular, convolutional neural networks (CNNs) are wide-known. CNN can detect image common features, form more complicated features, and recognize them. The objective of CNN is to alternate convolutional, sub-sampling and and max-pooling layers at the output. The authors in [11] investigated the use of CNNs for recognition of large-sized images. The focus was on the development of CNN with a small filter window size (3x3). The analysis of modern image processing systems, their advantages and disadvantages for image classification at the early stage of breast cancer detection were presented in paper [12]. The emphasis has been laid on the use of CNNs and deep learning technology. The article [13] presented the structure of CNN models used for researching and detecting breast cancer. The principles of applying CNNs for medical purposes were described in works [14, 15]. 3
Automated microscopy systems (AMSs) are widely used in biomedical image analysis. Automated microscopy system BioImageXD [16] covers both object-oriented and voxel-based approaches. The software can be used for both a simple visualization of multichannel temporary stacks and a complex 3D processing. The author of the article [17] described a method of computer processing of multispectral images of preparations used in medical biology research. The author [18] used hardware-software MetaMorph 7.1. in breast cancer research.
The authors of the article [19] analyzed the OncoDoc system to improve the quality of research for breast cancer diagnosis. OncoDoc has a database (DB) that distinguishes it from other systems. In the article [20], the authors examined and analyzed interactive software environment for visualization, correction, and analysis of 3D morphology.
In the work [21], he authors presented their analysis of studies in the field of immunohistochemistry of breast cancer using the ImageJ software based on the RGBmodel of histological images. In the article [22], the researchers showed the structure, advantages, and disadvantages of information automated system MECOS-CH. The most popular AMS include MECOS-C2, TissueFAXS, AnalySISFive, BioVision, VideoTesTMorpho 5.2, BioImageXD, Ariol, ImageJ, Motic Images Advanced 3.2, DiaMorph, Motic Video TesT Morfo 5.2, Cell D. The main AMS evaluation criteria are the following: operating modes for image segmentation algorithms, calculation of micro-objects’ numerical characteristics, and ability to work with external programs. Another important criterion is ability to output information, such as a report, in the form of charts and histograms. 4
Literature review has shown that the actual problem is biomedical image processing. Therefore, the purpose of the work is to develop new methods and tools for image processing at different levels of computer vision. 5
Biomedical images are images obtained by using visual imaging devices in medicine and biology [23]. Due to the visualization of various processes occurring in living organisms, it is possible to study the mechanisms of cells function, tissues and organs in humans and animals.
During the studies the following precancer pathologiesof the mammary gland such asnon proliferative mastopathy, cystic nonproliferative mastopathy, fibrotic nonproliferative mastopathy, proliferative mastopathy, and fibroadenoma have been investigated.
As a result of the research, the main morphological cytological signs of non-proliferative mastopathy were indicated, they include the following features: 1. Flattened apocrine epithelium. 2. Papillary structures formation. 3. Presence of secretory activity in cells. 4. Rounded hyperchromatic nuclei, located centrally. 5. A small number of hyperchromic monomorphic cells. 6. Cells are situated in layers. 7. Many phagocytes and histiocytesare in the background. 8. Presence of a secret around the cellular space. where ij is an element of the image.
The method of adaptive image processing consists of the following steps [24]:
Im = ( )
The expression for a two-dimensional median filtration can be represented as follows:
Im , =
[ + , + ( , ) ∈ ]; , ∈ 2
I where Imi, j is an image matrix element after the filtration;
Ws,t is an element of an aperture of an image with the size of m x n;
Imi, j is an input image matrix element. For filtration, the filter of 5x5 size is selected.
2. The next step is to quantify the image noisiness. To calculate the Peak Signal-toNoise Ratio (PSNR) [25], it is necessary to find out the mean square error (MSE) between two images:
MSE 1 m1 n1 2
ImI i, j Im i, j , mn i0 j0 where ImI and Im refer to a filtered and original images, respectively, the size is mxn. The PSNR value is defined as: (2) (3) PSNR 10 log10 MAX I2 ,
MSE where MAX I is a maximum value that is taken by the image pixel.
{ = 5 × 5, = 3 × 3, = 3 × 3; = 3 × 3; ≤ 24 > 24 where mw is a size of the median filter window, and gw is a size of the Gaussian filter window.
To reduce a level of additive noise, we apply the Gaussian filter. The transformation is represented as follows: The expression for the Gaussian filter convolution operation for the pixel with the coordinates x, y is the following:
Im =
× Im G x, y
ImIII mw* ImII , where σ is a radius of the convolution window.
Image filtering is represented as follows: where ImII is an input image, mw is a filter window, and ImIII is the image after filtering.
5. Histogram equalization.
Histogram equalization is represented by the transformation H:
Im
= (Im ) where ImIV is an image with a new value of the histogram, and ImIII is an input image.
Im = ∗ Im
To quantify the similarity of the image processed by known algorithms and the image processed by the expert, the criterion SSIM (structure similarity) was used. The difference between the two images A and B with the same size N × N is calculated by the formula:
SSIM ( A, B) (2
A B c1 )(2 AB c2 ) ( A2 B2 c1 )( A2 B2 c2 ) where
A B refer to average values A and B; AB is covariance A and B; 1 = ( 1 )2; 2 = ( 2 )2;
k1 = 0,01; k 2 = 0,03 are constants.
The comparative characteristics of the algorithms for automated image quality improvement by the SSIM criterion are shown in Table 1.
The graphical representation of the sequence of testing stages of histological and cytological image segmentation algorithms is shown in Figure 1.
The results of the automatic selection of histological and cytological image segmentation parameters are presented in Table 2. 8
For the classification of histological and cytological images the convolutional neural networks (CNN) were used [29].
The structure of the module for image classification is shown in Figure 2.
The comparative analysis of cytological and histological images classification methods is shown in Figure 3
Based on the developed image processing algorithms, "HIAMS" has been designed for diagnosis of breast cancer and precancerous conditions based on the analysis of cytological and histological images. System modules allow processing information about patients, processing images at different levels of computer vision. The key difference between the HIAMS intelligence system [30] and existing analogues is the availability of an adaptive graphical interface for different types of users and allocation of system access rights.
HIAMS structure. The generalized structure of the developed AMS is presented in Fig. 4. The main groups of system users are treating physician, diagnostic doctor, expert, assistant, and administrator. They communicate using a remote database and a remote FTP server. Currently, in medicine, scientists devote considerable attention to the design of databases for information systems that facilitate the work of physicians. The structure of such relational databases mostly makes it easy to formulate reports and statistical data on patients and their diagnoses. Most of the existing automated microscopy systems for image analysis do not have databases or they have a limited functionality.
A treating physician is responsible for the registration of patients. Every doctor has access only to his own patients. The results of image processing (preliminary processing, segmentation, etc.) are stored in the database. Each patient and each test has a unique identifier. The quantitative characteristics of microscopic objects in the image allow classifying them using known methods and algorithms, for example, by the method of support vectors. A diagnostic doctor makes a preliminary diagnosis based on image analysis. The doctor can form a set of quantitative features of microscopic objects and carry out a description of image qualitative characteristics. An expert can review the results of the research and make a diagnosis based on his or her own knowledge.
Conclusions
1. Analysis of the existing methods, algorithms, and recent tools for image processing at different levels of computer vision has been conducted.
2. Informative features of breast pre-cancerous conditions have been described. 3. The method of automated selection of segmentation algorithms based on the use of Gromov-Frechet metric, Gromov-Hausdorff metric, and knowledge base has been developed, which enabled to automatically select segmentation algorithms and their parameters.
4. The method of adaptive image processing based on filtering algorithms and rules of histogram alignment has been developed and it improves the quality of histological and cytological images by 16%.
5. Neural network method for histological and cytological image classification based on a combination of convolutional and sub-sampling layers and their input parameters has been developed, which allowed increasing the classification accuracy in comparison with the existing classifiers (SVM, k-means) by an average of 20%; 6. The generalized structure and components of the intelligent automated microscopy system for histological and cytological image processing have been developed, which unlike the existing analogs has adaptive graphical interface, remoted database, knowledge base of algorithms for image preliminary processing, segmentation and classification and it is a multi-user software system.
At present, the most precise method in the world practice of diagnosing pathological processes in oncology is the immunohistochemical method. Therefore, the very urgent first area for further research is the formation of a database of immunohistochemical images of breast dysplastic and cancerous conditions.
The second direction is the expansion of the database of histological and cytological images with new types of breast dysplastic and cancerous conditions, the search for new informational features for diagnosis. This direction requires the involvement of expert cytologists and histologists.
The third direction is the intellectual analysis of the quantitative and qualitative characteristics of the database to identify the regularities. This will allow obtaining new diagnostic rules, test them and discover new features. 9. Pratt, W. K.: Digital Image Processing: PIKS Scientific Inside. NY, USA: John Wiley & Sons,
Inc., 782 p. (2007) 10. Kazlouski, A., Sadykhov, R.: Plain objects detection in image based on a contour tracing algorithm in a binary image. In: 2014 IEEE International Symposium on Innovations in Intelligent Systems and Applications (INISTA) Proceedings, 23–25 June 2014. pp. 242–248 (2014) 11. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large–Scale Image
Recognition. In: Proceedings of ICLR, May 7 – 9, 2015, 1–14 (2015) 12. Abdelhafiz, D., et. al.: Survey on deep convolutional neural networks in mammography. In: Conference: 2017 IEEE 7th International Conference on Computational Advances in Bio and Medical SciencesAt: Orlando, FL, USA, 1–17 (2017) 13. Zhang, X., et. al: Whole mammogram image classification with convolutional neural networks. In: IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (2017) Kansas City, MO, USA Nov. 13, 2017 to Nov. 16, 700–704 (2017) 14. Qing, Li, et. al: Medical image classification with convolutional neural network. In: 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV) – Singapore, 10–12 Dec. 2014, 34-42 (2014) 15. Khoshdeli, M.: Feature–based Representation Improves Color Decomposition and Nuclear Detection Using Convolutional Neural Network. IEEE Transactions on Biomedical Engineering, (2017) 16. Kankaanpää, P., et. al: BioImageXD: an open, general–purpose and high–throughput image– processing platform. Nature methods, Vol. 9(7), 155–171 (2012) 17. Malov, A. M.: Computer processing of biomedical multichannel images using visualization of measures of similarity with the standard. Izvestiya VUzov. Instrument making, No. 52 (8), 74-79 (2009) 18. Yokoyama, Y., et. al: Loss of histone H4K20 trimethylation predicts poor prognosis in breast cancer and is associated with invasive activity. Breast Cancer Research, Vol. 16 (3), 66 (2014) 19. Seroussi, B.: Using OncoDoc as a computer–based eligibility screening system to improve accrual onto breast cancer clinical trials. Artificial Intelligence in Medicine, Vol. 29, Issues 1–2, 153–167 (2002) 20. Dercksen, V.: The Filament Editor: An Interactive Software Environment for Visualization, Proof–Editing and Analysis of 3D Neuron Morphology. Neuroinformatics, Vol. 12(2), 325– 339 (2014) 21. Vrekoussis, T., et. al.: Image Analysis of Breast Cancer Immunohistochemistry–stained Sections Using ImageJ: An RGB–based Model. Anticancer Research December, Vol. 29(12), 4995–4998 (2009) 22. Medovy, V. S.: Informational automated microscopy systems for the analysis of biomaterials.
Doctor and information technology, No. 6, 32-37 (2004) 23. Berezky, O. M., et. al.: Methods, algorithms and software tools for the processing of biomedical images. Ternopil: Economic Thought, TNEU, 330 p (2017) 24. Pitsun, O.Y.: Adaptive method for processing histological and cytological images. The Bulletin of the National University "Lviv Polytechnic". Computer Science and Information Technology, No. 864, 111-119 (2014) 25. Wang, Z.: Image quality assessment: From error visibility to structural similarity. In: Processing of the IEEE Transactions on Image Processing, Vol. 13, No. 4, 600–612 (2004) 26. Berezky, O.M: Adaptive method of image segmentation based on metrics. Scientific Bulletin of NLTU of Ukraine: a collection of scientific and technical works. Lviv: RVB NLTU of Ukraine, No. 28 (3), 110-123 (2018) 27. Berezsky, O.: Fréchet distance between weighted rooted trees. Matematychni Studii, Vol. 48,
No.2, 165-170 (2017) 28. Berezky, O. M, et. al.:. Development of metrics and methods of quantitative estimation of biomedical images segmentation. East-European Journal of Advanced Technologies, Vol. 6, No. 4, 4-11 (2017) 29. Berezsky, O., et. al.: Computer diagnostic tools based on biomedical image analysis. In: Proceedings of the 14th International Conference “The Experience of Designing and Application of CAD Systems in Microelectronics” (CADSM), Polyana–Svalyava, 388–391 (2017) 30. Certificate of registration of copyright for work №75360. Computer program "Intelligent system of diagnosis of precancerous conditions of the musculoskeletal system based on the analysis of histological and cytological images" HIAMS ". / O.M. Berezky, O.Y. Pitsun, G.M Melnik, P.B. Lyashinsky, P.B., Lyashinsky. Date of registration 14.12.2017.