Complex Automatic Evaluation of the Medical Images of the Paranasal Sinuses Radiy Radutnya,, AlinaNechyporenkob,c, Victoriia Alekseevad,e, Nadiia Yurevychd, Andrii Lupyrd and Vitaliy Gargind a National Aviation University of Ukraine, 1, Liubomyra Huzara Avenue, Kyiv, 03058, Ukraine b Technical University of Applied Sciences, 1Hochschulring, Wildau,, 115745, Germany c Kharkiv National University of Radio Electronics, Nauky Avenue 14, Kharkiv, 61166, Ukraine d Kharkiv National Medical University, 4 Nauky Avenue, Kharkiv, 61022, Ukraine e Kharkiv International Medical University, 38, Molochna Str., Kharkiv61001, Ukraine Abstract Evaluation of medical images is of key importance in the work of medical staff today. Especially this problem pays important role in otolaryngology. The aim of our work was to develop an automatic complex method for assessing the state of the human paranasal sinuses. Our research included 10 people of different sex and age, who were divided into groups, taking into account the recommendations of the WHO for 2019-2021. The structure of the mucous membrane of the maxillary sinus were calculated and compared. In the course of our research, an algorithm was developed for the automatic assessment of the state of the mucous membrane of the maxillary sinus and its bone walls according to the data of the spiral computed tomography. The difference between obtained data in the manual and automatic mode is minimal. Keywords 1 Automated measurement, Computed Tomography, Maxillary Sinus, Bone Thickness, Bone Density 1. Introduction Evaluation of medical images is of key importance in the work of medical staff today [1, 2]. In connection with the daily increasing load, the issue of automating the process is especially relevant, which would allow to analyze the images accurately, informatively and fastly [3, 4]. One of the areas of application of this technique can be the automation of the processing of histological images. Considering the modern literature data, there are no systems that would analyze images. We converted pathological images into electronic format and analyzed the data by grouping pixels into distinct segments (called objects) using pixel intensity / color information or image texture. [5, 6]. There are no simple and available systems for image analysis till now. This method of the processing of the histological specimens in otolaryngology is of particular importance, in particular, when assessing the state of the mucous membrane of the human paranasal sinuses [9, 10]. This problem is especially acute, considering the progressively increasing number of the inflammatory processes in the paranasal sinuses both in Ukraine and around the world [11]. We should also be taken into account that biopsy is not a routine method in otolaryngology [12]. Often, it is performed only in exceptional cases if we suspect the presence of malignant neoplasms and after surgery, although biopsy is of key importance in some cases for the final diagnosis. Proccedings Name, Month XX–XX, YYYY, City, Country EMAIL: radiy@yahoo.com; alinanechiporenko@gmail.com; vik13052130@i.ua; urevichi@ukr.net; lupyr_ent@ukr.net, vitgarg@ukr.net ORCID: 0000-0001-6339-5909; 0000-0002-4501-7426; 0000-0001-5272-8704; 0000-0001-7340-2850; 0000-0002-9896-163X; 0000-0001- 8194-4019 © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) More often, instrumental methods are used for diagnostic purposes, for example, a computed tomography (CT) [13, 14], which is easy to perform, non-invasive and gives reliable data. Often, both methods, a computed tomography and histological examination, can complement each other, thereby increasing the information content of the obtained results. Considering all of the above, the purpose of our work was to design an automatic complex method for assessing the state of the human paranasal sinuses. 2. Material and Methods Our research included 10 people of different sex and age, who were divided into groups, taking into account the recommendations of the WHO for 2019-2021 (see Table 1). The selection criterion for the examined persons was the presence of an unilateral cyst (see Fig. 1) of the maxillary sinus, which was subsequently removed endoscopically. During the removal of the cyst and the enlargement of the natural anastomosis, samples of the mucous membrane of the maxillary sinus were taken (see Fig. 2). Table 1 Distribution of the examined persons by sex and age Age 25-44 45-60 61-75 Male 1 2 3 Female 2 1 1 Total 3 3 4 Figure 1: CT scan of the maxillary sinuses with a cyst in the right one (axial projection) Figure 2: Samples of the examined mucous membrane of the paranasal sinuses The research was carried out on a spiral computed tomography (SCT) Toshiba Aquilion 4 (Japan), which is distinguished by the reliability of results and ease of use. The Asterion Super 4 System is a multi-slice spiral CT scanner for the whole body. This system generates 4 slices in one 0.75 second revolution using a multi-row detector with selectable thickness using. Staining with hematoxylin and eosin is most widespread method for investigation in histology, pathologic anatomy, biology. Nuclei are stained in blue Staining with hematoxylin, cellular cytoplasm is stained in pink-red with eosin as result [16]. Figure 3: The investigated parameters of the structure of the nasal mucosa (black line - the thickness of the epithelium with the basement membrane, yellow arrow - the cell nucleus under the basement membrane, blue - above the basement membrane, green - basement membrane) This method allows you to identify the cellular composition of the test sample accurately and informatively, to clarify the features of the location of cells and the size of the nuclei. In the course of the work, attention was paid to such parameters as the thickness of the basement membrane, the quantitative composition of the cells above and below the basement membrane, as the main markers of the inflammatory processes in the tissue (Fig. 3). Automatic evaluation of histological slides is a challenge for research both in relation to classical staining methods and in relation to the created ones. Immunohistochemical staining (IHC) allows selective identification of antigens (proteins) in cells and tissues using the principle of binding of specific antibodies to antigens and applying an additional dye [17]. The use of horseradish peroxidase is most often used to visualize antibody-antigen interaction; in this case, this staining effect can obtain using various methods in which the antibody is conjugated to the enzyme. The location of the stained structures can be membranous, vascular, nuclear, cytoplasmic depending on the localization of receptors Our study is one of the first attempts to automatically assess such staining. The obtained micropreparations after all stages of preparation were studied using an Olympus BX- 41 ”microscope with objectives x4, x100, x200, x400. The results were further processed by the Olympus DP-soft version 3.2 software, which allows to interpret the obtained data after the morphometric study accurately and informatively. Also, a histological examination of the state of the ciliated epithelium of the mucous membrane of the paranasal sinuses was carried out. In order to determine the accuracy of the method a calculation in a manual mode was carried out after carrying out the calculations in an automatic way for the correction and comparing the results. An algorithm of the automated measurement was described in our previous works [18-20]. 3. Results and Discussion During the work, the data of the studied signs of the structure of the mucous membrane were obtained both in manual and automatic modes (see Fig. 4 and 5). The results of the measurements are presented in Table 2. Table 2 The research results of indicators of the structure of the mucous membrane of the maxillary sinus by the manual and automatic mode Number Basement membrane A number of cells above A number of cells thickness, px the basement below basement membrane membrane Manu Automa Manu Automa Manu Automa ally tically ally tically ally tically 1 27 17 200 123 66 55 2 24 14 201 120 67 54 3 23 13 99 119 45 59 4 28 18 206 102 49 46 5 24 15 153 114 68 51 6 22 14 176 145 69 53 7 20 13 187 172 70 52 8 20 11 154 134 71 51 9 18 19 139 155 75 62 10 21 17 112 198 65 64 Figure 4: An example of automatic measurement of the thickness (px) of various areas of the mucous membrane of the paranasal sinuses. At the subsequent stages, automatic and manual measurements of the thickness and the density of the bone of the upper and lower walls of the maxillary sinus were carried out, as potentially dangerous for the development of the complications, according to the data of the spiral computed tomography. All stages of automatic image evaluation have been presented in our previous works. An example of determining the thickness of the skull bones in automatic and manual modes is shown in Fig. 6 A. At the subsequent stages, automatic and manual measurements of the thickness and the density of the bone of the upper and lower walls of the maxillary sinus were carried out, as potentially dangerous for the development of the complications, according to the data of the spiral computed tomography. All stages of automatic image evaluation have been presented in our previous works. An example of determining the thickness of the skull bones in automatic and manual modes is shown in Fig. 6 B. Figure 5: An example of automatic measurement of the number of cells below the basement membrane A B Figure 6: А - complex measurement of the thickness and density by the automatic method. B - an example of the manual measurement of the minimum density of the lower wall of the maxillary sinus. Thus, in the course of the research, the indicators of the structure of the mucous membrane of the maxillary sinus were calculated and compared. It should be noted that today there are only a rather small number of such works, which, in contrast to ours, are mainly devoted to studies on animals or the penetration of the mucous membrane with various medicines. When carrying out the study, we solved such important medical problems as identifying the signs that could serve as a marker of inflammatory changes in the sinus, namely: the quantitative composition of cells in the space under the basement membrane, the thickness of the basement membrane itself. Attention was also paid to bone thickness and density according to the SCT data as the main markers of destructive changes in the walls of the sinus [21]. The obtained results are presented in Table 3. Table 3 Research results of the thickness and density indicators on the example of the lower wall of the maxillary sinus by the manual and automatic model Number Density, Hu Thickness, mm Manu Automa Manu Automa ally tically ally tically 1 103 97 8 8,4 2 155 112 10 9,21 3 231 154 12 11,3 4 122 176 10 10,2 5 241 117 7 8,1 6 124 111 9 9,5 7 98 79 12 12,1 8 103 154 12 11,96 9 175 116 9 9,9 10 122 111 11 11,4 The research included the upper wall of the maxillary sinus due to its potential danger in terms of the development of intraorbital complications and the lower wall of the maxillary sinus in connection with the development through it the odontogenic maxillary sinusitis. Thus, it can be assumed that the work carried out is one of the first ones related to this medical topic [22]. The method of the automatic evaluation of the images deserves special attention. It is of particular importance since the workload on the medical staff [23] increases every day, which in turn can significantly affect the quality, accuracy and reliability of the obtained results and the correct interpretation of the data. It is known, the method of the automatic image assessment is currently used in various branches of medicine [24, 25]. Quite often it is also used in dentistry [26] or in otolaryngology, most often for the planning sugery, measuring the size of sinuses. In the presented research, an algorithm, for solving such a difficult task as the measuring density according to the SCT data, is shown, which is undoubtedly one of the key indicators of the structure of the bone tissue of the paranasal sinusesʾ walls. It is also interesting that there is a slight difference in the calculation of parameters in manual and automatic modes, which allows us to make an assumption about the reliability of our proposed method. The most variable parameter with the most significant difference in results of automatic and calculation was bone density. This difference may be connected with difficulties in the calculation process. This method of automatic calculation of medical parameters is promising nowadays and may be used in different fields of medicine [27-29] 4. Conclusions Thus, in the course of our research, an algorithm was developed for the automatic assessment of the state of the mucous membrane of the maxillary sinus and its bone walls according to the data of the spiral computed tomography. The difference between obtained data in the manual and automatic mode is minimal. The obtained data shows minimal average related error in the most of cases in the automatic detection of the structure of mucosa of Maxillary sinus as well as in the structure of density and thickness of its walls. The most variable parameter with the biggest difference in results the manual and automatic calculation was bone density. 5. References [1] S. Krivenko, S. Abramov, V. Lukin, B. Vozel, K. Chehdi, "Lossy DCT-based compression of remote sensing images with providing a desired visual quality," Proc. SPIE 11155, Image and Signal Processing for Remote Sensing XXV, 2019. [2] S.S.ͦKrivenko, S.A.ͦKrivenko, "Many-to-many linear-feedback shift register," in Electronics and Nanotechnology (ELNANO), 2014 IEEE 34th International Conference on, Kyiv, Ukraine, pp. 176-178, 2014. [3] Grue et al., "Normal ranges for automatic measurements of tissue Doppler indices of mitral annular motion by echocardiography. Data from the HUNT3 Study", Echocardiography, vol. 36, no. 9, pp. 1646-1655, 2019. DOI: 10.1111/echo.14476. [4] Y. Kao et al., "Automatic measurements of arterial input and venous output functions on cerebral computed tomography perfusion images: A preliminary study", Computers in Biology and Medicine, vol. 51, pp. 51-60, 2014. DOI: 10.1016/j.compbiomed.2014.04.015 [Accessed 17 October 2021]. [5] P. Bant et al., "Tonsillectomy in own material", OtolaryngologiaPolska, vol. 74, no. 6, pp. 29-34, 2020. DOI: 10.5604/01.3001.0014.1612. [6] V. Gargin, R. Radutny, G. Titova, D. Bibik, A. Kirichenko and O. Bazhenov, "Application of the computer vision system for evaluation of pathomorphological images", 2020 IEEE 40th International Conference on Electronics and Nanotechnology (ELNANO), 2020. DOI: 10.1109/elnano50318.2020.9088898. [7] E. Akagunduz, A. Bors and K. Evans, "Defining Image Memorability Using the Visual Memory Schema", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 9, pp. 2165-2178, 2020. DOI: 10.1109/tpami.2019.2914392. [8] L. Goetschalckx, P. Moors and J. Wagemans, "Image memorability across longer time intervals", Memory, vol. 26, no. 5, pp. 581-588, 2017. DOI: 10.1080/09658211.2017.1383435. [9] O. Shpachynskyi, V. Didkovskij and A. Kopchak, "Radiological changes in maxillary sinusmorphology after lateral sinus flooraugmentation", OtolaryngologiaPolska, vol. 75, no. 2, pp. 34-41, 2020. DOI: 10.5604/01.3001.0014.1679. [10] Z. Guo, Y. Liu, L. Qin, Y. Song, C. Xie and D. Li, "Longitudinal response of membrane thickness and ostium patency following sinus floor elevation: a prospective cohort study", Clinical Oral Implants Research, vol. 27, no. 6, pp. 724-729, 2015. DOI: 10.1111/clr.12655. [11] I. Zupanets, T. Zhulai, S. Shebeko, I. Otrishko and D. Hladkykh, "Histomorphological Study of a New Nasal Spray with Anti-inflammatory Properties Efficacy in Rabbits with Rhinosinusitis", Medical Archives, vol. 74, no. 1, p. 8, 2020. DOI: 10.5455/medarh.2020.74.8-13 [12] A. Saraswathula, J. Lee and U. Megwalu, "Patient preferences regarding the communication of biopsy results in the general otolaryngology clinic", American Journal of Otolaryngology, vol. 40, no. 1, pp. 83-88, 2019. DOI: 10.1016/j.amjoto.2018.10.002. [13] S. Becker and B. O’Malley, "Evaluation of sinus computed tomography scans", Current Opinion in Otolaryngology & Head and Neck Surgery, vol. 21, no. 1, pp. 69-73, 2013. DOI: 10.1097/moo.0b013e32835b09b7. [14] Y. Polyvianna, D. Chumachenko and T. Chumachenko, "Computer Aided System of Time Series Analysis Methods for Forecasting the Epidemics Outbreaks", 2019 IEEE 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM), 2019. DOI: 10.1109/cadsm.2019.8779344. [15] H. Bauknecht et al., "Impact of image noise levels, scout scan dose and lens shield on image quality and radiation exposure in z-axis dose-modulated neck MSCT on 16- and 64-slice Toshiba Aquilion scanners", European Journal of Radiology, vol. 73, no. 2, pp. 439-443, 2010. DOI: 10.1016/j.ejrad.2008.11.015. [16] S. Försch, F. Klauschen, P. Hufnagl and W. Roth, "Artificial intelligence in pathology", DeutschesAerzteblatt Online, 2021. DOI: 10.3238/arztebl.m2021.0011. [17] R.S. Nazaryan, L.S. Kryvenko, V.V. Gargin, “The role of nitric oxide synthase in the modulation of the immune response in atopic disease”, New Armenian Med J, vol.11, no 2, pp. 52-57, 2017 [18] R. Radutniy, A. Nechyporenko, V. Alekseeva, G. Titova, D. Bibik and V. Gargin, "Automated Measurement of Bone Thickness on SCT Sections and Other Images", 2020 IEEE Third International Conference on Data Stream Mining & Processing (DSMP), 2020. DOI: 10.1109/dsmp47368.2020.9204289. [19] V. Gargin et al., "Relationship between bone density of paranasal sinuses and adrenal steroids pattern in women during menopausal transition", Anthropological Review, vol. 83, no. 4, pp. 407- 418, 2020. DOI: 10.2478/anre-2020-0031. [20] A. Nechyporenko, V. Reshetnik, V. Alekseeva, N. Yurevych, R. Nazaryan and V. Gargin, "Implementation and analysis of uncertainty of measurement results for lower walls of maxillary and frontal sinuses", 2020 IEEE 40th International Conference on Electronics and Nanotechnology (ELNANO), 2020. DOI: 10.1109/elnano50318.2020.9088916. [21] S. Suttapreyasri, P. Suapear and N. Leepong, "The Accuracy of Cone-Beam Computed Tomography for Evaluating Bone Density and Cortical Bone Thickness at the Implant Site: Micro-Computed Tomography and Histologic Analysis", Journal of Craniofacial Surgery, vol. 29, no. 8, pp. 2026-2031, 2018. DOI: 10.1097/scs.0000000000004672. [22] D. Levy, P. Pecha, S. Nguyen and R. Schlosser, "Trends in complications of pediatric rhinosinusitis in the United States from 2006 to 2016", International Journal of Pediatric Otorhinolaryngology, vol. 128, p. 109695, 2020. DOI: 10.1016/j.ijporl.2019.109695. [23] D. Chumachenko, V. Balitskii, T. Chumachenko, V. Makarova, V. and M. Railian, "Intelligent expert system of knowledge examination of medical staff regarding infections associated with the provision of medical care", CEUR Workshop Proceedings, pp. 321, 2019. [24] A. Stenzinger et al., "Artificial intelligence and pathology: From principles to practice and future applications in histomorphology and molecular profiling", Seminars in Cancer Biology, 2021. DOI: 10.1016/j.semcancer.2021.02.011 [25] A. Binder et al., "Morphological and molecular breast cancer profiling through explainable machine learning", Nature Machine Intelligence, vol. 3, no. 4, pp. 355-366, 2021. DOI: 10.1038/s42256-021-00303-4. [26] S. Krivenko, V. Lukin, O. Krylova, L. Kryvenko and K. Egiazarian, "A Fast Method of Visually Lossless Compression of Dental Images", Applied Sciences, vol. 11, no. 1, p. 135, 2020. DOI: 10.3390/app11010135. [27] S. Panigrahi and T. Swarnkar, “Machine learning techniques used for the histopathological image analysis of oral cancer-A Review,” The Open Bioinformatics Journal, vol. 13, no. 1, pp. 106–118, 2020. [28] G. Kopanitsa, A. Dudchenko, and M. Ganzinger, “Machine learning algorithms in cardiology domain: A systematic review (preprint),” JMIR Medical Informatics, 2019. [29] S. M. S. Dashti and S. F. Dashti, “An expert system to diagnose spinal disorders,” The Open Bioinformatics Journal, vol. 13, no. 1, pp. 57–73, 2020.