Classifier of Liver Diseases According to Textural Statistics of Ultrasound Investigation and Convolutional Neural Network Іllya Yankovy, Oleh Ilarionov, Hanna Krasovska and Iryna Domanetska Taras Shevchenko National University of Kyiv, 24 Bogdana Gavrylishina Str,04116, Kyiv, Ukraine Abstract Liver disease is a significant threat to human life and requires accurate and rapid diagnosis. Traditional methods either carry additional health risks (biopsy) or are not accurate and fast enough (manual analysis of ultrasound diagnostics). This paper proposes a classifier of liver diseases based on ultrasound images. The uniqueness of this approach to solving the problem of disease classification is a combination of classical methods of texture analysis, radiomics and modern convolutional neural networks. The resulting architecture demonstrates higher indicative accuracy compared to radio-based models that do not use convolutional neural networks both in the classification among all types of diseases (growth from 77% to 87%) and in the classification in the approach "one against all" or the norm / disease (increase from 85% to 92%). Keywords 1 medical image processing, classification, texture analysis, radiomics, convolutional neural networks, sonography, liver diseases. 1. Introduction The liver is a multifunctional and vital human organ. Any damage to this organ is life-threatening and should be diagnosed as early as possible to begin treatment in time and avoid serious complications. At the same time, liver diseases affect about 30% of the world's population. More than 1.3 million people died from viral hepatitis in 2015 alone, according to the World Health Organization. World mortality from liver disease is on a par with that of tuberculosis, human immunodeficiency virus or malaria. [1] Timely and accurate diagnosis of liver disease is important because it can prevent such deadly complications as cirrhosis or liver cancer. Diagnosing the patient's condition requires the doctor to quickly make a clinical decision based on known initial information. In medicine, these are usually the external manifestations of the disease and the results of tests. The speed and accuracy of decision-making depends on the competence of the doctor, his / her clinical experience, and the ability to analyze large data sets on the characteristics of the disease in a particular patient. One of the sources of such data is medical visualizations of the human body with the help of special devices and further diagnosis by a doctor. However, this approach has significant shortcomings associated with the human factor and therefore in recent decades there has been a field of science that deals with the quantitative analysis of medical images using intelligent systems. In the field of image diagnostics, there is an important, but still unsolved problem of classifying diseases related to the liver on the basis of ultrasound diagnosis of the liver. Improper diagnosis can delay treatment and thus increase the likelihood of death for the patient. The relevance of this topic lies in the need for safe (without surgery), accurate and rapid diagnosis of ultrasound medical images of the liver using existing technologies of artificial intelligence and applied statistics. II International Scientific Symposium «Intelligent Solutions» IntSol-2021, September 28–30, 2021, Kyiv-Uzhhorod, Ukraine EMAIL: annavkrasovska@gmail.com (H.Krasovska); oilarionov@gmail.com (O.Ilarionov); irinadomanetskaya@gmail.com (I.Domanetska) ORCID: 0000-0003-1986-6130 (H. Krasovska); 0000-0002-7435-3533 (O. Ilarionov); 0000-0002-8629-9933 (I.Domanetska) ©️ 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 60 2. The aim of the study Review of the topic of disease diagnosis and development of a software application, based on the architecture of the classifier, capable of computational analysis of medical ultrasound images of the human liver based on machine learning and radiomics and provide qualitatively new information for further diagnosis by medical professionals. 3. Materials and research methods The study focuses on combining two different approaches (texture analysis and convolutional neural networks) to solve the problem of classifying liver diseases on ultrasound images. The proposed network architecture implements a combination of these methods. The clinical base of the study consists of 210 images of ultrasound diagnostics, including 120 images with normal liver and 90 images of patients. The images showed areas of interest (texture of the liver parenchyma, its functional tissue), pre-marked by medical professionals on ultrasound images in b- mode of the device (Fig. 1) - 291 normal and 223 in pathology. These images were obtained, recorded and anonymized as a result of examination of 64 patients (including 27 with diffuse diseases and 37 in normal) in the state institution "Institute of Nuclear Medicine and Radiation Diagnostics of the National Academy of Medical Sciences of Ukraine Figure 1: Medical image of human liver ultrasound and radiological diagnostics 4. Literature review Diffuse liver disease is a dangerous group of diseases, in the absence of which can develop cirrhosis and liver cancer (hepatocellular carcinoma) [2]. The main feature of this group of liver diseases is that they trigger the process of daily breakdown of liver cells - chronic inflammation, or the development of fibrosis [3]. Fibrosis is characterized by the appearance of denser connective tissue, which can be observed during ultrasound examination of the liver. The human body forms scars trying to isolate inflammation. Lack of treatment at this stage threatens the development of much more dangerous pathologies [2]. The final stage of fibrosis is the development of liver cirrhosis, in which the structure of the liver surface is irreversibly damaged due to mass scarring and all the basic functions of this organ are disrupted. 61 Cirrhosis significantly increases the risk of liver cancer and the likelihood of death [4, 5]. Therefore, diffuse liver disease without proper and timely treatment can be very dangerous for a person, and to choose a treatment strategy, you must first diagnose. The so-called "gold standard" or the main procedure for diagnosing the liver until the early 2000s was a biopsy. This is an invasive (surgical) procedure during which a fragment of liver tissue is obtained and analyzed. The direct disadvantage of this operation is the risk of probable harm to the patient, because there is a possibility of various complications, such as hematoma of the liver, or in extreme cases there may be massive internal bleeding that threatens human life and health. An alternative to biopsy is the use of medical imaging of ultrasound (sonography) of the human liver. Medical imaging is a scientific discipline that studies science-intensive processes of obtaining visual images of internal organs and parts of the human body that are used for clinical analysis and decision-making on further medical interventions. Medical imaging technology aims to reveal information about the structures hidden by skin and bones in order to effectively diagnose and treat people. Medical imaging is non-invasive, that is, without the introduction of any instruments into the human body [6]. The obtained images give medical professionals unique information about the condition of various body structures, be it bones, organs, muscles, tendons, nerves, cartilage, blood vessels and others. Based on these images, physicians conduct expert analysis, which can be considered as a solution of inverse mathematical problems. In this case, the cause (properties of living tissue) is obtained from the effect (the observed signal or medical image). The approach of ultrasound examination of internal organs is non-invasive, ie does not require surgery in the human body. Sonography is a method of medical examination using high-frequency sound waves, or ultrasound. This medical imaging is necessary for the safe diagnosis of human internal organs. The technical side of the process is to record ultrasonic waves reflected from the surfaces of internal organs [7]. The obtained images are studied by medical specialists manually and diagnosed on the basis of their expert knowledge. However, direct analysis of sonography images has a number of disadvantages associated with the human factor, namely: low accuracy of classification, at a level of about 80% compared to biopsy and low speed of clinical decision-making. To diagnose this method requires the consent of several medical professionals, which requires additional time. The available meta-analysis of the accuracy of the diagnostic test based on 43 publications for the method of sonography against a more accurate biopsy shows a specificity between 70-85% and a sensitivity in the range of 73-90% [8]. The human factor imposes significant limitations on the manual diagnosis of medical images: the accuracy and speed of clinical decisions. And these parameters are crucial in the diagnosis of liver disease because only a timely accurate diagnosis can save a person's life, while errors lead to delayed treatment, which can result in more serious consequences for the patient. It is proposed to consider the possibility of solving this problem using machine learning methods. But first, you need to provide a formal description of the selected task: There are M classes of images (areas of interest in the selected images). Classes are represented as definite finite or infinite sets of multidimensional objects 𝑂𝑖∗ , 𝑖=1,…, 𝑀 . It is assumed that 𝑂𝑖∗ ∪ 𝑂𝑗∗ = ∅. Classes are defined by educational subsets of areas of interest 𝑜𝑖𝑗 , 𝑗 = 1, … , 𝑛𝑖 , where 𝑛𝑖 − power subsets. Each data point is an image of the ultrasound diagnosis of the human liver. The problem lies in choosing the best classification algorithm 𝑜𝑖𝑗 from 𝑂𝑖∗ , 𝑖 = 1, … , 𝑀 with considering the original data sets 𝑂𝑖∗ , 𝑖 = 1, … , 𝑀. During the latter decade since 2010, the methods of artificial intelligence, namely the technologies of deep learning (from the English. "Deep learning") became widespread in science [9, 30, 31, 32]. The field of medical research is no exception, especially in the field of image processing of medical origin and precision medicine [10]. The area of greatest interest in AI methods was radiology, the number of articles using AI in which increased from 100-150 in 2007-2008 to more than 700-800 in 2016- 2017 [11]. The medical industry is interested in the ability to diagnose and classify medical images using intelligent algorithms with accuracy, at a level or higher than that of the average physician. At the same time, the system does not replace a person, but plays the role of a consultative tool only, because the responsibility for making clinical decisions remained with the medical specialist. Over time, this technology began to be used in the study of the human liver. There were a number of scientific publications (Table 1) on the use of convolutional neural networks used in different modalities (types 62 of medical imaging). As a metric in these studies, histological (and tissue-related) expert assessment of the severity of liver fibrosis according to the METAVIR system on a 5-level scale (F0-F4) is often used, where F0 is the absence of fibrosis, F4 is liver cirrhosis. Table 1 Use of convolutional neural networks applied on different modalities (types of medical visualizations). № Type of technology AUC, area under the Source ROC curve 1 In-depth training based on MRI data F4: 0.84; ≥ F3: 0.84; ≥ [12] (magnetic resonance imaging) F2: 0.85 2 In-depth training based on CT data F4: 0.73; ≥ F3: 0.76; ≥ [13] (computed tomography) F2: 0.74 3 In-depth training based on share-wave F4: 0.97; ≥ F3: 0.98; ≥ [14] elastography, which uses ultrasound to obtain a F2: 0.85 map of the hardness of the liver surface Among other medical imaging, AI methods have begun to be frequently applied to images of ultrasound examination of the human liver [15]. In the last few years, a study has been published on the use of MN, GN and, above all, convolutional neural networks for tasks related to the diagnosis of the liver and related systems of the human body based on images of ultrasound examination of the human liver (Table 2). Table 2 Tasks of diagnosing liver diseases using convolutional neural networks № The name of the task Precision Sensitivity Specificity Source 1 detection of fatty liver disease 100% 100% 100% [16] 2 detection and classification of 97.2% 98% 95.7% [17] different types of focal liver damage 3 assessment of hepatic steatosis 96.3% 100% 88.2% [18] using transfer learning methods 4 assessment of chronic liver disease 87.3% 93.5% 81.2% [19] A classic alternative to medical image analysis is methods that involve the use of certain hand- engineered data features that can be used effectively for specific types of tasks. Radiomics is a statistical method of medical research that extracts quantitative features using data characterization algorithms and is adapted for medical radiographic images [20]. One of the types of such images are sections of the texture of the surface of the liver, obtained by sonography (ultrasound). A texture is an image that can convey the characteristics of the surface of a particular object. This image contains information suitable for use in object classification and recognition tasks. In order to use this information, you must first obtain it from there. One of the most widely studied methods is texture analysis (TA). TA is a branch of science that studies images and investigates the description of the properties of images using textural features [21]. To calculate the textural features, different types of spatial relations of neighboring pixels or voxels (points in three-dimensional images) are considered, on the basis of which special tables are formed, from which values are obtained according to existing formulas. There are two types of texture statistics based on their image properties (Fig. 3). To calculate the textural attributes of the first order, histograms of images based on their level of intensity are used. To calculate the texture attributes of the second and higher orders, special matrices are used that demonstrate the relationship between adjacent pixels of the image, taking into account their level of intensity [23–27]. 63 Figure 3: Classification of textural features Among the texture statistics of the highest order there are a number of matrices, each of which conveys certain unique properties of the texture of the studied image. Texture statistics, methods of their construction and calculation of matrices are described in more detail in the publication [22]. First-order features, or histogram features, describe the statistical properties of pixels in a selected area of an image. Such signs can be, for example, the maximum, minimum, average and median values of the intensity in the selected area, the standard deviation from the mean, the asymmetry of the distribution. Second-order features, or texture features, describe the correlation of the values of neighboring pixels and the homogeneity of the selected area. For example, a high degree of correlation of the intensity of neighboring pixels will give a visually "smooth" texture, and their low correlation will lead to the effect of "roughness" of the selected area. Higher-order features describe the statistical features of images obtained from the original ones by applying various mathematical operations: Fourier transform, wavelet analysis, various filters. The Radiomics framework was used to calculate the properties of image texture statistics (Figure 4). This software library was developed by scientists from the Laboratory of Computational Image Processing and Bioinformatics of Harvard Medical School [28] for mass use by scientists in the field of precision medicine and diagnostics in problems using artificial intelligence technologies. The Radiomics framework is used in the feature extraction and feature selection stages to prepare the attributes to be used in the next steps and to test the classification algorithms. Recent publications [30] in the field of medical image processing demonstrate the greatest efficiency of using convolutional neural networks as a classifier in comparison with other classical methods of feature extraction. Convolutional neural networks (CONNs) are a class of deep neural networks specialized in the effective recognition of visual images and texture patterns. ZNM are used in the tasks of classification, object recognition and image processing due to the ability to achieve in these tasks performance that can be compared with human [31]. This type of intelligent system does not require significant pre- 64 processing of data and uses its own method of extracting and constructing features from data based on the use of convolutions, which distinguishes it from many other types. Convoluted ANN is a class of computer systems that is able to automatically select important features of images that are inaccessible to humans, which distinguish these images from each other. Figure 4: Approach to using the Radiomics library [29] Review studies [32] demonstrate that convolutional ANN is the most popular and effective method of working with both general images and specific medical images in the field of radiology. Taking into account previous observations and interest in comparing the new approach with the existing one, it was decided to develop a classifier architecture based on a convolutional artificial neural network in combination with a module that performs pre-processing and calculates textural features of incoming medical images (Fig. 5). Figure 5: The procedure for applying machine learning methods in the system. New methods are highlighted in orange, the proposed expansion compared to the previous study [21]. 65 The main idea of the new approach is that the output of the last aggregation layer of the convolutional neural network and the textural features are transmitted together at the input to the fully connected layer of the multilayer perceptron (Fig. 6). Figure 6: Prototype of the classifier system architecture In essence, an ensemble approach to solving the problem of classifying liver diseases has been proposed. Ensembles are a combination of several algorithms at once that learn at the same time and correct each other's mistakes. Today, they are the ones that give the most accurate results, so they are most often used by all large companies for which fast and accurate processing of large amounts of data is important. The ensemble approach is based on the use of several training algorithms in order to obtain better forecasting efficiency than could be obtained from each training algorithm separately. The final properties of the network architecture (Fig. 7) were chosen taking into account the available literature of the recommendations of experts in the field of neural networks [33]. Figure 7: The final version of the classifier architecture 66 The structure of the network can be described as follows: 1) At the input, the system receives an image of 32x32 pixels. 2) After that on 4 layers of a convolutional network in turn there is an operation of maximizing aggregation (from English max pooling) and convolution. 3) This is followed by flattening and transforming the data into a single vector. During the work of the neural network 3200 properties are formed, and also 75 properties are formed by means of the module which is engaged in calculation of texture statistics. 4) This is followed by 2 layers of a normal feedforward neural network to implement the final classification. At the output, the system receives 1 value of the most probable class to which the input image belongs. 5. Results The selected classifier`s architecture was tested on a clinical data set using k-fold cross-validation. Cross-validation is a statistical method used to estimate the performance (or accuracy) of machine learning models. It is used to protect against overfitting in a predictive model, particularly in a case where the amount of data may be limited. In cross-validation, you make a fixed number of folds (or partitions) of the data, run the analysis on each fold, and then average the overall error estimate. K-fold cross validation guarantees that the score of our model does not depend on the way we picked the train and test set. The data set is divided into k number of subsets and the holdout method is repeated k number of times. Because it ensures that every observation from the original dataset has the chance of appearing in training and test set, this method generally results in a less biased model compare to other methods. It is one of the best approaches if we have limited input data. The disadvantage of this method is that the training algorithm has to be rerun from scratch k times, which means it takes k times as much computation to make an evaluation [34]. The new approach demonstrates better results in a comparison with a previous study [21], which did not use neural network methods (Table 3). Table 3 The effectiveness of the proposed method in comparison with the previous study Approach to system development Classification among Classification of all diseases norm /disease Classical texture analysis, machine learning 77% 85% (preliminary developments) [21] Classical texture analysis + in-depth training 87% 92% (current work) 6. Conclusions Liver disease poses a significant threat to human health and therefore requires rapid and accurate diagnosis. Among traditional diagnostic methods, the surgical method of biopsy carries additional health risks, and ultrasound analysis without the use of intelligent image analysis systems is not accurate and time consuming. Therefore, there is a need to create an effective classification system for liver disease based on ultrasound using an objective quantitative assessment of image texture. This study offers a new approach to solving this problem through a combination of classical methods of texture analysis and convolutional neural networks. This approach includes the calculation of gray level matrices GLCM, GLRLM, GLSZM, GLDM, NGLDM and textural features to each of these matrices. Also, the input image passes through a convolutional neural network, and at the output of it the attributes of these methods are combined in a fully connected artificial neural network. The chosen architecture demonstrates its effectiveness as a classifier by improving the accuracy of classification compared to the radio-based system, which does not use in-depth training in both 67 classification among all classes of diseases (growth from 77% to 87%) and in the classification approach. one against all ”or norm / disease (increase from 85% to 92%).. 7. References [1] World Health Organization. Global Hepatitis Report, 2017/2017. [2] Bataller, R., Brenner, D. a. Science in medicine - liver fibrosis. Journal of Clinical Investigation. 2005. Vol. 115, no. 2. p.209–218. DOI: https://doi.org/10.1172/JCI24282 [3] Tchelepi, H., Ralls, PW, Radin, R., and others. Sonography of Diffuse. Ultrasound. 2002. Vol. 21, no. 9. p.1023–1032. DOI: https://doi.org/10.7863/jum.2002.21.9.1023. [4] Curley, S.A, Izzo, F., Ellis, LM, and others. Radiofrequency ablation of hepatocellular cancer in 110 patients with cirrhosis. Annals of Surgery. 2000. Vol. 232, no. 3. p.381–391. DOI: https://doi.org/10.1097/00000658-200009000-00010 [5] Materne, R., Smith, AM, Peeters, F., et al. Assessment of hepatic perfusion parameters with dynamic MRI. Magnetic Resonance in Medicine. 2002. Vol. 47, no. 1. p.135–142. DOI: https://doi.org/10.1002/mrm.10045 [6] Bushberg, JT, Seibert, A., Leidholdt Jr., EM, et al. The Essential Physics of Medical Imaging, 2nd ed .: American Journal of Roentgenology . 2003. 596p. [7] Aleshkevich, AI, Rozhkovskaya, VV, Sergeeva, II, and others. Fundamentals and principles of radiation diagnostics .: Minsk: Belarusian State Medical University, 2014. 86c. [8] Bohte, AE, Werven, JR Van, Bipat, S., and others. The diagnostic accuracy of US, CT, MRI and 1H-MRS for the evaluation of hepatic steatosis compared with liver biopsy: a meta- analysis. European Radiology. 2011. Vol. 21, no. 1. p.87–97. DOI: https://doi.org/10.1007/s00330-010-1905-5 [9] Domanetska I., Khaddad A., Krasovska H., Yeremenko B.n Corporate system users identification by the keyboard handwriting based on neural networks. International Journal of Innovative Technology and Exploring Engineering this link is disabled. 2019. Vol. 9(1), pp. 4156–4161. DOI: 10.35940/ijitee.A6101.119119 [10] Lee, J., Jun, S., Cho, Y., et al. Deep learning in medical imaging: general overview. Korean Journal of Radiology. 2017. Vol. 18, no. 4. p.570–584. DOI: https://doi.org/10.3348/kjr.2017.18.4.570. [11] Pesapane, F., Codari, M., Sardanelli, F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. European Radiology Experimental. 2018. Vol. 2, no. 1. DOI: https://doi.org/10.1186/s41747-018-0061-6 [12] Yasaka, K., Akai, H., Kunimatsu, A., et al. Liver fibrosis: Deep convolutional neural network for staging by using gadoxetic acid-enhanced hepatobiliary phase MR Images. Radiology. 2018. Vol. 287, no. 1. p.146–155. DOI: https://doi.org/10.1148/radiol.2017171928 [13] Yasaka, K., Akai, H., Kunimatsu, A., et al. Deep learning for staging liver fibrosis on CT: a pilot study. European Radiology. 2018. Vol. 28, no. 11. p. 4578–4585. DOI: https://doi.org/10.1007/s00330-018-5499-7. [14] Wang, K., Lu, X., Zhou, H., et al. Deep learning radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicenter study. Good. 2019. Vol. 68, no. 4. p.729–741. DOI: https://doi.org/10.1136/gutjnl-2018-316204 [15] Zhou, LQ, Wang, JY, Yu, SY, etc. Artificial intelligence in medical imaging of the liver. World Journal of Gastroenterology. 2019. Vol. 25, no. 6. p.672–682. DOI: https://doi.org/10.3748/wjg.v25.i6.672. [16] Biswas, M., Kuppili, V., Edla, DR, and others. Symtosis: A liver ultrasound tissue characterization and risk stratification in optimized deep learning paradigm. Computer Methods and Programs in Biomedicine. 2018. Vol. 155. p.165–177. DOI: https://doi.org/10.1016/j.cmpb.2017.12.016 [17] Hassan, TM, Elmogy, M., Sallam, ES Diagnosis of Focal Liver Diseases Based on Deep Learning Technique for Ultrasound Images. Arabian Journal for Science and Engineering. 2017. Vol. 42, no. 8. p.3127–3140. DOI: https://doi.org/10.1007/s13369-016-2387-9 68 [18] Byra, M., Styczynski, G., Szmigielski, C., et al. Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. International Journal of Computer Assisted Radiology and Surgery. 2018. Vol. 13, no. 12. p.1895–1903. DOI: https://doi.org/10.1007/s11548-018-1843-2 [19] Gatos, I., Tsantis, S., Spiliopoulos, S., and others. A Machine-Learning Algorithm Toward Color Analysis for Chronic Liver Disease Classification, Employing Ultrasound Shear Wave Elastography. Ultrasound in Medicine and Biology. 2017. Vol. 43, no. 9. p.1797–1810. DOI: https://doi.org/10.1016/j.ultrasmedbio.2017.05.002 [20] Parekh, V., Jacobs, MA Radiomics: a new application from established techniques. Expert Review of Precision Medicine and Drug Development. 2016. Vol. 1, no. 2. p.207–226. DOI: https://doi.org/10.1080/23808993.2016.1164013 [21] Nailon, WH Texture Analysis Methods for Medical Image Characterization. Intech. 2016. Vol. i, no. tourism. p.75-100. DOI: https://doi.org/10.5772/8912 [22] Yankovy, I., Nastenko, E. Classifier of the liver in children with pathology of the hepatobiliary system according to the texture statistics of ultrasound. Biomedical Engineering and Technology. 2019. No. 2. p.15–23. [23] Haralick, RM, Shanmugam, KS, Dinstein, I. Textural Features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics. 1973. p.610–621. DOI: https://doi.org/10.1109/TSMC.1973.4309314 [24] Haralick, RM Statistical and structural approaches to texture. Proceedings of the IEEE. 1979. Vol. 67, no. 5. p. 786–804. DOI: https://doi.org/10.1109/PROC.1979.11328 [25] Galloway, MM Texture analysis using gray level run lengths. Computer Graphics and Image Processing. 1975. Vol. 4. p.172–179. DOI: https://doi.org/10.1016/S0146-664X(75)80008-6 [26] Thibault, G., Fertil, B., Navarro, C., et al. Texture Indexes and Gray Level Size Zone Matrix Application to Cell Nuclei Classification. Pattern Recognition and Information Processing. 2009. No. November. p.140–145. [27] Amadasun, M., King, R. Texural Features Corresponding to Texural Properties. IEEE Transactions on Systems, Man and Cybernetics. 1989. Vol. 19, no. 5. p.1264–1274. DOI: http://dx.doi.org/10.1109/21.44046 [28] vanGriethuysen, JJM, Fedorov, A., Parmar, C., et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Research. 2017. Vol. 77, no. 21. p.104–107. DOI: doi: 10.1158/0008-5472.CAN-17-0339. [29] Lambin, P. Radiomics Digital Phantom. Cancer Data. 2016. DOI: http://dx.doi.org/10.17195/candat.2016.08.1 [30] Li, Q., Cai, W., Wang, X., et al. Medical image classification with convolutional neural network. 2014 13th International Conference on Control Automation Robotics and Vision, ICARCV 2014. Vol. 2014, p.844–848. DOI: https://doi.org/10.1109/ICARCV.2014.7064414 [31] Ciregan, D., Meier, U., Schmidhuber, J. Multi-column deep neural networks for image classification. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2012. p. 3642–3649. [32] Yamashita, R., Nishio, M., Do, RKG, and others. Convolutional Neural Networks: an Overview and Its Applications in Pattern Recognition. Smart Innovation, Systems and Technologies. 2018. Vol. 195. p. 21–30. [33] Brownlee, J. How To Improve Deep Learning Performance, 2019. URL: https://machinelearningmastery.com/improve-deep-learning-performance/Deep Learning Performance. [34] Ray, S. Improve Your Model Performance using Cross Validation (in Python and R), 2018. URL: https://www.analyticsvidhya.com/blog/2018/05/improve-model-performance-cross- validation-in-python-r/ 69