Liver disease is a significant threat to human life and requires accurate and rapid diagnosis. Traditional methods either carry additional health risks (biopsy) or are not accurate and fast enough (manual analysis of ultrasound diagnostics). This paper proposes a classifier of liver diseases based on ultrasound images. The uniqueness of this approach to solving the problem of disease classification is a combination of classical methods of texture analysis, radiomics and modern convolutional neural networks. The resulting architecture demonstrates higher indicative accuracy compared to radio-based models that do not use convolutional neural networks both in the classification among all types of diseases (growth from 77% to 87%) and in the classification in the approach "one against all" or the norm / disease (increase medical image processing, classification, texture analysis, radiomics, convolutional neural Organization. World mortality from liver disease is on a par with that of tuberculosis, human immunodeficiency virus or malaria. [1] Timely and accurate diagnosis of liver disease is important because it can prevent such deadly complications as cirrhosis or liver cancer.
networks, sonography, liver diseases.
The liver is a multifunctional and vital human organ. Any damage to this organ is life-threatening and should be diagnosed as early as possible to begin treatment in time and avoid serious complications. At the same time, liver diseases affect about 30% of the world's population. More than million people died from viral hepatitis in 2015 alone, according to the
2021 Copyright for this paper by its authors.
Review of the topic of disease diagnosis and development of a software application, based on the architecture of the classifier, capable of computational analysis of medical ultrasound images of the human liver based on machine learning and radiomics and provide qualitatively new information for further diagnosis by medical professionals.
The study focuses on combining two different approaches (texture analysis and convolutional neural networks) to solve the problem of classifying liver diseases on ultrasound images. The proposed network architecture implements a combination of these methods.
The clinical base of the study consists of 210 images of ultrasound diagnostics, including 120 images with normal liver and 90 images of patients. The images showed areas of interest (texture of the liver parenchyma, its functional tissue), pre-marked by medical professionals on ultrasound images in bmode of the device (Fig. 1) - 291 normal and 223 in pathology.
These images were obtained, recorded and anonymized as a result of examination of 64 patients (including 27 with diffuse diseases and 37 in normal) in the state institution "Institute of Nuclear Medicine and Radiation Diagnostics of the National Academy of Medical Sciences of Ukraine
Diffuse liver disease is a dangerous group of diseases, in the absence of which can develop cirrhosis
and liver cancer (hepatocellular carcinoma) [
Fibrosis is characterized by the appearance of denser connective tissue, which can be observed
during ultrasound examination of the liver. The human body forms scars trying to isolate inflammation.
Lack of treatment at this stage threatens the development of much more dangerous pathologies [
The so-called "gold standard" or the main procedure for diagnosing the liver until the early 2000s was a biopsy. This is an invasive (surgical) procedure during which a fragment of liver tissue is obtained and analyzed. The direct disadvantage of this operation is the risk of probable harm to the patient, because there is a possibility of various complications, such as hematoma of the liver, or in extreme cases there may be massive internal bleeding that threatens human life and health.
An alternative to biopsy is the use of medical imaging of ultrasound (sonography) of the human
liver. Medical imaging is a scientific discipline that studies science-intensive processes of obtaining
visual images of internal organs and parts of the human body that are used for clinical analysis and
decision-making on further medical interventions. Medical imaging technology aims to reveal
information about the structures hidden by skin and bones in order to effectively diagnose and treat
people. Medical imaging is non-invasive, that is, without the introduction of any instruments into the
human body [
The obtained images give medical professionals unique information about the condition of various body structures, be it bones, organs, muscles, tendons, nerves, cartilage, blood vessels and others. Based on these images, physicians conduct expert analysis, which can be considered as a solution of inverse mathematical problems. In this case, the cause (properties of living tissue) is obtained from the effect (the observed signal or medical image).
The approach of ultrasound examination of internal organs is non-invasive, ie does not require
surgery in the human body. Sonography is a method of medical examination using high-frequency
sound waves, or ultrasound. This medical imaging is necessary for the safe diagnosis of human internal
organs. The technical side of the process is to record ultrasonic waves reflected from the surfaces of
internal organs [
The human factor imposes significant limitations on the manual diagnosis of medical images: the accuracy and speed of clinical decisions. And these parameters are crucial in the diagnosis of liver disease because only a timely accurate diagnosis can save a person's life, while errors lead to delayed treatment, which can result in more serious consequences for the patient. It is proposed to consider the possibility of solving this problem using machine learning methods. But first, you need to provide a formal description of the selected task: There are M classes of images (areas of interest in the selected images). Classes are represented as definite finite or infinite sets of multidimensional objects ∗, =1,…, . It is assumed that ∗ ∪ ∗ = ∅. Classes are defined by educational subsets of areas of interest , = 1, … , , where − power subsets. Each data point is an image of the ultrasound diagnosis of the human liver. The problem lies in choosing the best classification algorithm from ∗, = 1, … , with considering the original data sets ∗, = 1, … , .
During the latter decade since 2010, the methods of artificial intelligence, namely the technologies
of deep learning (from the English. "Deep learning") became widespread in science [
In-depth training based on share-wave elastography, which uses ultrasound to obtain a map of the hardness of the liver surface
Among other medical imaging, AI methods have begun to be frequently applied to images of
ultrasound examination of the human liver [
A classic alternative to medical image analysis is methods that involve the use of certain handengineered data features that can be used effectively for specific types of tasks. Radiomics is a statistical method of medical research that extracts quantitative features using data characterization algorithms and is adapted for medical radiographic images [20]. One of the types of such images are sections of the texture of the surface of the liver, obtained by sonography (ultrasound).
A texture is an image that can convey the characteristics of the surface of a particular object. This image contains information suitable for use in object classification and recognition tasks. In order to use this information, you must first obtain it from there. One of the most widely studied methods is texture analysis (TA). TA is a branch of science that studies images and investigates the description of the properties of images using textural features [21]. To calculate the textural features, different types of spatial relations of neighboring pixels or voxels (points in three-dimensional images) are considered, on the basis of which special tables are formed, from which values are obtained according to existing formulas.
There are two types of texture statistics based on their image properties (Fig. 3). To calculate the textural attributes of the first order, histograms of images based on their level of intensity are used. To calculate the texture attributes of the second and higher orders, special matrices are used that demonstrate the relationship between adjacent pixels of the image, taking into account their level of intensity [23–27].
Among the texture statistics of the highest order there are a number of matrices, each of which conveys certain unique properties of the texture of the studied image. Texture statistics, methods of their construction and calculation of matrices are described in more detail in the publication [22].
First-order features, or histogram features, describe the statistical properties of pixels in a selected area of an image. Such signs can be, for example, the maximum, minimum, average and median values of the intensity in the selected area, the standard deviation from the mean, the asymmetry of the distribution.
Second-order features, or texture features, describe the correlation of the values of neighboring pixels and the homogeneity of the selected area. For example, a high degree of correlation of the intensity of neighboring pixels will give a visually "smooth" texture, and their low correlation will lead to the effect of "roughness" of the selected area.
Higher-order features describe the statistical features of images obtained from the original ones by applying various mathematical operations: Fourier transform, wavelet analysis, various filters.
The Radiomics framework was used to calculate the properties of image texture statistics (Figure 4). This software library was developed by scientists from the Laboratory of Computational Image Processing and Bioinformatics of Harvard Medical School [28] for mass use by scientists in the field of precision medicine and diagnostics in problems using artificial intelligence technologies. The Radiomics framework is used in the feature extraction and feature selection stages to prepare the attributes to be used in the next steps and to test the classification algorithms.
Recent publications [30] in the field of medical image processing demonstrate the greatest efficiency of using convolutional neural networks as a classifier in comparison with other classical methods of feature extraction.
Convolutional neural networks (CONNs) are a class of deep neural networks specialized in the effective recognition of visual images and texture patterns. ZNM are used in the tasks of classification, object recognition and image processing due to the ability to achieve in these tasks performance that can be compared with human [31]. This type of intelligent system does not require significant preprocessing of data and uses its own method of extracting and constructing features from data based on the use of convolutions, which distinguishes it from many other types. Convoluted ANN is a class of computer systems that is able to automatically select important features of images that are inaccessible to humans, which distinguish these images from each other.
Review studies [32] demonstrate that convolutional ANN is the most popular and effective method of working with both general images and specific medical images in the field of radiology. Taking into account previous observations and interest in comparing the new approach with the existing one, it was decided to develop a classifier architecture based on a convolutional artificial neural network in combination with a module that performs pre-processing and calculates textural features of incoming medical images (Fig. 5).
The main idea of the new approach is that the output of the last aggregation layer of the convolutional neural network and the textural features are transmitted together at the input to the fully connected layer of the multilayer perceptron (Fig. 6).
In essence, an ensemble approach to solving the problem of classifying liver diseases has been proposed. Ensembles are a combination of several algorithms at once that learn at the same time and correct each other's mistakes. Today, they are the ones that give the most accurate results, so they are most often used by all large companies for which fast and accurate processing of large amounts of data is important. The ensemble approach is based on the use of several training algorithms in order to obtain better forecasting efficiency than could be obtained from each training algorithm separately.
The final properties of the network architecture (Fig. 7) were chosen taking into account the available literature of the recommendations of experts in the field of neural networks [33]. The structure of the network can be described as follows: 1) At the input, the system receives an image of 32x32 pixels. 2) After that on 4 layers of a convolutional network in turn there is an operation of maximizing aggregation (from English max pooling) and convolution. 3) This is followed by flattening and transforming the data into a single vector. During the work of the neural network 3200 properties are formed, and also 75 properties are formed by means of the module which is engaged in calculation of texture statistics. 4) This is followed by 2 layers of a normal feedforward neural network to implement the final classification.
At the output, the system receives 1 value of the most probable class to which the input image belongs.
The selected classifier`s architecture was tested on a clinical data set using k-fold cross-validation.
Cross-validation is a statistical method used to estimate the performance (or accuracy) of machine learning models. It is used to protect against overfitting in a predictive model, particularly in a case where the amount of data may be limited. In cross-validation, you make a fixed number of folds (or partitions) of the data, run the analysis on each fold, and then average the overall error estimate.
K-fold cross validation guarantees that the score of our model does not depend on the way we picked the train and test set. The data set is divided into k number of subsets and the holdout method is repeated k number of times. Because it ensures that every observation from the original dataset has the chance of appearing in training and test set, this method generally results in a less biased model compare to other methods. It is one of the best approaches if we have limited input data. The disadvantage of this method is that the training algorithm has to be rerun from scratch k times, which means it takes k times as much computation to make an evaluation [34].
The new approach demonstrates better results in a comparison with a previous study [21], which did not use neural network methods (Table 3).
77% 87% 85% 92%
Liver disease poses a significant threat to human health and therefore requires rapid and accurate diagnosis. Among traditional diagnostic methods, the surgical method of biopsy carries additional health risks, and ultrasound analysis without the use of intelligent image analysis systems is not accurate and time consuming. Therefore, there is a need to create an effective classification system for liver disease based on ultrasound using an objective quantitative assessment of image texture.
This study offers a new approach to solving this problem through a combination of classical methods of texture analysis and convolutional neural networks. This approach includes the calculation of gray level matrices GLCM, GLRLM, GLSZM, GLDM, NGLDM and textural features to each of these matrices. Also, the input image passes through a convolutional neural network, and at the output of it the attributes of these methods are combined in a fully connected artificial neural network.
The chosen architecture demonstrates its effectiveness as a classifier by improving the accuracy of classification compared to the radio-based system, which does not use in-depth training in both of classification among all classes of diseases (growth from 77% to 87%) and in the classification approach. one against all ”or norm / disease (increase from 85% to 92%)..