Our software has been developed for ultrasound Bimaging [1] in neurology to detect hyperechogenicity of the substantia nigra [2], [3] which is probably one of the most common markers of Parkinson's Disease detectable on transcranial (TCS) ultrasound B-MODE images. The principle of the core algorithm based on binary thresholding enables to load not only ultrasound images. Thus, MR images could be also analyzed using this software tool. Clinical studies were published since 2014. The core of the software was improved, especially new ROI areas for different diagnoses.
In modern neurology and neurosurgery, MRI is one of the most progressive medical imaging for all perioperative phases [5]. MRI and diagnostic ultrasound are commonly considered as complementary diagnostic modalities; also for diagnosis confirmation.
In this study, we have three sets of T1 and T2 MR images (two basic types of MR images) [6] with different image resolution to analyze using the same approach as for ultrasound images; using echogenicity index as a feature to distinguish different brightness level. In comparison with ultrasound B-MODE images which we have used in previous studies, there is no native scale how to select a window 50 × 50 mm so we use the full width of the image, see Fig. 1.
Figure 1: Input MR image with selected ROI
We analyze three ROI areas with different size and shape, see Fig. 2. For each image, all three ROI areas are placed in the same position in the image. In other words, each image is analyzed thrice using three different ROI in the same position.
The size and shape of the ROI were defined in the past for B-MODE images. Originally, ROI1 was used for ncl. raphe analysis and ROI2 has been defined for substantia nigra area; both in B-MODE images. Square-shaped ROI3 has been used to analyze medial temporal lobe (MTL) in different case; in measurement of the black/white pixel ratio in the ROI 20 × 20 mm to judge a probability of MTL atrophy as a marker for the dementia [7].
In the case of B-MODE images, the echogenicity index should corresponds with echogenicity grade of the tissue. We can use the same index for MR images, in which the index should corresponds with brightness level of the examined part inside the ROI. The index is one numerical value computed using our software. More information about the methodology, see [8]; the paper focused on atherosclerotic plaques in B-MODE images, in which we have defined the index and its purpose. Simply, the index is computed as one number which can describe visual brightness level (echogenicity grade in US imaging). Our software is based on computing the area of remaining pixels after binary thresholding in the ROI. Let we have 256 intensity levels H i where i = 0, 1, ..., 255, the area is computed for each level. After that, the all computed areas are summed and the sum is divided by 100 to obtain the index given by
Due to the principle of binary thresholding, for lower echogenicity grade, the Echo-Index should be lower and for higher echogenicity the Echo-Index should be higher. This is an assumption which proceeds from the principle of binary thresholding. We have used the index in MR images to judge general reproducibility between two nonexperienced observers.
An example of achieved results for a selected image set of 14 images, is stated in Table 1. From achieved results we can judge the echogenicity index could be well-applicable in general as a feature in MRI analysis.
In Table 2 you can see the average differences for each ROI in four image sets. It is closely related to judge level of agreement which is almost perfect. The data in Table 2 shows that the differences between observers are minimal in the case of the same position of the ROI including a small shift. It seems that small ROI position changes which are not recognizable visually, have no significant influence on resulted echogenicity index. From achieved results, the range of the index is from 28.17 up to 38.89; very similar for each image set. According to the range and computed average differences between the observers, the difference between observers is smaller than 5 %.
In the case of US imaging, the image can be adjusted dynamically during examination by ultrasound probe settings; we can increase or decrease the brightness level according to examined tissue density. The echogenicity grade displayed on acquired digitized image can be different visually for the same tissue density. Due to this fact we need to analyze the image sets with same probe (image) settings to avoid incorrect echogenicity evaluation. See Fig. 3 The brightness settings should not be affected by settings during examination but there is other limitation in MR images corresponding with ROI selection in MR slices, see the following chapter.
Evaluation in MRI
Although it seems the echogenicity index is well reproducible, there is one important limitation. In Fig. 5, there is the example of using the same ROI size and shape to select a structure (it is not important from medical point of view at this moment). Due to weighted MRI, the examined structure can be smaller, larger, deformed or may not visible. In the case of ultrasound B-MODE images, like the substantia nigra in TCS images, the position and size is determined; only echogenicity grade is different corresponding the gain settings, angle, etc. In this case, totally different echogenicity grade can be obtained for the same patient but in different MR image. Thus, another ROI types will be defined in the future which should be better adapted for different MR images to examine structures. This limitation is also a barrier for automatic ROI selection discussed in the following chapter.
In our previous study dedicated to atherosclerotic plaques analysis in B-MODE images, we also have discussed the possibility to automatic learning of the plaque detection using ANN [10] and also a possibility to create a decisionmaking expert system to evaluate the echogenicity as a risk marker of the plaque [11]. Ultrasound imaging is widely used in atherosclerosis recognition to early diagnosis [12]. We have presented a draft of back-propagation ANN model to find a closed region of the plaque. In this field, ANN based on deep learning approach are widely used. In general, the ANN could be used to place ROI according to learning of some structure in MR image like in Fig. 1. However, the most important barrier is the fact that examined structure may vary in weighted MR images in each image due to intensity level [13]. See an example in Fig. 6 how the weighted MR images are different for the same examined patient. Thus, it could be hard to apply an automatic recognition a ROI described by shape or size when is changing in the weighted MR imaging.
The principle could be based on iterative learning using a convolutional neural network (CNN) which uses filtering to extract some features to recognize the region. CNN are designed to work with grid-structured inputs, like 2D images. There are many advanced techniques using CNN in medical imaging like in [14].
In 2020, we presented an idea to automatic segmentation based on boundary recognition of atherosclerotic plaques in B-MODE images [15]. It could be realized using iterative boundary recognition based on active contour algorithm boosted with CNN to train corresponding pairs input-output to learn the rules how to obtain the plaque border, see Fig. 7 in which the contours are shown and segmented plaque shapes after 25 iterations.
In the case of MRI, there is a different task. There are no exact borders to find the ROI. In Fig. 6, the weighted MR images are displayed; it could be hard to learn what to consider as a feature. Let to have a structure in MR image which is probably located equally based on radiologist's In this field, there is an interesting inspiration how to develop an automatic segmentation using deep learning approach in T1 and T2 weighted images [16]. The desired goal is to train the ANN to extract some features of the examined structure to place the ROI to the correct position.
To CNN training, the back-propagation algorithm is used; similarly as in linear feed-forward ANN architecture. Input image is represented as a single vector w × h × d where w and h represent image resolution and d to be color depth, in this case d = 1 (for RGB channels d = 3). Each pixel is represented as intensity value in the range of 0 to 255. CNN uses ReLU (Rectified Linear Unit) activation function instead of sigmoid or hyperbolic tangent in traditional multi-layer backpropagation networks. In general, CNN has the following layers and functions:
1. input layer (as a single vector w × h × d) Convolutional layer with ReLU and pooling layer are designed for feature extraction and the fully-connected layer with softmax function is used to classification. In our case, we need to recognize a structure in MR image which is defined by a radiologist, e.g. in Fig. 5 and/or in Fig. 10. The process is illustrated in Fig. 9.
Deep learning paradigm is based on learning rules from inputs and desired outputs. This is main difference from traditional programming when we have inputs, rules and we need to create outputs. Deep learning requires large data amount to efficiency. In comparison with traditional neural networks and learning, deep learning should achieve better accuracy related to increasing data amount [18]. In a critical point, depending on data complexity and its structure, conventional paradigms could be inefficient due to overfitting so the learning rate is low or stopped.
In MR images, we can use deep learning approach to learn the rules to recognize the ROI. In general, deep learning is focused on training with pairs input-output from large datasets, e.g. thousands of images. Thus, when we need to learn a specific structure in MR images, the training is based on input-output training set to learn rules, i.e. features, to find an appropriate structure to place a predefined ROI. The idea of deep learning using CNN is illustrated in Fig. 9. 10. Consider a task to find the highlighted anatomic structure (square-shaped ROI). It seems, there is really hard to learn the features of the structure because it is from small to bigger area in which the structure is located.
To effective training and learning the network, a large set of images is needed to learn how to recognize the structure from input-output training set. For example, we can learn the edge, the brightness difference, the shape, e.g. roundness, height/width ratio, etc. and another feature. In Figure 10: Six MR images with highlighted square-shaped ROI this task, deep learning could be applied to help to extract the features to recognize the structure. The background of the principle of the image convolution algorithm in CNN you can find in [17] and also in [18] which is a comprehensive guide to deep learning paradigm. In 2021, a paper focused on multi-classification of brain tumors in MRI using CNN, including deep performance evaluation, has been published [19]. In future, automatic finding of the ROI could be one of the main goals in our long-term research.
There are many ways for practical implementation. One of the most known to be Keras, a high-level modular API developed for Python programming language using GPU acceleration. More information, code samples (including using for CT scans) are available on Keras.io website.
The goal of the paper is to show how to use echogenicity index, computed with ultrasound B-MODE images, in MR images. To this purpose, we have analyzed sets of T1 and T2 MR images. The principle of the analysis is equal as for B-MODE images. The core of the algorithm is based on binary thresholding of the images in grayscale. Within this MR images analysis, the main idea is also applicable in MR images; the higher index value should correlate with higher brightness intensity and vice versa. Achieved results show the principle of the echogenicity index could be applied for B-MODE images and MR images independently. It seems, the echogenicity index is well applicable to observe different brightness in MRI equally as in the case of B-MODE images. The obtained differences are not significant, but the software is more sensitive than visual assessment in general.
Finally, we can recommend using this methodology in future clinical studies focused on the analysis of MRI using different ROI shapes and sizes according to examined structure in MR image. In future, we will use a new ROI areas, like a circle-shaped and/or free-hand closed area, defined by an experienced sonographer. It is related to examined structures in MR images, see Fig. 8 as the example.
In parallel, we are working on analysis of the echogenicity index differences between a light area and a dark area within the same ROI.
This