The presented paper is dedicated to image processing of ultrasound B-images in neurosonology. Atherosclerotic plaques in in-vitro B-images are analyzed in this study. The content is divided into two core parts. The first one is focused on computing Echo-Index value inside the atherosclerotic plaque as a defined Region of Interest. From achieved results is obvious that the EchoIndex is well-reproducible value in general. Totally of 278 images were analyzed by two non-experienced observers and by an experienced sonographer to validate the result. Basic statistical descriptors were calculated to judge the level of agreement. In this part, the ROI were selected manually. The second part is focused on approach to automatic selection of the ROI. Manual drawing of the border is time-consuming. Our idea is to use active contour algorithm (ACM; Active Contour Model) to eliminate the black background from the displayed plaque. Using ACM can be useful way to select ROI automatically. The main issue is to separate the shape of the plaque from neighbor structures, especially from the bottom of the tube. To adapt ACM, the principle of the convolutional neural network can be used to extract the feature of the shape to select a correct ROI. Thus, CNN can be trained and learnt to adapt number of iterations of the ACM (or another parameter) based on supervised learning from properly bordered examples. The main disadvantage is time-consuming process and high performance to be needed to train and learn the CNN. In fact, in this study, there is no real design of CNN but the primary goals are defined to realize in future.
In recent few decades, ischemic stroke caused by
atherosclerosis is one of the top causes of the mortality
worldwide. In modern neurology, ultrasound B-imaging is
one of the diagnostic tools to detect atherosclerotic plaques
in general. The diagnostic ultrasound [
In 2011, we developed a software tool B-MODE Assist using binary thresholding algorithm to detect hyperechogenicity of the substantia nigra. Due to general properties of B-images in grayscale, the software can be used also for the atherosclerotic plaques but in slightly adapted form.
For this research, a set of 278 B-images (in-vitro) is processed. In Fig. 1, an example of the B-image in which the plaque is displayed, is stated.
Our recent research has been based on computing the echogenicity index (called Echo-Index) to comparison of the risk of the plaque using our developed software and a visual assessment by an experienced sonographer. 2
The echogenicity index (Echo-Index) is one numeric value
which could corresponds with the echogenicity grade. The
Echo-Index is computed in closed Region of Interest, in
this case in the atherosclerotic plaque. To obtain the
EchoIndex, we use own developed software tool B-MODE
Assist, originally developed for substantia nigra echogenic
area computation. Hyperechogenic substantia nigra is a
detectable parkinsonic marker in B-image. The algorithm
is fully described in our previous publications, e.g. [
In the case of analysis of the substantia nigra, this
algorithm was genuinely useful. The speed of decreasing of
the area inside the ROI has been observed. For substantia
nigra, an equal shape of the ROI is used; an elliptical shape
with area of 50 mm2. It was sufficient for clinical studies,
e.g. [
In the case of atherosclerotic plaques, the main difference is the fact that ROI is selected by a free-hand shape and the area is different for each plaque. In Fig. 2, six different shapes of the plaque are presented as a closed ROI selected in the window 20 20 mm.
The idea of the Echo-Index is considered as a number which can describe echogenicity grade inside the plaque; inside the free-hand ROI. Let H is the brightness value of a pixel and T to be the threshold then AT is the computed area for each threshold T in the range of 0 T 255. The sum is computed 255 AREASU M = å AT
T =0 and the AREASU M value is divided by 100
Due to the principle of binary thresholding, for lower echogenicity grade, the Echo-Index should be lower and for higher echogenicity the Echo-Index should be higher. This is an assumption which proceeds from the principle of binary thresholding. Thus, in the case of low echogenicity, for low T threshold the computed area should be very low and vice versa. In consequence of this principle, the sum for low echogenicity is low and for high echogenic ROI the sum is higher. During the future work it should be confirmed whether this idea is correct or not; especially in comparison with visual assessment. 2.1
The goal of the pilot study [
The reproducibility of the Echo-Index has been proved as well-acceptable in general considering inexperience in neurosonology of both of the observers. The Echo-Index does not evince significant difference in case of the same image analyzed by each observer; each of them draws ROI slightly in different shape. Table 1 shows an example of computed Echo-Index for 22 images between 2 observers; there are no significant differences due to similarly drawn ROI manually by each observer.
Some images from the set were also analyzed by an experienced sonographer; ROI drawn precisely; and very similar values for Echo-index have been achieved. • range, variance for Echo-Index values measured by observer1 - first measurement R11, var11 • range, variance for Echo-Index values measured by observer1 - second measurement R12, var12 • range, variance for Echo-Index values measured by observer2 - first measurement R21, var21 • range, variance for Echo-Index values measured by observer2 - second measurement R22, var22 • maximum and mean difference between two observers max(obs), mean(obs) • correlation coefficient between Echo-Index values from 2 observers robs • maximum and mean difference between measured values from observer1 between two weeks max(obs12w) • maximum and mean difference between measured values from observer2 between two weeks max(obs22w) • correlation coefficient of the Echo-Index values from observer1 between 2 weeks robs1 • correlation coefficient of the Echo-Index values from observer2 between 2 weeks robs2 Obtained results are summarized in Table 2.
Maximum values are stated in absolute value because the difference of the Echo-Index between observers or measurement can be also negative. In the case of measurement during the first phase (week), for 220 values from 278, i.e. 79.1 %, the difference under 100 between observers has been achieved. In the case of the second phase, for 214 values from 278, i.e. 76.9 %, the difference under 100 between observers has been achieved. Due to achieved results, the Echo-Index can be considered as well-reproducible value between 2 independent, nonexperienced observers and also between 2 measurements from the same observer. In general, the basic idea "smaller Echo-Index means lower echogenicity" which was not confirmed from the point of view of an experienced sonographer, who compared images with different plaque risk level in which different echogenicity grade is obvious and there is no significant correlation between visual assessment and computed Echo-Index.
Although the Echo-Index seems like a reproducible value,
it must be thoroughly examined if the value corresponds
with visual assessment by an experienced sonographer.
The idea is to create a decision-making expert system
using a knowledge base of echogenicity grades determined
by experienced sonographer. In the future, the
decisionmaking system can be developed to use as a tool to
evaluate the probability risk of the plaque in accordance with
Echo-index value in determined intervals. A draft of the
system were presented in [
This part of the paper is focused on automatic selection of the ROI instead of manual process. A manual selection of the plaque is used up to now. However, each selection of the plaque takes up to 2 minutes, see in Fig. 2 that some shapes are relatively simple but another one are very complex to its exact selection.
We need to find a way how to select the plaque from the
input B-image (Fig. 3) automatically. The idea is based
on removing the black solid background; B-image can be
comprised of the background in the major. Put differently,
it is desired to detect edges of the plaque to select the
Region of Interest in which the Echo-Index is computed. The
active contour algorithm is one of the segmentation
techniques which is used to detect the background and to
extract the foreground of the image. It is based on the
iterative process. In many recent studies were demonstrated
that active contour segmentation technique is well
applicable for medical images against its complex
morphological structure. One of many studies focused on using active
contour segmentation in clinical image processing is
available in [
The goal of the active contour is to remove the black background and to extract the border of the plaque. See in Fig. 1. As a result, the goal is to execute the steps 1 to 3 automatically. The algorithm is based on iterative steps to acquire the edge which separates segmented objects in the image. The sensitivity is the primary parameter; how sensitive the detection is. Thus, what is considered as background and what is considered as foreground bordered by the edge. For example, we tried to segment the plaque using threshold level segmentation with 3 different thresholds (T = 15, T = 25 and T = 40). You can see in Fig. 4 that higher threshold level has the influence on segmented area. In the case of T = 40, the plaque is segmented as isolated object with no bottom of the tube. To get an optimal result, the threshold is important. In the case of active contour, the number of iterations is an important parameter like the threshold. At the beginning, initial mask is set and increasing of iterations produces better border of the object. In Fig, 4 is demonstrated that after thresholding there are many isolated objects with small area. We need to create a ROI mask which is one closed boundary (one segmented object). Using Active Contour could be a useful way to perform it.
Initially, ACM with 25 iterations has been performed. See Fig. 5 in which the results are demonstrated for 2 different images.
The results can be used in general but as an experimental study to further improvement. Firstly, we need to separate the plaque from the tube bottom. See Fig. 6 for some examples of the segmented masks.
Surely, we can increase the number of iterations in the algorithm but no one object is get. When higher number of iterations is used, the border is get more precisely but still more than one segment is obtained. 3.2
Better results of the ACM were achieved with
semiautomatic input. After selection of the window with the
plaque, click into the plaque to understanding which
segment is useful for us and set the threshold level (as H
intensity level). This level is automatically computed by Otsu
algorithm which can be well applicable as a segmentation
method for medical images, for example in a study from
2012 [
In Fig. 9 initial rectangle ROI and segmented area using ACM after only 10 iterations is shown. B-images have a complex structure with a lot of isolated pixels caused by noise and artifacts, see Fig. 7. To faster and more accurate results of ACM, the brightness enhancement can be recommended as a pre-processing phase at the beginning. The idea is to "clear" image from isolated pixels and enhance the brightness to better finding the contour. Let H to be brightness of the pixel. For example, all pixels with H < 20 can be set as H = 0 (as the background) and all pixels with H > 80 are transformed as H + 30. The effect is presented in Fig. 10.
In Fig. 8 there are results after for H = 20 and click into the plaque. The results are more accurate in comparison with using ACM for the whole image.
The main benefit of this transformation is to eliminate noise in low echogenicity levels but some image artifacts are enhanced too. Although it can be useful pre-processing phase to achieve better ACM results. 4
Due to limitations mentioned above, the algorithm should
be adaptable to find an appropriate border of the plaque.
The result of the active contour algorithm depends
primarily on brightness of the image. In other words, it is
necessary to adapt the algorithm to achieve the best
sensitivity in accordance with brightness of the input image.
Using a neural network is one of the ideas how to improve
border finding. In 2018, we published a pilot study with
an experimental draft of a feed-forward neural network to
recognize the shape of the plaque using edge detection
operators [
The goal of the CNN is to train and to learn the shape of the plaque using active contour algorithm; in other words to estimation of the active contour properties to find the best contour of the plaque in B-image. In image processing, CNN represent an eminently suitable tool for automatic segmentation using deep learning, not only for medical images.
The idea is inspired from the research [
CNN should work as a supervised learning model. Therefore, we can use correctly drawn borders for training and learning process of the CNN. The convolutional layer of the CNN is designed to extract the features using a convolution kernel operation, e.g. 5 5. In Fig. 11 a simplified model of CNN is illustrated. The convolutional layer is based on computing 2-D convolution. Let f (x; y) is an input image, g(x; y) to be an output image and the w to be the convolution kernel, 2-D convolution is computed by g(x; y) = w f (x; y) (3) on 2-D image. There are usually more than one convolutional layer in CNN to detect edges, gradient to higher level of segmented objects.
In Fig, 11 there is an example of the process what we need. No classification of the object is necessary, only to extract the mask to learn ACM model.
Design of the CNN is nontrivial, long-term process due to complexity of B-images. The goal is to interconnect the CNN with ACM algorithm to reliable detection of the plaque separately from neighbor objects (bottom of the tube). To training and learning, correct examples of plaque borders will be used. Interconnection with ACM could automatically help predict number of iterations to achieve better segmentation results of the plaque. Design, training, learning and testing of CNN are time-consuming tasks for which we need a large dataset. Although automatic ROI selection is useful and faster, there is one meaningful relation between ROI and computed Echo-Index. To Echo-Index computing, currently only manual selection of the ROI is used. So, each ROI was precisely selected, and the difference between observers was minimal. In the case of automatic selection, ROI can be selected inaccurately in comparison with manual border drawing so Echo-Index could be different. Observe the example in Fig. 12 between manual and ACMbased border. In the case of ACM, ROI is selected including black background inside; it is undesirable. Currently, manual selection is more accurate but this a starting point to develop the ACM boosted by CNN to create an automatic segmentation of the plaque.
Due to this fact, the goal is minimize difference between manual drawing and using ACM with trained CNN to optimize the accuracy. Using CNN could predict an optimal number of iterations from training set of correctly selected ROI. 5
This paper is divided into two main parts. The first one is focused on reproducibility of the Echo-Index between two non-experienced observers for atherosclerotic plaques in B-images. In all images, the plaque was selected manually.
In the second part, the approach to automatic plaque selection instead of manual drawing the border, is discussed. Using active contour segmentation algorithm with possibility to improve the accuracy using a convolutional neural network (CNN) could be useful.
In general, the Echo-Index can be considered as a wellreproducible value. From achieved statistical results between two non-experienced observers in sonography, there is no significant difference. However, Echo-Index is not usable from the clinical point of view because there is satisfactory correlation between Echo-index and visual assessment of the echogenicity grade which should be an indicator of the risk level in general. So, lower Echo-Index value does not correspond to lower echogenicity and vice versa.
The second part is dedicated to automatic ROI selection to eliminate manual drawing the border. Currently, it is only idea to do it. It seems that using active contour algorithm could be useful. In addition, it can be supported by CNN to smart adaptation of the ACM parameters. However, this is time-consuming task to effective learn the CNN. It is one of the key goals for future work to improve automatic detection of the atherosclerotic plaque without manual drawing of the border. Faster and reliable solution using artificial intelligence components is expected.
This research is supported by the project IT4Innovations Excellence in Science - LQ1602 and data supported by Ministry of Health of the Czech Republic, grants nr. 1628628A.