Dealing with artifacts in order to segment medical images stills a challenge. In this paper we propose new appropriate features for Magnetic Resonance Images (MRIs) data that allow enhancing tissue segmentation, regardless levels of the inherent artifacts, namely noise and intensity non uniformity (INU). We show that using features based on spatial entropy of intensities with diferent classifiers has significantly enhanced the brain matter segmentation. In addition to their powerful discrimination, the proposed features are computationally low. Experimentation was conducted using a brain web database, and the obtained results have allowed us to conclude that the proposed new features are well suited to represent MRI data for image segmentation.
Pattern-based classification approaches have wined more interest in the last decade. According
to such approaches, not only data are used to infer features, but also patterns that exist in data
are used also for classification, and for feature definition. It has been stated in several works [
In this work, we are interested in the classification of MRI data where we propose a new energy based feature, namely entropy of intensity. Indeed, in contrast to most of the published works, and instead of using the raw image data, stored individually at the diferent voxels, we consider the neighborhood of the voxels to form patterns, and then use these latter as features for classification. To do this, and in order to proof that entropy-based coding allows best results, we will proceed according to two diferent patterns: In the first, we use a simple aggregation of the voxels that surround the voxel in question. So, the pattern in this case consists of the set of the voxels that form the neighborhood of the voxel to classify.
For the second pattern, we use an energy-based coding; assuming that the energy defined within the neighborhood of a voxel represents well its interaction with its neighborhood. The model is therefore based on defining, within the neighborhood of every voxel, an energy function represented by a spatial entropy. Such an energy function allows considering, in addition to the image intensity, the specific geometrical features that characterize MRI data.
We show that the aggregation of data according to the considered patterns, especially with
energy coding, as well as the classification of these patterns with two classifiers: Neural
Network classifier and Naïve Bayes classifier, allow to significantly improve the results of
classiifcation of MRI data. According the conducted experimentation, MRI segmentation based on
the proposed features was strongly enhanced. Such enhancement is due first to the ability of
the entropy to capture interactions between neighboring voxels. Secondly the diversity of
feature instances of entropy allows overcoming both the over-training problem that characterizes
MRI data, and the problem of the early convergence of the classification algorithm when it is
optimization-based such as in the neural based methods [
The remain of the paper is organized as follows: in Section 2 we present a review of segmentation methods for MRI data, including those based on machine learning, and we review the main works published in the literature. In Section 3 we introduce our approach by presenting the data aggregation method through energy coding. Section 4 is devoted to the experimentation of the proposed features, where we introduce the used MRI database, the obtained results, as well as an analysis and a discussion on the obtained results.
Image segmentation is one of the most important tasks in pattern recognition using visual data. It consists of subdividing the pixels / voxels of an image into distinct and homogeneous regions. There are dozens of diferent segmentation methods. However, all these methods can be classified according to three main families: 1. Contour-based methods: the common principle to these methods is that they proceed to detect discontinuities in visual data. These discontinuities represent the edges in the image. The detected edges are generally disjoint and open, and therefore they must to be joined and closed for adequate use in the subsequent recognition process. 2. Region-based methods: their principle consists in grouping the pixels / voxels of the image having similar features, in disjoint but homogeneous subsets according to a given homogeneity criterion. These homogeneous subsets are called regions. 3. Methods by classification: their major asset is that they allow learning from the labeled data called training set. Segmentation by classification consists of assigning a label to any pixel / voxel of the image using a classifier (single or ensemble, classical or deep). Given that we are interested in this last family in this work, we devote the remainder of this section to introduce some methods of MRI segmentation by data classification methods.
Classification-based segmentation methods can be themselves subdivided into two
subcategories:
1. Heuristic-based methods: where one or more heuristics are considered to define a pixel
/ voxel labeling criterion. The heuristics consider given priors, related to the image, to the
noise, or to the distortions that the image could undergo during its acquisition. For instance,
we can cite the Fuzzy c-mean (FCM) algorithm [
Magnetic Resonance Imaging (MRI) is of great importance for the establishment of correct
diagnoses and thus the prescription of appropriate treatments. The segmentation of a MRI
consists in extracting the main tissues for which physicians and radiologists are mainly interested.
These tissues are respectively CSF (Cerebrospinal Fluid), GM (Gray Matter) and WM (White
Matter), for structural MRI, and also LM (Lesion Matter) for pathological MRI. Several methods
of MRI segmentation have been published, starting with contour detectors, passing through
region extractors, and ending with machine-learning based methods. Richard et al. [
Several works have proposed machine-learning methods for MRI segmentation. Some of
them have combined classifiers, where mainly the unique used feature was the voxel intensity.
In most of reviewed works, authors proceeded to extract features then use the the latter as
inputs for classifiers in order to label MRI data. Rajasree et al. have considered a fractal
representation of MRI data, by using the Brownian move technique [
Gustavo et al. have combined Genetic Algorithms (GA) and Adaboost classification to detect
the tumor area in the MRI [
Entropy based features for MRI processing are rare in the literature. Sarita et al. have combined probabilistic neural network and wavelet entropy for feature extraction to classify MRI data [14]. Entropy based features were also used with optimization-based clustering, such as in the work introduced by Pham et al. [15], where authors combined fuzzy entropy clustering and multi-objective particle swarm optimization. Contrary to the previous works, where entropy is computed according likelihood states of pixels, our entropy-based features are spatial and are computed based on disparities between the intensities of the pixels/voxels in a given neighborhood.
We define in this section two pattern-based features that will be tested for brain tissue classification. In the two cases, Multilayer Perceptron and Naïve Byes classifiers are used to label the voxels in the MRI volume. The MRI is first preprocessed using skull-strip algorithm, namely FSL Brain Extraction Tool (BET) [16, 17], to remove non brain tissues. In our case we have preferred to avoid the noise filtering, given that, firstly, the MRI data is usually altered for the voxels in the neighborhood of the diferent tissues, and secondly because the proposed patterns allow to reduce the efect of the noise during the classification, even without MRI data smoothing, given that they consider the pixel/voxel in question and its neighborhood.
The MRI volume obtained after skull-striping is a set of voxels that each one can belong to one of the three remaining tissues, namely, the Cerebro-Spinal Fluid (CSF), the Gray Matter (GM), and the White Matter (WM). Each of them is characterized by its mean intensity and the corresponding standard-deviation ( , ) , ∈ { , , }. We assume also that the intensity distribution in each tissue is Gaussian (see Formula 1) ( , , ) = 1 √ 2 21 ( − )2/2 2 (1) where is the intensity of the voxel at the location .
For this fist pattern-based classification of the MRI data, we consider for each voxel in the MRI volume its neighboring voxels that surround it, except for voxels that are situated on the sides of the volume where a particular processing is dedicated. The Neural Network and the Naïve Bayes classifiers are learned based on a training MRI volume with its ground truth labeling. Figure 1 depicts the principle of MRI segmentation based of this pattern. According to this pattern, a voxel is not labeled according to its alone intensity but according to the intensities of the voxels surrounding it. So, noise and INU are indirectly considered, because voxels that are wrongly labeled due to high variations of their intensities, are corrected using their respective neighborhoods. Furthermore, the partial volume efect artifact is also considered. At the borders of the diferent tissues a voxel is likely labeled according to its intensity, and according to the dominant class in the neighborhood. So the resulting class for such voxel will be likely that of the tissue with close intensity and high occurrence.
Our proposed entropy-based pattern aims to capture interactions between the voxels belonging to a local neighborhood. Such interactions can be represented according to an energy function. So, the proposed pattern for a given voxel in the MRI volume is a vector of three components, where each component represents the spatial entropy of the intensities of the similar voxels in the neighborhood. Such subsets of similar voxels are obtained by the -means algorithm, applied on the voxel’s neighborhood with three classes (CSF,GW,WM) (see equation 2).
where denotes the set of the voxels belonging to the class , and is the probability that the voxel belongs to the class , and: = − ∑ × 2
=
√12 21 ( − )2/2 2 ∑ √12 21 ( − )2/2 2 , are respectively the mean and the standard-deviation of the intensities of the voxels belonging to the class and situated in the neighborhood of the voxel in question ( ). So, a clustering by the -means algorithm is performed at the voxel neighborhood, so the three subsets of voxels and their respective couples of ( , ), ∈ { , , } are obtained. As it can be noticed on Fig. 2,the vector of features that will be used for classification by Neural Network or Naïve Byes is composed of the intensity of the voxel in question and the three spatial entropies 1, 2 and 3, obtained according to the clustering of the set of the voxels forming the local neighborhood. Such a pattern captures well the interactions of the voxels, (2) (3) and expresses well the spatial constraints that exist within the MRI data. Entropies 1, 2, and 3 allow to distinguish the cases were the voxel is in the neighborhood of a tissue border or not. Also, they allow distinguishing if a voxel is afected by a high deviation due to noise or not. Obviously, the intensity of the voxel in question is considered for classification, so the resulting class is likely that of the tissue with the closest mean intensity, but adjusted if needed by the voxels in the neighborhood.
For the case with considering the entropies of the voxels in the neighborhood of the voxel in question, the vector of features that will be used for classification by Neural Network or Naïve Byes is composed of the intensity of the voxel in question and the entropies of all its neighbors calculated according the same principle for the central voxel.
The experimentation of the proposed patterns has been done using MRI volumes from the well known database brain web [18]. This database provides a large set of MRI volumes with their ground truth labeling that accords researchers to test their machine-learning based methods, and quantitatively evaluate them. Furthermore, MRIs can be obtained according to various levels of artifacts, namely noise, and INU. All MRIs are 181 × 217 × 181 voxels of size. In this work, they are considered only MRIs with T1 modality.
Two main indexes are usually computed to evaluate and compare segmentation methods based on classification and clustering. They are namely Jaccard and Dice indexes. Based on true positive ( ), true negative ( ), and false positive ( ) labeling instances, Jaccard coeficient is expressed as follows:
Dice coeficient can be expressed as: = =
We opted for Dice index given that it is more cited in the literature and it has been considered in works with which we compare our results.
In order to show the impact of entropy on improving the segmentation quality, we introduce a series of experiments with variation of several experimental elements. First, we present the segmentation results using the gray levels of the voxel and those of its neighbors. Then, we show the results obtained by using the entropy as a classification feature, and finally the results obtained based on the entropy but with including the voxels in the neighborhood of the voxel to be classified. We also considered two diferent classifiers, in order to show that the improvement of the results is not due to the classifier itself, but to the proposed features, namely the spatial entropy. Theses classifiers are respectively the Neural Network classifier and the Naïve Bayes one.
The obtained results for the white and the gray matter are relatively acceptable, especially for low levels of noise and INU. However, the results are very unsatisfactory for cerebrospinal lfuid (CSF). This is due to the fact that the CSF voxels are located in narrow regions where the neighborhood of such voxels is overlapping on neighboring tissues, which corrupts the classification data.
According the results introduced in Table 2, we notice a significant improvement in the segmentation results when the spatial entropy is used as a classification feature. This improvement INU/N can be explained by the capture, in the expression of the entropy, of the interaction between neighboring voxels. This interaction is expressed as an energy, formulated by the spatial entropy. A voxel is not classified solely according to its intensity but according to the strength of its interaction with neighboring voxels.
Table 3 presents the segmentation results using spatial entropy and by considering the voxel in question and its neighboring voxels.
Compared to the results of the entropy of the voxel taken alone, we note a strong robustness against noise and INU (see tables 2 and 3). For instance, for white matter, the variation of the Dice index is from 97.48 to 93.44 for a variation of noise from 1 to 7‘% and for an INU of 0%, while this variation was from 98.21 to 87.48 for the same variation of noise. It is the same for the robustness against the INU. The variation is from 98.21 to 82.38 for the INU varying from 0% to 90% with a noise level set at 1%, against a variation from 97.48 to 85.68 for the same variation of the INU (see table 2). It remains the same for the other two tissues, namely the gray matter and the CSF (see tables 2 and 3). The entropies respectively of the voxel to be classified and those of the neighboring voxels form a set of features have allowed to better segment the tissues even with high levels of noise and INU. Figure 3 shows the intensity of variations according to the level of noise with constant INU level.
For low values of the INU, the method based on the entropy of the voxel taken alone, produces better results compared to the method where the voxel is considered with its neighborhood. However, we can see in Figure 4 that taking into account the neighborhood improves the results in terms of robustness against noise and INU. The segmented images obtained with voxel neighborhood have more regular boundaries, and several voxels have been correctly reassigned. This can be explained by the discarding of the voxels whose gray levels are close to that of the tissue in question, but in reality they are noise voxels, located outside the region of the considered tissue.
All the experiments carried out using the Neural classifier were re-conducted under the same conditions using the Naïve Bayes classifier. In terms of performance, we have seen a slight drop in Dice index values for all the test images. However, we have also seen a high improvement in results with the use of entropy. We also noticed a stability of the results against noise and INU when the entropy is used as a feature when taking into account the neighborhood of the voxel. Figure 5 shows the variation of the Dice index according the noise level for the white matter for INU = 40%. Like the Neural classifier, we notice a stabilization of the results against noise.
In order to show the efectiveness of the proposed features, we introduce in table 4 a comparison between the obtained MRI data classification results and those of some well cited works from the literature. We have considered MRI with 20% INU and diferent noise levels, and we compared results for WM and GM tissues.
We can notice from the previous table that our method, especially when the neighborhood is taken into account, scores nearly close to the best one, namely Fast [19] compared to the others [20, 21]
According the diferent results introduced below, we notice the strong improvement in segmentation results when spatial entropy is used as classification feature. Indeed, despite taking the neighborhood into account when classifying voxels based on alone intensity, the results were not so satisfactory, especially with high levels of noise and INU. However, the use of only the spatial entropy value of the voxel to classify has allowed a large increase in the values of the Dice index, even with high levels of noise and INU. This can be explained by the fact that the proposed spatial entropy, in addition to its ability to consider the voxel and its neighborhood, and as it has an energy nature, it expresses the interaction force between the voxels in the MRIs. Thus, a voxel is not classified solely according to its value or the values of its neighbors, but also according to the force of interaction of the voxel with its neighborhood. Taking into account neighboring voxels in the context of spatial entropy has shown better robustness against noise and INU, especially when the levels of these artifacts are high. Such a result can be explained by the widening of the voxel’s field of interaction beyond its local neighborhood.
In this work, we have introduced a new feature for MRI data representation, allowing to considerably improving the classification of voxels, and thus the segmentation of this type of images. It consists of the spatial entropy, whose interest is to capture the interaction between neighboring voxels, which will allow the latter to be better classified. We considered two cases of use of the proposed spatial entropy: As one feature of the voxel to be classified, and as a set of features of the voxel in question and its neighboring voxels. The obtained results, by varying the diferent artifact levels, showed a strong improvement in the results for the first use case, and a good robustness for the second case. In future work, the proposed feature should be tested with diferent classifiers, and using ensembles of classifiers, as well as deep classifiers. [13] A. de Brébisson, G. Montana, Deep neural networks for anatomical brain segmentation,
CoRR abs/1502.02445 (2015). URL: http://arxiv.org/abs/1502.02445. arXiv:1502.02445. [14] M. Saritha, K. Paul Joseph, A. T. Mathew, Classification of mri brain images using combined wavelet entropy based spider web plots and probabilistic neural network, Pattern Recogn. Lett. 34 (2013) 2151–2156. URL: https://doi.org/10.1016/j.patrec.2013.08.017. doi:10.1016/j.patrec.2013.08.017. [15] T. X. Pham, P. Siarry, H. Oulhadj, A multi-objective optimization approach for brain mri segmentation using fuzzy entropy clustering and region-based active contour methods., Magnetic resonance imaging 61 (2019) 41–65. [16] S. Smith, Fast robust automated brain extraction, Human Brain Mapping 17 (2002). [17] M. JENKINSON, Bet2 : Mr-based estimation of brain, skull and scalp surfaces, Eleventh Annual Meeting of the Organization for Human Brain Mapping, 2005 (2005). URL: https: //ci.nii.ac.jp/naid/10030066593/en/. [18] C. Cocosco, V. Kollokian, R.-S. Kwan, A. Evans, Simulated brain database homepage, 1997.
URL: https://brainweb.bic.mni.mcgill.ca/brainweb, accessed: 2020-06-13. [19] Y. Zhang, M. Brady, S. Smith, Segmentation of brain mr images through a hidden markov random field model and the expectation-maximization algorithm, IEEE Transactions on Medical Imaging 20 (2001) 45–57. [20] J. Ashburner, K. J. Friston, Unified segmentation, 2005. [21] B. Caldairou, N. Passat, P. Habas, C. Studholme, F. Rousseau, A non-local fuzzy segmentation method: Application to brain MRI, Pattern Recognition 44 (2011) 1916–1927. URL: https://hal.archives-ouvertes.fr/hal-00476587. doi:10.1016/j.patcog.2010.06.006.