A Generic Multi-Agent Framework for Medical-Image Segmentation?

Abstract

Medical images o er visual representations of human bodies' complex internal structures. One of the most common process applied to those images is segmentation. It consists in dividing an image into a set of regions of interest. Human anatomical complexity and medical image acquisition methods make the segmentation of medical images very complex. Several solutions (algorithms and devices) have thus been proposed to automatize this process. However, most existing solutions were developed for one type of images and/or require several inputs of the user. In this demo, we propose a generic multi-agent framework for medical image segmentation. This framework is based on a set of autonomous and interactive agents that use a modi ed region growing algorithm and cooperate to segment the images. Experiments were performed on brain MRI simulated images and the obtained results are promising.

Keywords

• Medical Images
• Segmentation
• Multi-Agent Framework
• Interaction
• Region Growing Algorithm

-

Medical imaging requires several complex tasks such as segmentation. Image segmentation is a key technique in providing non-invasive information about human body structures. Lately, it became an essential tool used in medicines to collect information about patient's health condition in a non-invasive manner.

The image segmentation research community has developed several algorithms and tools. They are often based on thresholding [ 24 ], region growing [ 1, 14 ], and methods borrowed from other disciplines like Markov Random Fields [ 13 ] and Deformable Models [ 16 ]. Most existing segmentation techniques are ad hoc. They are speci c to the type of images that were designed for. Meanwhile, new medical tools, e.g. PET-SCAN, generate several types of images. In this context, we propose a generic framework for medical image segmentation. ? https://youtu.be/oCVLA81gI3c This framework is designed as a system composed of two di erent types of autonomous agents, which interact and coordinate in an environment (the treated image) to perform the segmentation process. Firstly, the rst set of agents start classi cation process based on pixels gradient level and use this classi cation to initiate the second set of agents. These latter starts then exploring the image to identify the regions, they perform region growing process and explore the possible merging between regions. Finally, they remove noise to improve the quality of the regions.

The aim of this demo is to describe the proposed framework and illustrate the behavior of the implemented multi-agent system on medical images. It is organized as follows. Section 2 describes the context and related work. Section 3 describes the framework and Section 4 gives some details about its implementation. 2

Related Work

The automation of the medical image segmentation is a very challenging problem due to the nature of images acquisition methods and artifacts. Many research projects were published in this eld. The following section gives an overview of medical image segmentation issues and presents the related work. 2.1

Medical Image Segmentation

Medical image segmentation is a key technique in providing non-invasive information about human body structures. It helps radiologists to visualize and study the anatomy of those structures. The so obtained information is used for di erent purposes like pathologies diagnostic, pre-operative planning and image guided surgical Procedures, diseases progress tracking and treatment planning [ 11 ].

An accurate segmentation is vital in medical imaging, but it is di cult to obtain, and therefore remains an open issue. The di culty of segmenting those images comes from di erent factors including their methods of acquisition and sampling (CT Scans and magnetic resonance for example) that generate noise (acquisition) and partial volume e ect (sampling), the complexity of human anatomical structures, the tissues intensity non-homogeneity, the closeness in gray level of di erent soft tissues (low contrast) [ 22 ] .

A wide range of medical image segmentation approaches were introduced for segmenting di erent modalities and pathologies. Those approaches use di erent techniques such as threshold [ 24 ] , region growing [ 1, 14 ] , Markov Random Fields [ 13 ], Fuzzy and Hard Clustering [ 2, 10 ], and Deformable Models [ 16 ]. They are often coupled with preprocessing techniques to reduce noise or artefacts, or with manual initialization [ 6 ]. All those approaches employ a monolithic, sequential and centralized systems to perform complex tasks [ 3 ]. To improve segmentation system e ciency, some approaches explore the usage of systems based on cooperative problem resolution called Muli-Agent Systems. 2.2

Multi-agent Segmentation Approaches

Recently, signi cant applications in di erent elds of the health care domain were developed using the multi-agent paradigm [ 5 ]. Among those applications, is the medical image segmentation. Several multi-agent approaches have been proposed to deal with medical image segmentation. We distinguish two categories. In the rst category, the agents encapsulate one of the various existing methods and try to improve it using di erent mechanisms (distribution, information di usion. . . ). The second category, on the other hand, attempt to exploit the potential of multi-agent systems by using the tools that they provide like coordination and cooperation. In this category we can nd solutions that rely on social coordination mechanisms such as ant colonies and social spider colonies [ 17 ]. For instance, Djemame et al. [ 9 ] use self-organization and adaptation of social spider in a MAS to extract homogeneous regions of an image. Liu and al.[ 15 ] use agents with living beings behavior to extract brain structures in a scan image. Richard et al. [ 21 ] use cooperative and interactive behaviors to respectively distribute the work and propagate information among agents of their MAS to segment medical images.

According to the literature, region growing method (RGM) is well suited for multi-agent medical image segmentation systems in both previously mentioned categories. Some systems use RGM on images sub parts, to label pixels from a set of known classes like [ 20 ] with MR brain images or [ 7 ] working on CT image scans. Other MAS approaches associate RGM with other segmentation methods, such as [ 12 ] with Fuzzy C-Mean method on brain MR images, [ 3 ] with Region Fusions on the same type of image, and also with Region Fusion in the process of microaneurysm detection in fundus images by [ 18 ]. 2.3

Discussion

The previous sections describe interesting and innovative approaches. Most of those approaches provide promising results. However, they often su er of one or several drawbacks [ 4 ]: { Each approach is speci c to the type of images that was designed for and does not support generalization; { Each approach is based on a prior knowledge or a training data set that a ects the segmentation results; { Each approach requires a user intervention for setting up parameters or thresholds.

In the aim of overcoming some of these disadvantages, we developed the adaptive multi-agent approach for medical image segmentation. The proposed approach was implemented in a Multi-agent dedicated framework. The latter is described in the following section.

MAMES: A

Segmentation

Multi-Agent framework For Medical image

MAMES relies on an evolution of the multi-agent approach presented in [ 4 ]. The system uses two populations of agents to perform segmentation of medical images. The di erent agents are situated in an environment that is de ned as a two-dimensional matrix of pixels. Each pixel contains two types of information: the scalar gray level intensity of the corresponding pixel in the processed image, and a vector value of the gradient on this pixel. This gradient value is obtained by the application of a Sobel lter on the initial image.

The image segmentation is performed in two steps: 1) classi cation of pixels and 2) region detection. These two steps are realized by two di erent populations of interactive and situated agents and are described in the following sections. 3.1

Classi cation of pixels

The rst population of agents aim is classifying image pixels into two categories according to their gradient level: class 1 (C1) with height gradient values (edge pixels and neighborhood) and class 2 (C2) with low gradient values (region pixels).

The agents of this population (named Thresholding Agents) use this classi cation to choose the seed pixels where to create the entities of the second population of agents (named Region Growing Agents).

The thresholding agents are dispersed homogeneously in the environment, where each agent is assigned to a subpart of the image. Then, the agents are launched to perform the following steps: 1. Thresholding Behavior: After its activation, a Thresholding Agent (TAgent) analyzes its assigned area by comparing the gray level (GL) of its pixels. If all pixels have the same GL, the whole area is classi ed as C2 pixels. Otherwise, the TAgent uses a classic K-means clustering algorithm to perform the classi cation of pixels area according to their gradient value. Then, the agent labels those pixels according to the resulting classi cation. 2. Adjacency Graph Creation: TAgents interact together and generate Adjacency Graphs where agents are the vertices. In those graphs, two vertices are adjacent if there is a discontinuous linear sequence of C2 pixels connecting them in the image. At the end of this process, each subset of connected TAgents creates a Region Growing Agent (RGAgent). This RGAgent is initialized in the position (pixel) of the TAgent in the graph, having the highest number of connected neighbors. 3.2

Region Detection

The goal of the region agents is to detect the regions composing the image. Those agents (RGAgent), similarly to [ 4 ] use a modi ed version of the growing approach proposed in [ 19 ]. A RGAgent starts growing its region from its seed pixel (previously set by the TAgent). When the region becomes su ciently large, the agent explores possible merging with its neighbors. Lastly, RGAgents starts to nalize their regions by using the GL and the position of pixels for noise removing process. 1. Initial Region Growing: This rst phase is used to determine the characteristics of the region. Starting from the seed pixel, the agent uses a random walk and adds to its initial region Rinit any pixel P classi ed in C2 and with a gray level similar to the gray level of the growing region. 2. Final Region Growing: In this phase, the agent exploits the information collected during the previous phase. It calculates the mean gray level ERinit (G) and standard derivation (Rinit) of the pixels of the initial region. Those two values are then used to evaluate the predicate of pixel assimilation during this region growing phase.

Starting from the seed pixel, the agent creates its nal region, and its neighbors' pixels are considered as the contours of this region. Thus, at each step of execution, the agent browses the list of its contour pixels, assimilates all of those pixels that satisfy its predicate, and updates then its contours. This growing process is repeated while some contour pixels satis es the agent predicate. 3. Merging: In this step, RGAgents use region neighborhood concept, where two regions are considered as neighbor if they have borders in common. Thus, The RGAgents attempt to expand their regions by merging with their neighbors. They use the contract net protocol [ 23 ] to evaluate the relevance of a merge by comparing the standard derivation before and after a possible merge. This evaluation is then used to choose the best merging among the list of neighbors. According to this selection, RGAgents perform merging, update their neighborhood and restart searching for another merge. The process is repeated until no merge possibility is detected within neighbors list. 4. Region Finalization: Each RGAgent tries to assimilate all the pixels that actually belong to its region, but are a ected by a noise e ect due to the acquisition method. RGAgent browses the unassimilated situated inside his region and assimilates all the ones that satisfy a predicate where the GL of the pixel and its position in the region are considered.

When an agent cannot add anymore pixels to its region, it marks the bordering pixels as contours and self-deactivates. The whole system stops when all agents are deactivated. 4 4.1

Implementation and Experiments

Implementation

The Multi-agent framework MAMES consists of a population of reactive and situated agents with simple behavior and limited communication. Moreover, the segmentation process is time-consuming the reason why we implemented MAMES with C# and MS.Net framework instead of using available multi-agent tool such as JADE or MADKIT. Our implementation allows us to maintain total control of the system and fully optimize its performances. Our agents are implemented as simple C# objects concurrently executed under the control of a scheduler.

To exploit MAMES, We also develop a desktop application with a simple and functional interface (see Fig. 1). The Application can process the standard 2D image formats and can also extract 2D slices from image volumes. The tool was designed with the C# Winform technology and o ers the following features: { Opening and viewing digital 2D images to process. { Opening digital 3D Volumes and extract a 2D slice for segmentation. { Segmenting an image with the proposed multi-agent approach. { Viewing the classi cation resulting from the distributed Thresholding rst step. { Viewing the resulting regions detected by the system on the initial image. { Viewing the characteristics of the resulting regions (size, average gray level, standard derivation).

{ Viewing the detected contours.

The Fig. 1 represents the main interface of our prototype tool that we use to perform our experimentation. The obtained results are presented further in this section. In order to verify the e ciency of our multi-agent image segmentation framework and to insure his robustness, we perform some experiments with di erent sorts of medical images.

The rst set of our experiments were performed on a brain MRI phantom image simulated using the phantom database produced by McConnell Brain Imaging Center at Montreal Neurological Institute [ 8 ]. Using this simulated images permits the veri cation of the e ciency of our approach, despite the variation of image artifacts such as noise and intensity non uniformity (INU) in a quantitative evaluation. For this evaluation, the system is tested for the detection of white mater region on eighteen versions of a brain MRI slice (six di erent noise levels and three di erent INU levels). The used evaluation metric is the coe cient (kappa), also known as Dice similarity coe cient [ 25 ]. This coe cient is commonly used in the medical image processing to evaluate the performance of segmentation algorithms which has a prede ned ground truth information or dataset. It is calculated using the following formula [ 4 ]: =

2 T P (2 T P ) + F P + F N (1) where T P , F P and F N are the numbers respectively of True Positives, False Positives and False Negatives instances of pixel labeling. The value of the coe cient well expresses the segmentation quality.

The results of our experiments are presented bellow: coe cient for White Matter extraction with di erent noise and INU 100 95 iten 90 ce iff o C a appK 85 80 75 0% INU 20% INU 40% INU 0 2 4 Noise Level 6 8

10

Fig. 2: Application main form

Table 1 and Fig.2 demonstrate the e ectiveness of our system even with the increase of noise level and INU level. Those results are very promising considering that the system doesn't need any learning dataset or pre-treatment, and the unique experimentally de ned parameter is the agents population initial size. (a) Initial Image

(b) Detected Edges (c) Region 1 Fig. 3: Segmentation example of a brain slice with 20% INU level and 3% of noise

Finally, to verify the capacity of our system in segmenting di erent sorts of medical images, we tested it on a computerized tomography image of a human abdomen and on a cranial scan in addition to the MRI brain images. Fig.4 and Fig.5 allow a visual evaluation of the results. We can note that even with di erent kinds of images, our system was able to extract regions with e ciency. (a) Initial Image

(b) Detected Edges (c) Region 1 (c) Region 1

(d) Region 2 (e) Region 3 (f) Region 4

Fig. 5: Segmentation example of a human cranial scan

Conclusion

In this paper, we presented a multi-agent framework for medical image segmentation (named MAMES). MAMES implemented multi-agent systems that allow the simultaneous detection of di erent regions without any input of the user related to the image sort such as the region number, thresholds or some characteristics pre-de ned with a learning phase or a prior knowledge.

MAMES relies on two populations of interactive agents (Thresholding Agents and Region Growing Agents) for image segmentation. The rst population, named Thresholding Agents, classify the pixels and interact together to chose the best initial positions of the second population agents', named Region Growing Agents. The latter create then new regions and start the growing process. When the evolution of the region becomes not possible, the agents start a coordination process to perform the best merging. This process is repeated until no more possible merge were found. The agents then nalize their regions with a noise removing step.

MAMES was used to segment several medical images. The performed segmentation experiments validated its implementation and targets characteristics performance such as the genericity of MAMES and the segmentation quality.

References

1. Adams , R. , Bischof , L. : Seeded region growing . IEEE Transactions on pattern analysis and machine intelligence 16 ( 6 ), 641 { 647 ( 1994 )

2. Albouy-Kissi , A. , Cormier , S. , Tranquart , F. : Perfusion quanti cation of contrastenhanced ultrasound images based on coherence enhancing di usion and competitive clustering . In: Image Processing (ICIP) , 2012 19th IEEE International Conference on. pp. 2321 { 2324 . IEEE ( 2012 )

3. Benamrane , N. , Nassane , S. : Medical image segmentation by a multi-agent system approach . In: German Conference on Multiagent System Technologies . pp. 49 { 60 . Springer ( 2007 )

4. Bennai , M.T. , Guessoum , Z. , Smaine , M. , Cormier , S. , Mezghiche , M. : Towards a generic multi-agent approach for medical image processing . In: The 20th International Conference on Principles and Practice of Multi-Agent Systems (PRIMA 2017 ). ( 2017 )

5. Chakraborty , S. , Gupta , S.: Medical application using multi agent system{a literature survey . Journal of Engineering Research and Applications 4 ( 2 ), 528 { 546 ( 2014 )

6. Chen , C. h .: Handbook of pattern recognition and computer vision . World Scienti c ( 2015 )

7. Chitsaz , M. , Woo , C.S.: Medical image segmentation using a multi-agent system approach . Int. Arab J. Inf. Technol . 10 ( 3 ), 222 { 229 ( 2013 )

8. Collins, D.L. , Zijdenbos , A.P. , Kollokian , V. , Sled , J.G. , Kabani , N.J. , Holmes , C.J. , Evans , A.C. : Design and construction of a realistic digital brain phantom . IEEE transactions on medical imaging 17(3) , 463 { 468 ( 1998 )

9. Djemame , S. , Nekkache , M. , Batouche , M.: A multi-agent system for image segmentation a bio-inpired approach . In: Proceedings of the Third international conference on Innovation and Information and Communication Technology . pp. 17 { 17 . British Computer Society ( 2009 )

10. Duncan , J.S. , Ayache , N.: Medical image analysis: Progress over two decades and the challenges ahead . IEEE transactions on pattern analysis and machine intelligence 22 ( 1 ), 85 { 106 ( 2000 )

11. Elnakib , A. , Gimelfarb , G. , Suri , J.S. , El-Baz , A. : Medical image segmentation: A brief survey . In: Multi Modality State-of-the-Art Medical Image Segmentation and Registration Methodologies , pp. 1 { 39 . Springer ( 2011 )

12. Haroun , R. , Boumghar , F. , Hassas , S. , Hamami , L. : A massive multi-agent system for brain mri segmentation . In: International Workshop on Massively Multiagent Systems . pp. 174 { 186 . Springer ( 2004 )

13. Held , K. , Kops , E.R. , Krause , B.J. , Wells , W.M. , Kikinis , R. , Muller-Gartner , H.W. : Markov random eld segmentation of brain mr images . IEEE transactions on medical imaging 16(6) , 878 { 886 ( 1997 )

14. Kupinski , M.A. , Giger , M.L. : Automated seeded lesion segmentation on digital mammograms . IEEE Transactions on medical imaging 17(4) , 510 { 517 ( 1998 )

15. Liu , J. , Tang , Y.Y. : Adaptive image segmentation with distributed behavior-based agents . IEEE Transactions on Pattern Analysis and Machine Intelligence 21 ( 6 ), 544 { 551 ( 2002 )

16. McInerney , T. , Terzopoulos , D. : Deformable models in medical image analysis: a survey . Medical image analysis 1(2) , 91 { 108 ( 1996 )

17. Moussa , R. , Beurton-Aimar , M. , Desbarats , P. : On the use of social agents for image segmentation . In: International Conference on complex systems and applications ( iccsa 2009 ) ( 2009 )

18. Pereira , C. , Veiga , D. , Mahdjoub , J. , Guessoum , Z. , Goncalves , L. , Ferreira , M. , Monteiro , J.: Using a multi-agent system approach for microaneurysm detection in fundus images . Arti cial intelligence in medicine 60(3) , 179 { 188 ( 2014 )

19. Pohle , R. , Toennies , K.D.: Segmentation of medical images using adaptive region growing . In: Medical Imaging 2001 . pp. 1337 { 1346 . International Society for Optics and Photonics ( 2001 )

20. Richard , N., Dojat , M. , Garbay , C. : Automated segmentation of human brain mr images using a multi-agent approach . Arti cial Intelligence in Medicine 30 ( 2 ), 153 { 176 ( 2004 )

21. Richard , N., Dojat , M. , Garbay , C. : Distributed markovian segmentation: Application to mr brain scans . Pattern Recognition 40 ( 12 ), 3467 { 3480 ( 2007 )

22. Sharma , N. , Aggarwal , L.M. , et al.: Automated medical image segmentation techniques . Journal of medical physics 35(1) , 3 ( 2010 )

23. Smith , R.G. : The contract net protocol: High-level communication and control in a distributed problem solver . IEEE Trans. Computers 29 ( 12 ), 1104 { 1113 ( 1980 )

24. Xu , M. , Luk , W. , Cutler , P. , Digby , W. : Local threshold for segmented attenuation correction of pet imaging of the thorax . IEEE Transactions on Nuclear Science 41 ( 4 ), 1532 { 1537 ( 1994 )

25. Zijdenbos , A.P. , Dawant , B.M. , Margolin , R.A. , Palmer , A.C. : Morphometric analysis of white matter lesions in mr images: method and validation . IEEE transactions on medical imaging 13(4) , 716 { 724 ( 1994 )