=Paper=
{{Paper
|id=Vol-1194/visceralISBI14-3
|storemode=property
|title=Automatic Multi-Organ Segmentation Using Fast Model Based Level Set Method and Hierarchical Shape Priors
|pdfUrl=https://ceur-ws.org/Vol-1194/visceralISBI14-3.pdf
|volume=Vol-1194
}}
==Automatic Multi-Organ Segmentation Using Fast Model Based Level Set Method and Hierarchical Shape Priors==
https://ceur-ws.org/Vol-1194/visceralISBI14-3.pdf
Automatic multi-organ segmentation using fast model
based level set method and hierarchical shape priors
Chunliang Wang Örjan Smedby
Center for Medical Imaging Science and Department of Medical and Health
Visualization(CMIV),Linköping University Sciences (IMH), Linköping University
Linköping, Sweden Linköping, Sweden
chunliang.wang@liu.se orjan.smedby@liu.se
Abstract
An automatic multi-organ segmentation pipeline is presented.
The segmentation starts with stripping the body of skin and
subcutaneous fat using threshold-based level-set methods.
After registering the image to be processed against a stan-
dard subject picked from the training datasets, a series of
model-based level set segmentation operations is carried out
guided by hierarchical shape priors. The hierarchical shape
priors are organized according to the anatomical hierarchy
of the human body, starting with ventral cavity, and then
divided into thoracic cavity and abdominopelvic cavity. The
third level contains the individual organs such as liver, spleen
and kidneys. The segmentation is performed in a top-down
fashion, where major structures are segmented first, and their
location information is then passed down to the lower level to
initialize the segmentation, while boundary information from
higher-level structures also constrains the segmentation of the
lower-level structures. In our preliminary experiments, the
proposed method yielded a Dice coefficient around 90% for
most major thoracic and abdominal organs in both contrast-
enhanced CT and non-enhanced datasets, while the average
running time for segmenting ten organs was about 10 min-
utes.
1 Introduction
Automatic segmentation of anatomical structures has great value for both clinical and epidemiolog-
ical studies. Some common examples include using a brain segmentation tool for quantitative mea-
Copyright c by the paper’s authors. Copying permitted only for private and academic purposes.
In: O. Goksel (ed.): Proceedings of the VISCERAL Organ Segmentation and Landmark Detection Benchmark at
the 2014 IEEE International Symposium on Biomedical Imaging (ISBI), Beijing, China, May 1st , 2014
published at http://ceur-ws.org
25
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
surements of brain structure changes to study Alzheimer’s disease [FSB+ 02], using an automated
lung segmentation method to define the region of interest for computer-aided diagnosis (CAD)
methods for more efficient screening and earlier detection of tumors, and using liver segmentation
for surgery planning to achieve more precise and better cancer treatment [HVGS+ 09]. Besides
these single organ applications, the multi-organ segmentation methods have broader applications,
such as radiotherapy planning and semantic image segmentation and content retrieving [SKM+ 10].
Many automated organ segmentation methods have been proposed in the literature, such as the
active shape model (ASM) [CTCG95], atlas-based methods [ISR+ 09] and machine-learning-based
methods [ZBG+ 08]. The robustness of these single-organ approaches is usually unsatisfactory.
This is related to the fact that the boundary between two organs may be inadequately defined
due to limited resolution and intensity similarity. Even with the help of shape priors, most algo-
rithms still have difficulties in discriminating between another organ and anatomical variation of
the same organ. Recently, a number of multi-organ segmentation approaches have been proposed,
thanks to the improving performance of modern computers and the increasing recognition of the
advantages of considering multi-organ simultaneous in the image models. Okada et al. proposed
a hierarchical organization of organ ASMs [OYH+ 08], where the inter-organ position changing is
decoupled from the individual organs morphological variations. Promising results were obtained in
their upper-abdominal organ segmentation in contrast-enhanced CT scans. Wolz et al. proposed
a hierarchical atlas registration and weighting scheme, which sequentially picks the close-looking
atlases, best-matching organ atlases and best-fitting segmentation patches in a three-level coarse-
to-fine registration pipeline [WCM+ 12]. A few machine learning based methods were also reported
[MSW+ 11, KSZ+ 11]. In [WS14], we proposed an automatic multi-organ segmentation method us-
ing hierarchical-shape-prior guided level sets. The hierarchical shape priors are organized according
to the anatomical hierarchy of the human body, so that the major structures with less population
variation are at the top, and smaller structures with higher irregularities are linked at a lower level.
The segmentation is performed in a top-down fashion, where major structures are segmented first,
and their location information is then passed down to the lower level to initialize the segmentation,
while boundary information from higher-level structures also constrains the segmentation of the
lower-level structures. The proposed method delivered relatively accurate results in non-enhanced
CT datasets [WS14]. In this paper, we extend the framework to process both non-enhanced and
contrast-enhance CT datasets, by introducing an iterative organ intensity estimation step.
2 Methods
Figure 1 summarizes the processing pipeline of the proposed segmentation framework, which can
be roughly divided into three phases: preprocessing, hierarchical shape model guided multi-organ
segmentation and iterative organ intensity estimation. Detailed descriptions of these phases are
given in the following sections.
2.1 Preprocessing
A skin and subcutaneous fat stripping step is first carried out to remove the large variation of the
subcutaneous fat distribution among the population. This is done with a two-step threshold-based
level set segmentation combined with mathematical morphology operations. First, the surface of
the human body is segmented with a threshold of 300 HU and an initial seed region set to cover the
whole volume. The resulting mask is then processed with an erosion operator to remove the skin.
Finally a second round threshold-based level set segmentation is carried out with the threshold
set to 0 HU. After subcutaneous fat stripping, the musculoskeletal figure of a patient tends to
vary less from patient to patient. A straightforward rigid registration is carried out between the
26
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
Figure 1: The processing pipeline of the proposed multi-organ segmentation framework.
unseen patient and a selected standard subject. This standard subject (common-looking subject)
was manually selected by visually comparing the appearance among the sample group. The air-
filled lung areas in both datasets are set to the fat tissue intensity to reduce their influence on the
registration, so the skeletons are better aligned. The transformation matrix from the registration
step is used to initialize the position of the hierarchical shape model. For non-enhanced CT datasets,
a cropping step is introduced to limit the remaining processing to the torso. The largest torso cross-
section area is estimated by finding the largest connected region (2D) within the musculoskeletal
figure among all axial slices. The starting and ending slice of the torso is then defined as the
first slice, on either side of the largest torso cross-section slice, in which the width of the largest
connected region (2D) is below half the width of the largest torso cross-section area.
2.2 Hierarchical shape model guided multi-organ segmentation
The hierarchical shape model used in this study is shown in Figure 2. To generate statistical shape
priors for individual structures, all segmentation masks of the corresponding organ are registered
to the common-looking subject. To link a statistical shape prior to its parent structures space, the
statistical mean shape is registered against a trust zone created by thresholding the probability
atlas of that anatomical structure in the upper-level structures space. More detailed description of
building the hierarchical shape priors can be found in [WS14].
The segmentation is performed in a top-down fashion, i.e. ventral cavity is first segmented, and
then divided into thoracic cavity and abdominopelvic cavity. The third level contains the individual
organs such liver, spleen and kidneys. The location information of a higher level structure is
passed down to the lower level to initialize the segmentation. Within the same level, structures are
segmented sequentially from left to right as the order listed in Figure 2. Segmented regions are set
to different empirically defined likelihood values to guide the following segmentation.
2.3 Iterative organ intensity estimation
In the proposed hierarchical-shape-prior guided level set framework, the external speed function is
an intensity mapping function, which is similar to the threshold function in the threshold-based
level set method proposed by Lefohn et al. [LCW03]. In [WS14], the upper and lower thresholds
are empirically defined beforehand for different structures. Since the intensity of some organs in
contrast-enhanced CT scans can vary depending on the circulation rate and acquisition timing,
we introduced an iterative approach to estimate the intensity range of heart, liver, kidney and
spleen. An organs upper and lower threshold are estimated to be M + 1.5σ and M − 1.5σ, where
M and σ is the mean and standard deviation of the voxel intensity within the current segmented
area. All voxels with intensity lower than 30HU are excluded from the calculation of M and σ.
27
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
Figure 2: The hierarchical shape model used in this study
The intensity estimation is repeated every 15 iterations of the model fitting process. The iterative
intensity estimation stops when the changing rates of M and σ are both lower than a threshold (5
HU). The fixed thresholds reported in [WS14] are used as the initial setting for these organs in the
beginning of organ segmentation.
2.4 Model-guided level set using coherent propagation
In this study, the model-based level set method proposed by Leventon et al. [LGF02](Leventon,
Grimson, and Faugeras 2002) is adapted for individual structure segmentation at different levels.
Making this method efficient and accurate is essential for the usability and robustness of the whole
framework. In our earlier papers [WFS11, WFS14]), we proposed a fast level set method using
coherent propagation, which achieved 10100 times speed-up in various segmentation tasks when
compared with the sparse field level set algorithm. In [WS14], we extended the coherent propagation
method to model-based level sets, which can not only speed up the level set propagation, but also
reduce the frequency of shape-prior registration by taking advantage of the convergence detection
of the coherent propagation. In this new framework, the model fitting operation is only repeated
if the contour has moved a certain distance from the previously estimated model.
3 Results
.The proposed method was trained on 7 training CT datasets and tested on 5 non-enhanced CT
datasets and 5 contrast-enhanced CT datasets. These CT images are down-sampled to 333 mm
resolution, whereas the segmentation results are up-sampled to the original resolution for evalua-
tion. All these datasets were obtained from the Visceral Benchmark 1 site (visceral.eu) [HML+ 12].
Overall, the proposed method yielded a Dice coefficient around 90% for most major organs. De-
tailed results are listed in Table 1. The average processing time for segmenting all ten major organs
is about 10 minutes (excluding the resampling steps) on an 8-core Mac Pro (2.26GHz). Figure 3
shows an example of the segmentation results at different stages from one testing dataset.
28
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
Table 1: SEGMENTATION RESULTS
Organ Non-enhanced CT Contrast-enhanced CT
Name Dice Average Dice Average
coefficient Hausdorff coefficient Hausdorff
(%) distance (%) distance
Liver 0.904 0.46 0.887 0.65
Spleen 0.887 0.45 0.842 0.87
Left lung 0.971 0.07 0.956 0.15
Right lung 0.972 0.06 0.942 0.20
Left kidney 0.729 3.63 0.896 0.27
Right kidney 0.777 1.21 0.890 0.28
Bladder 0.806 0.78 0.738 1.59
Figure 3: Segmentation results at different stages. A, segmentation result after skin and subcu-
taneous fat stripping; B, segmentation result of the ventral cavity; C, segmentation result of the
second level structures; D, segmentation results of the third level structures.
4 Discussion and Conclusion
The proposed segmentation method has a number of limitations. First, the statistical shape priors
for different structures were trained on 7 subjects, which can over-constrain the segmented area (cf.
liver segmentation in Figure 3D). Second, as the top-down strategy suffers from the accumulated
error being passed down along the hierarchy tree, a bottom-up feedback path should be added
to allow the lower structure to recover the higher level errors. Future work also includes improv-
ing segmentation accuracy by using more edge-based image terms and extending the framework
to handle MRI images. In conclusion, a multi-organ segmentation framework using hierarchical
shape priors is presented. This method gradually improves the estimation of the organ location by
first segmenting out large and regular-shaped structures. The appearance of organs is iteratively
estimated based on statistical analysis of preliminary segmentation results. Preliminary results on
non-enhanced and contrast-enhance CT datasets are encouraging.
References
[CTCG95] Timothy F. Cootes, Christopher J. Taylor, David H. Cooper, and Jim Graham. Active
shape models-their training and application. Computer vision and image understand-
ing, 61(1):3859, 1995.
[FSB+ 02] Bruce Fischl, David H. Salat, Evelina Busa, Marilyn Albert, Megan Dieterich, Chris-
tian Haselgrove, Andre van der Kouwe, Ron Killiany, David Kennedy, Shuna Klave-
29
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
ness, Albert Montillo, Nikos Makris, Bruce Rosen, and Anders M. Dale. Whole brain
segmentation: Automated labeling of neuroanatomical structures in the human brain.
Neuron, 33(3):341–355, January 2002.
[HML+ 12] Allan Hanbury, Henning Mller, Georg Langs, Marc Andr Weber, Bjoern H. Menze,
and Tomas Salas Fernandez. Bringing the algorithms to the data: Cloudbased bench-
marking for medical image analysis. In Information Access Evaluation. Multilinguality,
Multimodality, and Visual Analytics, page 2429. Springer, 2012.
[HVGS+ 09] T. Heimann, B. Van Ginneken, M.A. Styner, Y. Arzhaeva, V. Aurich, C. Bauer,
A. Beck, C. Becker, R. Beichel, G. Bekes, F. Bello, G. Binnig, H. Bischof, A. Bornik,
P. Cashman, Ying Chi, A. Cordova, B.M. Dawant, M. Fidrich, J.D. Furst, D. Fu-
rukawa, L. Grenacher, J. Hornegger, D. Kainmuller, R.I. Kitney, H. Kobatake,
H. Lamecker, T. Lange, Jeongjin Lee, B. Lennon, Rui Li, Senhu Li, H.-P. Meinzer,
G. Nemeth, D.S. Raicu, A.-M. Rau, E.M. van Rikxoort, M. Rousson, L. Rusko, K.A.
Saddi, G. Schmidt, D. Seghers, A. Shimizu, P. Slagmolen, E. Sorantin, G. Soza, R. Su-
somboon, J.M. Waite, A. Wimmer, and I. Wolf. Comparison and evaluation of meth-
ods for liver segmentation from CT datasets. IEEE Transactions on Medical Imaging,
28(8):1251–1265, 2009.
[ISR+ 09] I. Isgum, M. Staring, A. Rutten, M. Prokop, M.A. Viergever, and B. van Ginneken.
Multi-atlas-based segmentation with local decision fusion-application to cardiac and
aortic segmentation in CT scans. IEEE Transactions on Medical Imaging, 28(7):1000–
1010, July 2009.
[KSZ+ 11] Timo Kohlberger, Michal Sofka, Jingdan Zhang, Neil Birkbeck, Jens Wetzl, Jens Kaf-
tan, Jrme Declerck, and S. Kevin Zhou. Automatic multi-organ segmentation using
learning-based segmentation and level set optimization. In Medical Image Computing
and Computer-Assisted InterventionMICCAI 2011, page 338345. Springer, 2011.
[LCW03] A. Lefohn, J. Cates, and R. Whitaker. Interactive, gpu-based level sets for 3D seg-
mentation. In Proceedings of the MICCAI conference, page 564572, 2003.
[LGF02] M. E Leventon, W. E.L Grimson, and O. Faugeras. Statistical shape influence in
geodesic active contours. In Computer Vision and Pattern Recognition, 2000. Proceed-
ings. IEEE Conference on, volume 1, page 316323, 2002.
[MSW+ 11] Albert Montillo, Jamie Shotton, John Winn, Juan Eugenio Iglesias, Dimitri Metaxas,
and Antonio Criminisi. Entangled decision forests and their application for semantic
segmentation of CT images. In Gbor Szkely and Horst K. Hahn, editors, Information
Processing in Medical Imaging, number 6801 in Lecture Notes in Computer Science,
pages 184–196. Springer Berlin Heidelberg, January 2011.
[OYH+ 08] Toshiyuki Okada, Keita Yokota, Masatoshi Hori, Masahiko Nakamoto, Hironobu Naka-
mura, and Yoshinobu Sato. Construction of hierarchical multi-organ statistical atlases
and their application to multi-organ segmentation from CT images. In Medical Image
Computing and Computer-Assisted InterventionMICCAI 2008, page 502509. Springer,
2008.
[SKM+ 10] Sascha Seifert, Michael Kelm, Manuel Moeller, Saikat Mukherjee, Alexander Cavallaro,
Martin Huber, and Dorin Comaniciu. Semantic annotation of medical images. In SPIE
Medical Imaging, page 762808762808, 2010.
30
Wang and Smedby: Model Based Level Set and Hierarchical Shape Priors
[WCM+ 12] Robin Wolz, Chengwen Chu, Kazunari Misawa, Kensaku Mori, and Daniel Rueckert.
Multi-organ abdominal CT segmentation using hierarchically weighted subject-specific
atlases. In Medical Image Computing and Computer-Assisted InterventionMICCAI
2012, page 1017. Springer, 2012.
[WFS11] Chunliang Wang, Hans Frimmel, and Örjan Smedby. Level-set based vessel segmenta-
tion accelerated with periodic monotonic speed function. In Proceedings of the SPIE
Medical Imaging Conference, pages 79621M–79621M–7, 2011.
[WFS14] Chunliang Wang, Hans Frimmel, and Örjan Smedby. Fast level-set based image seg-
mentation using coherent propagation. Medical Physics, 41(7):073501, 2014.
[WS14] Chunliang Wang and Örjan Smedby. Automatic multi-organ segmentation in non-
enhanced CT datasets using hierarchical shape priors. In ICPR2014 Acepted, 2014.
[ZBG+ 08] Yefeng Zheng, Adrian Barbu, Bogdan Georgescu, Michael Scheuering, and Dorin Co-
maniciu. Four-chamber heart modeling and automatic segmentation for 3-d cardiac
CT volumes using marginal space learning and steerable features. Medical Imaging,
IEEE Transactions on, 27(11):16681681, 2008.
31