An automatic multi-organ segmentation pipeline is presented. The segmentation starts with stripping the body of skin and subcutaneous fat using threshold-based level-set methods. After registering the image to be processed against a standard subject picked from the training datasets, a series of model-based level set segmentation operations is carried out guided by hierarchical shape priors. The hierarchical shape priors are organized according to the anatomical hierarchy of the human body, starting with ventral cavity, and then divided into thoracic cavity and abdominopelvic cavity. The third level contains the individual organs such as liver, spleen and kidneys. The segmentation is performed in a top-down fashion, where major structures are segmented rst, and their location information is then passed down to the lower level to initialize the segmentation, while boundary information from higher-level structures also constrains the segmentation of the lower-level structures. In our preliminary experiments, the proposed method yielded a Dice coe cient around 90% for most major thoracic and abdominal organs in both contrastenhanced CT and non-enhanced datasets, while the average running time for segmenting ten organs was about 10 minutes.
Automatic segmentation of anatomical structures has great value for both clinical and epidemiological studies. Some common examples include using a brain segmentation tool for quantitative measurements of brain structure changes to study Alzheimer's disease [FSB+02], using an automated lung segmentation method to de ne the region of interest for computer-aided diagnosis (CAD) methods for more e cient screening and earlier detection of tumors, and using liver segmentation for surgery planning to achieve more precise and better cancer treatment [HVGS+09]. Besides these single organ applications, the multi-organ segmentation methods have broader applications, such as radiotherapy planning and semantic image segmentation and content retrieving [SKM+10]. Many automated organ segmentation methods have been proposed in the literature, such as the active shape model (ASM) [CTCG95], atlas-based methods [ISR+09] and machine-learning-based methods [ZBG+08]. The robustness of these single-organ approaches is usually unsatisfactory. This is related to the fact that the boundary between two organs may be inadequately de ned due to limited resolution and intensity similarity. Even with the help of shape priors, most algorithms still have di culties in discriminating between another organ and anatomical variation of the same organ. Recently, a number of multi-organ segmentation approaches have been proposed, thanks to the improving performance of modern computers and the increasing recognition of the advantages of considering multi-organ simultaneous in the image models. Okada et al. proposed a hierarchical organization of organ ASMs [OYH+08], where the inter-organ position changing is decoupled from the individual organs morphological variations. Promising results were obtained in their upper-abdominal organ segmentation in contrast-enhanced CT scans. Wolz et al. proposed a hierarchical atlas registration and weighting scheme, which sequentially picks the close-looking atlases, best-matching organ atlases and best- tting segmentation patches in a three-level coarseto- ne registration pipeline [WCM+12]. A few machine learning based methods were also reported [MSW+11, KSZ+11]. In [WS14], we proposed an automatic multi-organ segmentation method using hierarchical-shape-prior guided level sets. The hierarchical shape priors are organized according to the anatomical hierarchy of the human body, so that the major structures with less population variation are at the top, and smaller structures with higher irregularities are linked at a lower level. The segmentation is performed in a top-down fashion, where major structures are segmented rst, and their location information is then passed down to the lower level to initialize the segmentation, while boundary information from higher-level structures also constrains the segmentation of the lower-level structures. The proposed method delivered relatively accurate results in non-enhanced CT datasets [WS14]. In this paper, we extend the framework to process both non-enhanced and contrast-enhance CT datasets, by introducing an iterative organ intensity estimation step. 2
A skin and subcutaneous fat stripping step is rst carried out to remove the large variation of the subcutaneous fat distribution among the population. This is done with a two-step threshold-based level set segmentation combined with mathematical morphology operations. First, the surface of the human body is segmented with a threshold of 300 HU and an initial seed region set to cover the whole volume. The resulting mask is then processed with an erosion operator to remove the skin. Finally a second round threshold-based level set segmentation is carried out with the threshold set to 0 HU. After subcutaneous fat stripping, the musculoskeletal gure of a patient tends to vary less from patient to patient. A straightforward rigid registration is carried out between the unseen patient and a selected standard subject. This standard subject (common-looking subject) was manually selected by visually comparing the appearance among the sample group. The airlled lung areas in both datasets are set to the fat tissue intensity to reduce their in uence on the registration, so the skeletons are better aligned. The transformation matrix from the registration step is used to initialize the position of the hierarchical shape model. For non-enhanced CT datasets, a cropping step is introduced to limit the remaining processing to the torso. The largest torso crosssection area is estimated by nding the largest connected region (2D) within the musculoskeletal gure among all axial slices. The starting and ending slice of the torso is then de ned as the rst slice, on either side of the largest torso cross-section slice, in which the width of the largest connected region (2D) is below half the width of the largest torso cross-section area. 2.2
The hierarchical shape model used in this study is shown in Figure 2. To generate statistical shape priors for individual structures, all segmentation masks of the corresponding organ are registered to the common-looking subject. To link a statistical shape prior to its parent structures space, the statistical mean shape is registered against a trust zone created by thresholding the probability atlas of that anatomical structure in the upper-level structures space. More detailed description of building the hierarchical shape priors can be found in [WS14].
The segmentation is performed in a top-down fashion, i.e. ventral cavity is rst segmented, and then divided into thoracic cavity and abdominopelvic cavity. The third level contains the individual organs such liver, spleen and kidneys. The location information of a higher level structure is passed down to the lower level to initialize the segmentation. Within the same level, structures are segmented sequentially from left to right as the order listed in Figure 2. Segmented regions are set to di erent empirically de ned likelihood values to guide the following segmentation. 2.3
In the proposed hierarchical-shape-prior guided level set framework, the external speed function is an intensity mapping function, which is similar to the threshold function in the threshold-based level set method proposed by Lefohn et al. [LCW03]. In [WS14], the upper and lower thresholds are empirically de ned beforehand for di erent structures. Since the intensity of some organs in contrast-enhanced CT scans can vary depending on the circulation rate and acquisition timing, we introduced an iterative approach to estimate the intensity range of heart, liver, kidney and spleen. An organs upper and lower threshold are estimated to be M + 1:5 and M 1:5 , where M and is the mean and standard deviation of the voxel intensity within the current segmented area. All voxels with intensity lower than 30HU are excluded from the calculation of M and . The intensity estimation is repeated every 15 iterations of the model tting process. The iterative intensity estimation stops when the changing rates of M and are both lower than a threshold (5 HU). The xed thresholds reported in [WS14] are used as the initial setting for these organs in the beginning of organ segmentation. 2.4
In this study, the model-based level set method proposed by Leventon et al. [LGF02](Leventon, Grimson, and Faugeras 2002) is adapted for individual structure segmentation at di erent levels. Making this method e cient and accurate is essential for the usability and robustness of the whole framework. In our earlier papers [WFS11, WFS14]), we proposed a fast level set method using coherent propagation, which achieved 10100 times speed-up in various segmentation tasks when compared with the sparse eld level set algorithm. In [WS14], we extended the coherent propagation method to model-based level sets, which can not only speed up the level set propagation, but also reduce the frequency of shape-prior registration by taking advantage of the convergence detection of the coherent propagation. In this new framework, the model tting operation is only repeated if the contour has moved a certain distance from the previously estimated model. 3
.The proposed method was trained on 7 training CT datasets and tested on 5 non-enhanced CT datasets and 5 contrast-enhanced CT datasets. These CT images are down-sampled to 333 mm resolution, whereas the segmentation results are up-sampled to the original resolution for evaluation. All these datasets were obtained from the Visceral Benchmark 1 site (visceral.eu) [HML+12]. Overall, the proposed method yielded a Dice coe cient around 90% for most major organs. Detailed results are listed in Table 1. The average processing time for segmenting all ten major organs is about 10 minutes (excluding the resampling steps) on an 8-core Mac Pro (2.26GHz). Figure 3 shows an example of the segmentation results at di erent stages from one testing dataset.
Spleen Left lung Right lung Left kidney Right kidney Bladder The proposed segmentation method has a number of limitations. First, the statistical shape priors for di erent structures were trained on 7 subjects, which can over-constrain the segmented area (cf. liver segmentation in Figure 3D). Second, as the top-down strategy su ers from the accumulated error being passed down along the hierarchy tree, a bottom-up feedback path should be added to allow the lower structure to recover the higher level errors. Future work also includes improving segmentation accuracy by using more edge-based image terms and extending the framework to handle MRI images. In conclusion, a multi-organ segmentation framework using hierarchical shape priors is presented. This method gradually improves the estimation of the organ location by rst segmenting out large and regular-shaped structures. The appearance of organs is iteratively estimated based on statistical analysis of preliminary segmentation results. Preliminary results on non-enhanced and contrast-enhance CT datasets are encouraging. [FSB+02]
Bruce Fischl, David H. Salat, Evelina Busa, Marilyn Albert, Megan Dieterich, Christian Haselgrove, Andre van der Kouwe, Ron Killiany, David Kennedy, Shuna Klave[HML+12] [ISR+09] [KSZ+11]
I. Isgum, M. Staring, A. Rutten, M. Prokop, M.A. Viergever, and B. van Ginneken. Multi-atlas-based segmentation with local decision fusion-application to cardiac and aortic segmentation in CT scans. IEEE Transactions on Medical Imaging, 28(7):1000{ 1010, July 2009.
Timo Kohlberger, Michal Sofka, Jingdan Zhang, Neil Birkbeck, Jens Wetzl, Jens Kaftan, Jrme Declerck, and S. Kevin Zhou. Automatic multi-organ segmentation using learning-based segmentation and level set optimization. In Medical Image Computing and Computer-Assisted InterventionMICCAI 2011, page 338345. Springer, 2011. A. Lefohn, J. Cates, and R. Whitaker. Interactive, gpu-based level sets for 3D segmentation. In Proceedings of the MICCAI conference, page 564572, 2003.
M. E Leventon, W. E.L Grimson, and O. Faugeras. Statistical shape in uence in geodesic active contours. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, volume 1, page 316323, 2002. [MSW+11] Albert Montillo, Jamie Shotton, John Winn, Juan Eugenio Iglesias, Dimitri Metaxas, and Antonio Criminisi. Entangled decision forests and their application for semantic segmentation of CT images. In Gbor Szkely and Horst K. Hahn, editors, Information Processing in Medical Imaging, number 6801 in Lecture Notes in Computer Science, pages 184{196. Springer Berlin Heidelberg, January 2011. [OYH+08]
Toshiyuki Okada, Keita Yokota, Masatoshi Hori, Masahiko Nakamoto, Hironobu Nakamura, and Yoshinobu Sato. Construction of hierarchical multi-organ statistical atlases and their application to multi-organ segmentation from CT images. In Medical Image Computing and Computer-Assisted InterventionMICCAI 2008, page 502509. Springer, 2008. [SKM+10] Sascha Seifert, Michael Kelm, Manuel Moeller, Saikat Mukherjee, Alexander Cavallaro, Martin Huber, and Dorin Comaniciu. Semantic annotation of medical images. In SPIE Medical Imaging, page 762808762808, 2010. [WFS11] [ZBG+08]
Chunliang Wang, Hans Frimmel, and O rjan Smedby. Level-set based vessel segmentation accelerated with periodic monotonic speed function. In Proceedings of the SPIE Medical Imaging Conference, pages 79621M{79621M{7, 2011.
Chunliang Wang, Hans Frimmel, and O rjan Smedby. Fast level-set based image segmentation using coherent propagation. Medical Physics, 41(7):073501, 2014.
Chunliang Wang and O rjan Smedby. Automatic multi-organ segmentation in nonenhanced CT datasets using hierarchical shape priors. In ICPR2014 Acepted, 2014. Yefeng Zheng, Adrian Barbu, Bogdan Georgescu, Michael Scheuering, and Dorin Comaniciu. Four-chamber heart modeling and automatic segmentation for 3-d cardiac CT volumes using marginal space learning and steerable features. Medical Imaging, IEEE Transactions on, 27(11):16681681, 2008.