This work presents a method for multi-organ segmentation in whole-body CT images based on a multi-atlas approach. A robust and e cient feature-based registration technique is developed which uses sparse organ speci c features that are learnt based on their ability to register di erent organ types accurately. The best tted feature points are used in RANSAC to estimate an a ne transformation, followed by a thin plate spline re nement. This yields an accurate and reliable nonrigid transformation for each organ, which is independent of initialization and hence does not su er from the local minima problem. Further, this is accomplished at a fraction of the time required by intensity-based methods. The technique is embedded into a standard multi-atlas framework using label transfer and fusion, followed by a random forest classi er which produces the data term for the nal graph cut segmentation. For a majority of the classes our approach outperforms the competitors at the VISCERAL Anatomy Grand Challenge on segmentation at ISBI 2015.
Segmentation is a key problem in medical image analysis, and may be used for numerous applications in medical research and clinical care. In this paper, a pipeline for the segmentation of whole-body CT images into 20 di erent organs is presented. The approach is based on multi-atlas segmentation, see [KSK+10, HKA+10, WSD+13] and the references therein, an approach which is known to produce state-of-the-art results for several segmentation tasks. The method requires pair-wise registrations from a set of atlas images to the unknown target image.
In principle, there are two di erent approaches to image registration, feature-based and intensitybased registration, see the surveys [KBG+11, SDP13]. Intensity-based methods are capable of producing accurate registrations but are sensitive to initialization and often slow. Feature-based methods are usually faster, but may risk failing due to many outlier correspondences between the images. Our approach is an adapted feature-based method that utilizes the speed of general feature-based methods while trying to eliminate the risk of establishing incorrect point-to-point correspondences between the images by identifying reliable feature points. We show that reliable organ localization can be computed using (i) robust optimization techniques and (ii) learned feature correspondences. 2
Our system segments each organ independently of each other using a multi-atlas approach. The pipeline has three steps:
3. Graph cut segmentation with a Potts model.
These steps will now be described in more detail.
1. Feature-based registration with RANSAC. In order to register an atlas image to the target, a feature-based approach is used. Sparse features are extracted according to Svarm et al. [SEKO15], which uses a method similar to SIFT for feature detection and SURF for feature description. Typically around 8,000{10,000 features are extracted from a 512 512 800 CT image, which takes less than 30s. Correspondences are obtained by matching a subset of the features in the atlas image to the features in the target. The matching is done with a symmetric neighbor approach, where each descriptor is matched to its nearest neighbor. The organ-speci c subset of atlas features is determined as a pre-processing step in the following way. For each atlas image and organ, golden transformations are established to the other atlas images using the ground truth segmentations. Then, based on these transformations, one can check which features in the atlas image of interest that are most accurate, and rank the features accordingly. We have found empirically that using the top 300 best features for each organ provides robust and reliable registration. Standard RANSAC with the truncated `2 as cost function is used in order to remove outliers. The optimization is run 500; 000 iterations and the truncation threshold was set to 30 mm. See Figure 1 for an example.
Finally, a coordinate transformation from the atlas to the target image is computed by applying thin plate splines (TPS) to the remaining correspondences, and thereafter used in order to transfer the labels of the atlas to the target image. The thin plate spline method proposed in [CR00] was used. One registration takes less than 10s in total. See Figure 2 for an example.
For several of the organs, the registrations are re ned with a standard intensity-based method. More speci cally, we used NiftyReg [ORS+01] which takes around 100{200s per registration.
2. Label fusion with a random forest classi er. The pairwise feature-based registrations give us a rough idea where the organ is located. In order to fuse the transferred labels, one can compute an average voxel map, denoted P . Hence, for voxel i, the map P (i) gives a number between 0 and 1 which can (intuitively) be interpreted as the probability of voxel i belonging to the organ. For example, if half of the atlas images say that voxel i should be organ, then P (i) = 0:5.
The map P largely ignores the local appearance around the target organ and in order to improve the accuracy of the estimate, the map P along with a few other features is used to train a random forest classi er. We use Sherwood [CSK11] to train and evaluate large random forest instances e ciently. Given a target volume I and the map P , we begin by smoothing both using a Gaussian kernel with standard deviation = 1, resulting in two new volumes denoted Is and Ps. Using cross validation we also determine a threshold level, , for P and construct a distance map D, where each voxel in D equals the (signed) distance to the boundary surface of the binary volume P > , that is, the map P thresholded at . For each volume I we thus obtain 5 features per voxel i: I(i), Is(i), P (i), Ps(i) and Pt(i). The output of the classi er produces yet another map, denoted Pr, which is a re ned estimate of the location of the organ. As previously, Pr can be interpreted as the probability for each voxel belonging to the organ of interest. See Figure 3 for an example.
3. Graph cut segmentation with a Potts model. The random forest classi er generates a new estimate Pr(i) for each voxel i, but each decision in the classi er is taken independently of the output of the neighboring voxels of i. This has a tendency of producing noisy boundary estimates and therefore we will regularize the solution by using a standard Potts model. The nal solution can then by computed with graph-cuts.
GT x?Pr x?
P
The Potts model penalizes neighboring voxels if they take di erent labels. Let xi be a Boolean indicator variable for voxel i, i.e., xi 2 f0; 1g. Then, for two neighboring voxels xi and xj , the cost should be zero if xi = xj and otherwise, where is a positive scalar. This cost can compactly be written as xi(1 xj ). Further, the data cost for voxel i is set to take value 1=2 Pr(i) if xi = 1 and zero otherwise. This favors voxels with probabilities in the interval [0:5; 1] to be foreground and voxels with [0; 0:5] to be background.
In summary the nal segmentation, x?, is given by the solution to the optimization problem: n 1 x? = argmin X xi 2 x2f0;1gn i=1
n X X i=1 j2N (i) Pr(i) + ij xi(1 xj ); (1) where is a regularization weight and ij compensates for anisotropic resolution. For all organs we use a 6-connected neighborhood N . In order to save memory and speed-up calculations we only process a volume around the zero level of the distance map D with a 20 voxels margin. The function in (1) is submodular and is minimized e ciently using the graph-cut implementation of [JSH12]. 3
All the tuning parameters in our system have been set by leave-one out cross validation on the rst 15 of the 20 whole-body CT images available in the VISCERAL challenge. The 5 remaining images have been used to validate the performance of the random forest classi er and the graph-cut segmentation. In the training phase, the rst 15 images have served as the atlas set, while in the nal version all 20 images are utilized in the atlas.
Our system has been evaluated on a test set of 10 whole-body CT images by the organizers of the VISCERAL Anatomy Grand Challenge at ISBI 2015. Note that this test set is only available to the organizers. The nal results are given in Table 1 together with the best competitors to date: CMIV - \Center for Medical Image Science and Visualization, Linkoping University", HES-SO - \University of Applied Sciences Western Switzerland" and
SIAT - \Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences". [CR00] Left Kidney Right Kidney Spleen Liver Left Lung Right Lung Unirary Bladder Muscle Body of Left Rectus Abdominis Muscle Body of Right Rectus Abdominis Lumbar Vertebra 1 Thyroid Pancreas Left Psoas Major Muscle Right Psoas Major Muscle Gallbladder Sternum Aorta Trachea Left Adrenal Gland Right Adrenal Gland Average
Our 0.784 0.790 0.703 0.866 0.972 0.975 0.698 0.551 0.519 0.718 0.549 0.408 0.806 0.787 0.276 0.761 0.753 0.92 0.373 0.355 0.678 0.874 0.923 0.952 0.957
We have demonstrated that by using a feature-based approach to multi-atlas segmentation, it is possible to reliably locate and segment organs in whole-body CT images with state-of-the-art results. Still, there is room for improvement. For example, there is no guarantee that the system produces a valid organ shape. We are currently working on ways to directly incorporate such shape priors in the framework. Further, the speed of the system can be improved, for example, by circumventing the need to perform 20 pairwise registrations for every new target image [ANEK15]. [ANEK15] J. Alven, A. Norlen, O. Enqvist, and F. Kahl. Uberatlas: Robust speed-up of featurebased registration and multi-atlas segmentation. Scandinavian Conference on Image Analysis, 2015. To appear.
H. Chui and A. Rangarajan. A new algorithm for non-rigid point matching. IEEE Conference on Computer Vision and Pattern Recognition, 2:44{51, 2000.
A. Criminisi, J. Shotton, and E. Konukoglu. Decision forests: A uni ed framework for classi cation, regression, density estimation, manifold learning and semi-supervised learning. Foundations and Trends R in Computer Graphics and Vision, 7:81{227, 2011. [HKA+10] RA. Heckemann, S. Keihaninejad, P. Aljabar, D. Rueckert, JV. Hajnal, and A. Hammers. Improving intersubject image registration using tissue-class information bene ts robustness and accuracy of multi-atlas based anatomical segmentation. NeuroImage, 51(1):221{227, 2010. [JSH12]
O. Jamriska, D. Sykora, and A. Hornung. Cache-e cient graph cuts on structured grids.
IEEE Conference on Computer Vision and Pattern Recognition, pages 3673{3680, 2012.