10. CURAC-Jahrestagung, 15. - 16. September 2011, Magdeburg Evaluation of 3D Ultrasound Image Registration E. Efstathiou¹, T.M. Deserno², C. Münzenmayer¹, T. Wittenberg¹, T. Bergen¹ ¹ Fraunhofer Institute for Integrated Circuits IIS, Erlangen, Germany ² RWTH Aachen University, Aachen, Germany Contact: efstates@iis.fraunhofer.de Abstract: Image registration plays a crucial role for the accurate reconstruction of an organ from partial ultrasound volumes and the subsequent accurate resection of a lesion/tumor with an optimally minimal damage of the healthy tissue. With the help of the Insight Toolkit (ITK), various state-of-the-art voxel-based 3D image registration algorithms were inves- tigated, implemented and evaluated, allowing for the assessment of an accurate ultrasound image registration scheme. The investigation of the 3D space was based on an investigation of the 2D space, where the image registration compo- nents showing low performance were sorted out. The performance was assessed by calculating the standard deviation (SD) of the resulting difference images. Overall the mutual information and joint histogram based metrics showed low performance (2D - SD up to 25,9), whereas the Powell direction set algorithm in combination with the mean squares metric showed a better performance (3D - SD: 22,4). Keywords: 3D ultrasound, image registration, evaluation 1 Problem Minimally invasive surgery (MIS) and similar technologies, such as NOTES (Natural Orifice Transluminal Endoscopic Surgery), LESS (Laparo-Endoscopic Single Site Surgery) or SILS (Single Incision Laparoscopic Surgery), are known to reduce the trauma for the patients. Nevertheless, the small incisions made to the patient limit also the orientation, the visual range as well as the maneuverability of the surgeon. In order to support the surgeon with navigation assistance and intra-operative lesion localization capability during a minimally invasive tissue resection, a detailed spatial repre- sentation of the organ under examination is necessary. Especially, in the case of a liver lesion resection, an intra- operative reconstruction of the liver from 2D/3D ultrasound images can provide additional information during the in- tervention, which provide the surgeon support and the capability of comparing pre-operative data (e.g. liver CT) with intra-operative data (US). In this assignment the registration of ultrasound volumes plays a crucial role, e.g. for volume stitching to represent the entire organ. Furthermore, the role of accurate ultrasound registration is also highlighted in the area of the geometric calibration of an electromagnetically tracked ultrasound system [1], where it determines the qual- ity of the pursued calibration. In this paper, we focus on comparing different image registration schemes for volumetric ultrasound registration. 2 Methods In order to assess for an optimal image registration method for accurate ultrasound image registration, several initial experiments were conducted in the 2D space, investigating various combinations of metrics and optimizers. Based on these initial experiments, optimizer-metric combinations with low performance were excluded from the experiments in the 3D space. Due to the common characteristics of the 2D and 3D ultrasound images, we assume that this convention is valid. The investigation of various metrics allows the assessment of the quality of the representation of the similarity of two images. Additionally, the investigation of various optimizers allows the assessment of the quality of the optimi- zation succeeded, i.e. how good the optimizer suits to the metric and guides itself to the extremum of the metric. The experiments were succeeded through the development of an image registration framework, which utilizes already implemented image registration components by the Insight Toolkit (ITK). This accounts for robustness of the algo- rithms used and minimization of the researcher bias and promotes the reproducibility of the results. The experiments investigated the performance of a registration with respect to both the choice of a metric and an opti- mizer (Figure 1). Their influence on the registration result is considered much more significant than the influence of the interpolator. For this reason, a linear interpolator was utilized, because of its very good quality to computational load ratio. In addition, the transformation between successive acquisitions can be theoretically given by a translation or rigid 17 10. CURAC-Jahrestagung, 15. - 16. September 2011, Magdeburg transformation, but due to ultrasound imaging inaccuracies and taking into account that the non-deformable-body con- straint is well fulfilled for the utilized liver phantom, affine transformations were applied. Although the results of the current study cannot be directly applied to clinical cases, where deformable transformations have to be considered, they provide important information for applications, where complex multiple level image registration schemes are used. Bi- nary masking was also used in order to determine the regions of interest within the images under registration. Fixed Image Metric Optimizer Interpolator Moving Image Transformation Fig. 1: Image registration algorithm: a) transformation: maps the moving on the fixed image, b) interpolator: maps the non-grid positions of the transformed moving image on the grid positions of the fixed image, c) metric: evaluates the quality of the registration, d) optimizer: minimizes/maximizes the metric in the transformation parameter space In particular, the selection of optimizers and metrics was based on the state-of-the-art ultrasound image registration al- gorithms [2, 3, 5], according to an extended literature research. A list of the optimizers and metrics that were examined during our study is shown in Table 1. Optimizers Metrics Mean Squares (MS) Nelder-Mead downhill simplex (Amoeba) Normalized Cross Correlation (NCC) Powell direction set Correlation Coefficient (CC) LBFGS (quasi-Newton) Histogram-based Mutual Information (HMI) Polak-Ribiere (conjugate gradient) Histogram-based Normalized Mutual Information (HNMI) Regular Step Gradient Descent (RSGD) Mattes Mutual Information (MMI) Table 1: Optimizers and metrics that were used in the current study Fig. 2: Examples of 2D and 3D ultrasound images and the respective difference images 18 10. CURAC-Jahrestagung, 15. - 16. September 2011, Magdeburg Due to its high computation times and high susceptibility to noise the Viola-Wells mutual information was excluded from the research. Histogram binning with 64 bins was used in the case of the joint histogram-based metrics. Based on an evaluation of similarity measures for subtraction radiology [4], the quality of the registration was assessed by the calculation of the standard deviation (SD) of the resulting difference image, in accordance with an additional literature research. The 2D and 3D ultrasound images used referred to respective freehand ultrasound acquisitions of a liver phantom (Kyoto Kagaku IOUSFAN) acquired with identical settings with an ALOKA ProSound α7 ultrasound system. In 2D, the ultrasound images were pre-processed with a Gaussian filter with variance 2,0 . Since the Gaussian filter is a typical low-pass filter, it suppresses the speckle only partially, in comparison to filters specially developed for speckle suppression. For two manually chosen pairs of ultrasound images (with small and large displacements respectively), the experiments in 2D concerned the convergence of an optimizer to a metric from a 7  7 grid of different start points cen- tered on the extremum of the metric lying on intervals of 10 pixels. This results in 2940 ultrasound image registration experiments (2 image pairs  49 starting points  5 optimizers  6 metrics). In 3D, a homogeneous three-level image resolution pyramid with downsampling factors of 8, 4 and 2 per level with prior variable Gaussian filtering was utilized. Here, the experiments concerned the registration of a pool of eleven ultra- sound images with each other initially aligned on their centers of mass. 3 Results As an image registration quality measure, the standard deviation of the difference image computed over the overlapping region of the moving and fixed images was calculated for all the resulting difference images. The diagram given in Fig- ure 3 displays the mean standard deviations of the resulting difference images of the 2D registration experiments for every combination of optimizers and metrics. Best results were obtained with the LBFGS optimizer and the MS metric (SD: 19,9), while the same optimizer and the HNMI metric performed worst (SD: 25,9). Fig. 3: Mean values of the standard deviations of the resulting differ- ence images for all optimizer-metric combinations for the 2D registra- tion experiments With regard to the 2D registration experiments, the most robust optimizer-metric combinations of the 2D experiments, Fig. 4: Mean values of the standard deviations of the resulting differ- ence images for various optimizer-metric combinations for the 3D registration experiments 19 10. CURAC-Jahrestagung, 15. - 16. September 2011, Magdeburg namely the mean squares and normalized cross correlation metrics and all the optimizers (apart from the Nelder-Mead downhill simplex) yielded good results and were further investigated in the 3D registration experiments. The diagram given in Figure 4 displays the mean standard deviations of the resulting difference images of the 3D registration ex- periments for every combination of the optimizers, except for Amoeba, and the MS and NCC metrics. Best results were obtained with the Powell optimizer and the MS metric (SD: 22,4), while the LBFGS optimizer and the NCC metric per- form worst (SD: 26,6). With respect to the computation times, a 2D image registration experiment has an approximate duration of less than 5min (image size 800  600 ), whereas a 3D image registration experiment has an approximate duration of 7 to 12min (image size 281  211 254 ). In both cases the full image content was used after applying binary masking. 4 Discussion Figure 3 shows that the mutual information based metrics tend to perform worse than the others regardless of the opti- mizer, because of the remaining speckle, which makes these metrics noisy and creates local extrema for the optimizers to get possibly trapped into. In detail, the Mattes mutual information metric performs far better than the other two mu- tual information based metrics, thanks to its independence from the joint histogram, whose calculation is severely af- fected by the unsuppressed speckle. In addition, the mean squares and normalized correlation metrics perform quite similarly, whereas the correlation coefficient metric performs slightly worse, because of the dependence of its imple- mentation on the joint histogram. Furthermore, a careful observation of the diagram of Figure 4 shows that the mean squares metric performs overall bet- ter than the normalized cross correlation with every optimizer. The diagram confirms that normalized cross correlation is highly sensitive to the changes caused by affine transformations. In addition, the Powell direction set algorithm per- forms far better than the others, since it is not dependent on any derivatives, which may guide the optimizer to local ex- trema, since their calculation is affected by speckle. On the other hand, the LBFGS algorithm converges quickly to a solution, thanks to the Newton direction, which provides quadratic convergence depending on the metric. In a nutshell, the 3D ultrasound image registration is succeeded with the use of appropriate preprocessing and masking and the utilization of a rough-to-fine multi-resolution image registration scheme and it is accurate enough to be utilized in geometric calibration of electromagnetically tracked ultrasound systems. The results are promising to be applied to minimal invasive surgery under consideration of an extension to deformable registration schemes in a further validation with clinical datasets. 5 References [1] T. Bergen, E. Efstathiou, T. Wittenberg, C. Münzenmayer, and C. Winter, Geometric calibration of 3D ultra- sound with a tracking system for volume stitching, Proceedings of Computer Assisted Radiology and Surgery, 6(1): 258−260, Jun 2011 [2] J. F. Krücker, G. L. LeCarpentier, J. B. Fowlkes, and P. L. Carson, Rapid Elastic Image Registration for 3D Ul- trasound. IEEE Transactions on Medical Imaging, 21(11):1384–1394, Nov 2002 [3] J. Kybic, M. Unser, Fast Paramteric Elastic Image Registration, IEEE Transactions on Image Processing, 12(11):1427–1442, 2003 [4] T. Lehmann, A. Sovakar, W. Schmiti, and R. Repges, A comparison of similarity measures for digital subtrac- tion radiography, Computers in Biology and Medicine, 27(2):151–167, 1997 [5] R. Shekhar and V. Zagrodsky, Mutual Information-Based Rigid and Non-Rigid Registration of Ultrasound Volumes, IEEE Transactions on Medical Imaging, 21(1):9–22, Jan 2002 This work was supported by the Fraunhofer Internal Programs under Grant No. MAVO 817 775. 20