In computer assisted liver surgery, the use of pre-operative 3D computer tomography (CT) images provides basic orientation and valuable information about the patient liver. During the surgical intervention, the accuracy of this information is reduced by motion and deformation of the organ. The fusion of intra-operative ultrasound (US) imaging with the pre-operative data is the next step in order to improve this situation. As pre-requisite, the identification of corresponding structures in US and CT (such as blood vessels) is required. Within this paper, the integration of an ultrasound vessel segmentation algorithm in a navigation system for liver surgery is presented. Initial results obtained on patients undergoing liver resection are evaluated.
The surgical navigation system is built as a transportable setup containing an NDI Vicra Camera (Northern Digital Inc, Canada), an integrated Terason T3000 ultrasound system with an 8IOA intra-operative probe (Teratech Corporation, USA) and a Shuttle barebone PC (Shuttle Inc, Taiwan) with a touch-screen monitor (ELO Touchsystems, USA). Instrument tracking is enabled by a navigation toolset composed of adapters to the existing surgical tools (CUSA, ablation devices, US probe) and a pointer calibration unit.
Before each navigated liver surgery, pre-operative tri-phase CT data was processed by MeVis Distant Services. The MeVis analysis provides segmentation of all the important structures (vessels, tumors, surface) as well as several resection proposals. The resulting 3D models are evaluated by the surgeons and appropriate visualization models are selected and loaded into the navigation system.
The navigated surgery starts by an initial rigid registration of the liver. Four landmarks are manually defined on the virtual liver model. Then, the tip of the CUSA is calibrated with the pointer calibration unit. Using the CUSA tip, the surgeon points the four manually defined landmarks on the real liver, which are automatically recorded by the navigation system. Through the alignment of these two point sets, initial rigid registration is obtained.
In a second stage, US images are acquired and segmented in real-time. The US probe is pre-calibrated using an US calibration unit pre-operatively [8]. Using the probe, vessels are traced by the surgeon. The obtained segmentation and images are recorded with the corresponding position tracking information for further data analysis.
The vessel segmentation algorithm is based on the approach proposed by Dagon in [7] may be shortly described as: 1. 2. 3. 4.
Vessel image mask generation Vessel seed point detection Vessel approximation using an elliptical model
Outliers removal The vessel mask generation starts with a scale-space blob detection [9], which is performed by generating scale-space images Il with eight different scale levels Sl using a Gaussian Kernel of: The pixel intensities of the scale-space images are then summed in a scale-space map IMap with the following equation: σ = 3 ⋅ Sl
n IMap ( x, y) = ∑ [In ( x, y) - Il ( x, y)] l =1 (1) (2) Thus, dark features in the US image that present a large scale-space lifetime are enhanced. A binary threshold tsc is then applied on the scale-space map to obtain the features binary mask IMask.
The features mask is then cross-correlated with a binary circular mask of radius rc and normalized. A threshold tcorr is applied on the obtained image and a connected component labeling algorithm is run in order to identify the highest intensity pixel in each region, which is then labeled as seed point.
To approximate the blood vessel, an elliptical model is applied. To detect the vessel border, rays are regularly casted around the seed point on the initial scale-space mask IMask. The edge points are detected at the first value change in the ray intensity profile. The edge points are then used to initialize a direct least square ellipse fitting algorithm [10], [11]. Finally, outliers are filtered by defining minimal and maximal ellipse axes ratio and axes average length. The C++ implementation of the segmentation algorithm was based on the code provided by Dagon [7] which was improved by adding outlier removal in the vessel seed point detection step and by integrating a region of interest (ROI) provided by the direct interface to an US imaging device (described in Section 2.D)
The segmentation algorithm was integrated into our navigation software [1],[2]. In this way, the segmentation is applied in real-time onto the images generated by the US probe. The obtained segmentation is visualized in both the 2D US image and in the 3D world viewer. A touch-screen user interface was developed for easy use in the operation room environment. The user can switch the segmentation algorithm on and off, change segmentation parameters and record both US images and the obtained segmentations. The segmentation steps can be visualized to better understand the parameters respective influences (Fig. 1). The whole sequence can be loaded and segmented again later with different parameters. A validation application was developed for the result analysis. The recorded US images are loaded again with the obtained automatic vessel segmentation centers. The user then manually defines whether the detected vessel centers are correct, wrong (false positive) or missing (Fig. 2, right). The application then outputs the total number of each vessel center classification.
About 2’200 images (300x430 pixels) were recorded and segmented during one surgical intervention at the Inselspital in Bern, Switzerland. The US probe parameters for brightness, contrast and time gain control were set to the default values. The average time to process one image was 70ms, which resulted in a frame rate of 7 frames per second. Six subsets including a total of 538 images were selected for post-operative analysis with optimized segmentation parameters. Table 1 and 2 present the results obtained with the parameters: tsc = 65, rc = 10 pixels, tcorr = 0.01, Region of Interest: 193x371 pixels. Processed images 109 42 147 73 13 154
Vessels detected 374 158 421 228 25 295
Correct classification 250 55 223 73 22 278
Wrong classification 124 103 198 155 3 17
The average value of correctly classified vessels is 60.03%. The average number of missing vessels per frame is 1.7 and the average value of wrongly classified vessels per frame is 1.1. 4
The number of missing vessels per frame is relatively high and is mainly caused by small vessels. When the threshold value tsc is set to a low value, small vessels are better detected but the amount of wrongly classified vessels from image noise increases as well. Wrongly classified vessels come from homogeneous noise regions in the images and from artefacts lying under the liver borders as illustrated in Fig. 2. When comparing our results with the ones presented by Dagon [7], we see improved outcomes in the amount of correctly segmented vessels (40% by Dagon, 60% in this work). However, the results are not directly comparable as different image data and slightly different validation methodologies were used. The images used herein were acquired in real surgery on human patients where Dagon used an ex-vivo porcine liver specimen under isolated perfusion. The validation approaches differ in that the distance between the manual and automatic segmented vessel centers was not considered for the vessel classification within this work. The higher computation time of 70ms vs 25ms in this work can be explained by the larger size of the US images used with the Terason imaging device and the added load due to the visualization of the segmentation steps as shown in Fig. 1. As we aim to use the segmentation results for intra-operative registration, a further reduction of wrongly classified vessels is desirable. This should be partly solved by detecting the lower organ borders. Another way for reducing wrongly classified vessels is to use the 3D position information of the segmented vessels in the prior images to predict their positions in the next frame. In general, the large number of segmentation parameters made it difficult to obtain a good segmentation while in the operating room. Reducing their number or an automatic adjustment of some parameters might solve this issue.
Another effect observed when comparing the segmentation results from different acquisitions is the high sensitivity to the US image quality. We believe that by adjusting the US imaging parameters, in particular brightness, contrast and time gain control, the number of missing vessels per frame should decrease. The development of the vessel-based registration methods will show whether a reliable detection of large vessels or detection of smaller vessels with higher error rate are preferable.
Finally, the computation time of 70ms per frame is not yet acceptable for a real-time application. Considering that there should be more processing for the registration afterwards. This should be improved by code optimization and/or using GPU computing or multi-threading.