Minimally invasive procedures such as biopsy or thermal tumor ablations share a similar planning scenario: a linear path for a needle shaped instrument has to be found such that the needle tip can reach a defined target such as a tumor. The path should be chosen according to several criteria, which influence safety, practicability and, in the case of ablation techniques, the ablation success of the procedure. In this work we present a method to explore all possible access paths to a given target with respect to clinically relevant criteria. The method leverages the graphics processing unit to sample these criteria. Parallel coordinates are utilized to interactively explore the resulting set of paths. Finally, we present an interactive selection technique in combination with a real-time visualization of the selected path and the patient anatomy.
Our method can be outlined by the following steps: (1) Sampling of clinical criteria for all paths, (2) Visualization of path properties and (3) Interactive exploration of access paths. These steps are described in the following sections. Sampling of clinical criteria for all paths We discretize the domain of possible paths for the sampling process. To represent all possible directions for one specific target point, we generate one equidistant cylindrical projection (also called equirectangular projections) per clinical criterion and store it into a 2D image map. The center of projection is the target point. In the scope of this work we will refer to these projections as cylindrical projections. One projection typically covers a horizontal angular range of 360° and a vertical range of 90° (assuming that a vertical angulation of more than 45° either in the cranial or caudal direction is not clinically relevant). This domain can be sampled with various resolutions resulting in different computation times. To sample possible paths for a whole target region, the projection is repeated for multiple points on a grid inside the target volume. Hence, the number of projections depends on the number of criteria, the chosen grid size and the size of the target volume. We utilize various methods to generate the cylindrical projections of the criteria (see Fig. 1). To sample the insertion depth, the z-Buffer of a rendering of the patient skin is projected. For that, the cylindrical projection is derived from a cube map [7] generated by the graphics processing unit (GPU). For the sampling of distances to risk structures, a 3D Euclidean Distance Transformation (DTF) to these structures is computed in a preprocessing step. During projection, a GPU computed Minimum Intensity Projection of this DTF is used to determine the shortest distance to any risk structure for any given path (for more details see [2]). The maps containing the vertical angulation and the in-plane angle can be generated generically since they are independent of the patient anatomy.
Finally, the 2D projection image maps are parsed. Each pixel represents one path. All valid paths (paths that do not collide with any risk structure) are extracted from the images and stored with all corresponding criteria into a database.
Each path is visualized as a two-dimensional line sequence intersecting each axis at the point that corresponds to the paths value for the criterion that is represented by the axis. Fig. 2 illustrates the concept of parallel coordinates for path criteria visualization. To enable visual distinction of paths even for large numbers of paths we select the brightness and opacity of the lines depending on the criteria values. Lines with low (i.e. good) criterion values are displayed brighter than lines with high values.
Interactive exploration of access paths The initial display of the parallel coordinates plot contains all paths (see Fig 3 upper left). To reduce the number of displayed paths, we utilize restrictor sliders on the axes to select ranges of the criteria (see Fig. 3 lower left). Lines outside of the defined ranges disappear. Thus, the set of displayed paths can be quickly reduced to the interesting candidates. The remaining paths can be explored by simply moving the mouse over the plot: the line below the mouse cursor is automatically displayed in red. The respective path is visualized in real-time in a 3D view together with a rendering of the anatomy. A click with the left mouse button on the plot either locks or unlocks the selection. 3
For a dataset with a resolution of 512 x 512 x 172 voxels (voxel size: 0.625, 0.625, 0.625), the whole computation has been finished in 114 seconds on a standard computer (Core i7 940 2.93 GHz, 12 GB RAM, GeForce GTX 670, Windows 7 64Bit). The computation duration depends both on the image size and the used sampling grid sizes. We used an angular resolution of 0.5 pixel per degree. Hence, the projection maps have a resolution of 180 x 45 pixels. Positions inside the target tumor are sampled on a 3.75 x 3.75 x 4.0 mm grid. For the example dataset this yields 48 sample positions inside the tumor (diameter: 21 mm). The portion of the computation time needed to prepare the parallel coordinates plot based on the projection images depends on the number of possible paths and therefore varies depending on the anatomical situation. In our example 78306 possible paths have been extracted.
We conducted an initial informal study with 3 testees (2 radiologists, but not with an interventional background, 1 computer scientist with focus on human computer interaction). The goal was to assess the general usability of the system. The testees were able to quickly filter the large range of paths and to explore the remaining paths with the mouse over interaction. We used the user experience questionnaire developed by SAP [8] to evaluate the user experience with our system. All of the six scales (attractiveness, perspicuity, efficiency, dependability, stimulation and novelty) resulted in values between 1.75 and 2.41. Although the number of testees was quite low and the testees did not belong to the targeted user group, this indicates a positive user experience. 4
Although we focused on needle based procedures in the liver, the method is transferable to other organs. Depending on the intervention scenario, only few modifications might be required. The presented method can be used to plan ablation procedures. However, the incorporation of a simulation of the ablation result as an additional criterion is still work in progress. It poses a challenge since projection is not an option due to the complex nature of the simulation. Computational times are already low, but there is still room for improvement. Besides optimization of the implementation, adjustment of the parameters seems promising. Reduction of the horizontal angular range of the projections to 180° for example results in a considerable speed up.
Due to the large number of paths displayed in the parallel coordinates plot, distinction of the paths is not optimal yet and range modification using the sliders is inevitable. We investigated coloring of the lines based on clustering. This could either be done based on the criteria values or based on principal insertion regions (like intercostal spaces). However, in both cases, clustering results in too many clusters and colorization cannot be applied easily. Hence, this remains an area of ongoing research.
The mentioned study is only a first step towards evaluation of our method. Besides demonstrating the fundamental suitability of the method for the problem at hand, it also showed certain areas with potential for further improvements. The testees proposed to use sensible default settings for the restrictors in order to reduce the visual complexity from the beginning. Furthermore a path could be preselected based on weighting factors (see [4]) to serve as a starting point for the exploration. It was also proposed to display the criteria values of the selected path in the plot and to update the display as soon as the path is manipulated manually in 3D space. 5
We presented a method to explore the properties of all valid linear paths for target regions during the planning of minimal invasive procedures in the liver. After a preprocessing time in the range of a few minutes, the user can interactively investigate the paths using a parallel coordinates plot. A key concept for scoping with the immense amount of paths is the restriction of allowed criteria values using restrictor sliders. Identification of a suited path is carried out interactively by moving the mouse over the parallel coordinates plot. The line under the mouse is highlighted and the according path is visualized in a 3D rendering in real-time. Ongoing research focuses on better differentiability of the lines in the parallel coordinates plot and on the integration of ablation simulation methods for ablation procedures. Furthermore, extensive evaluation of the proposed method is future work. Especially the comparison with the typical clinical workflow by interventional radiologists is crucial to assess the benefit of our approach. 6