Visual support for positioning hearing implants H. Ramm1,3, O.S. Morillo Victoria1, I. Todt2, H. Schirmacher4, A. Ernst2, S. Zachow1,3, H. Lamecker1,3 ¹ Medical Planning Group, Zuse Institut Berlin, Germany ² Department of Otolaryngology, Head and Neck Surgery, Unfallkrankenhaus Berlin,Germany ³ 1000shapes GmbH, Berlin, Germany 4 Beuth Hochschule für Technik Berlin, Germany Contact: heiko.ramm@1000shapes.com Abstract: We present a software planning tool that provides intuitive visual feedback for finding suitable positions of hearing implants in the human temporal bone. After an automatic reconstruction of the temporal bone anatomy the tool pre-positions the implant and allows the user to adjust its position interactively with simple 2D drag- ging and rotation operations on the bone's surface. During this procedure, visual elements like warning labels on the implant or color encoded bone density information on the bone geometry provide guidance for the de- termination of a suitable fit. Keywords: bone anchored hearing implant, surgery planning, segmentation, visualization 1 Motivation Bone anchored hearing implants directly transmit sound to the inner ear by means of bone conduction. Those systems are indicated for mixed and conductive hearing loss of a specific extent. The Vibrant Bonebridge (MED-EL Elektromedizinische Geräte GmbH, Innsbruck, Austria) is an active bone conduction system, where the implant is com- pletely positioned under the skin. It consists of an electronic transmission unit that receives signals from an exterior, magnetically attached audio processor, and a floating mass transducer (FMT) that converts the electronic signals into mechanical bone vibrations. During surgery, a valid positioning of the cylindrical FMT (see Fig. 1) is limited by a num- ber of factors, e.g. good conduction properties, vulnerable structures, and a sufficient bone quality for a proper fixation within the cortical bone (dense outer bone layer). The sinodural angle is the preferred site to place the FMT (see Fig. 1). Figure 1: Anatomy of the temporal bone (left) and a corresponding CT slice (middle). FMT geometry with screw posi- tions indicated as yellow cylinders (right). To allocate space for the FMT the surgeon has to mill away bone, a procedure that requires a high degree of experience and precision. 3-dimensional (3D) computed tomography (CT) data is usually acquired preoperatively [1]. There are two major criteria that are relevant for the surgeon to judge on the suitability of an implant position: the cortical thick- ness at the screw positions and the penetration of vulnerable structures. The cortical thickness is crucial for fixation and for optimal conduction. The cortical bone should at least cover 3 mm of each screw for an optimal fit. The second crite- rion is the penetration of vulnerable structures. If there is not enough space in the mastoid bone for the FMT without 116 harming the ear canal, the dura, the inner ear or the sigmoid sinus, the implantation cannot be performed. Some struc- tures, for example the dura, can be penetrated to a certain degree. Here, it is important to know the depth of the penetra- tion. To our knowledge there is currently no software solution available that provides automatic segmentation and al- lows the surgeon to try different 3D implant positions and quantify the suitability of a position w.r.t. the above require- ments. The goal of this work was to develop a prototype of a decision support system for an intuitive positioning of bone conduction implants based on individual patients’ anatomy. Related work: Damann et al. [2] investigated the feasibility of positioning hearing aids in the mastoid bone based on standard software packages. Their approach requires manual segmentation of the CT data (approx. 45 min.) and does not provide feedback on important parameters like the distance to vulnerable structures. Waringo et al. [3] propose a framework that optimizes the position of hearing aids in the bone and computes the milling volume. The surgeon has no visual information about the bone structure or manual control over the implant position. Salah et al. [4] introduce an in- teractive method to perform a virtual mastoidectomy based on semi-automatic segmentation of CT data (approx. 15 min.) with an application in cochlear implant planning. To our knowledge there is currently no method available that provides automatic segmentation of the temporal bone region. Todd et al. [5] argue that this might be related to the large number of complex shapes and high variation of structure size within this region. Contribution: We present a software prototype for the preoperative assessment of suitable positions of hearing implants in the temporal bone. Our method performs a fully automatic geometric reconstruction of the anatomical structures that are relevant for the positioning of the implant, in 2 to 3 minutes. This reconstruction result is then used to automatically pre-position and interactively adjust the implant geometry on the bone with only a few mouse manipulations. During this interactive process, visual feedback is presented to the user that provides intuitive guidance for finding a suitable position. The tool is implemented as an extension to the software ZIBAmira (Zuse Institute Berlin, Berlin, Germany) and is freely available for research purposes (http://www.1000shapes.com/bonebridgeviewer). 2 Materials and Methods The visualization tool builds upon a three stage process (see Fig. 2). First, the image data is automatically segmented and geometries of the mastoid bone and the relevant structures at risk are generated. Second, distance fields for fast look-up are generated to efficiently compute the distance to vulnerable structures and the bone density. Third, CT data, reconstructed anatomical models and precomputed look-up fields are combined to generate intuitive 3D and 2D visuali- zations of a chosen implant position. In the following we will describe segmentation, pre-computation and visualization in more detail. Figure 2: Left: Diagram showing the data and methods involved in the framework. Right: Statistical shape model (SSM) of the temporal bone that serves as an atlas of the relevant anatomical regions (a) and (b), bone density map in- dicating low HU values as dark areas (c) and a risk structure distance map for querying signed distance and name of vulnerable structures, e.g. dura (d). 2.1 Automatic Segmentation of the Temporal Bone from CT-Data For an automatic segmentation a statistical shape model (SSM) of the temporal bone has been generated from 37 CT scans following the approach presented in [6]. The database used for training the SSM included scans of adults only (aged 29 to 73) without anomalies of the temporal bone anatomy. The SSM is represented as a triangular mesh contain- ing 31,810 triangles. It consists of so called patches, i.e. regions on the surface, describing the outer cortical shell of the skull and the structures at risk (see Fig. 2(a) and (b)). By adaptation of the SSM to new unsegmented image data, we 117 reconstruct the individual shape of the temporal bone anatomy. At the same time the predefined patch structure serves as a local atlas and allows for identification of structures at risk. Following the framework presented in [7] the fully automatic segmentation process comprises the following three phas- es: (i) 3D pose initialization of the SSM within the image data, (ii) adaption of the SSM to the given image data. and (iii) an unconstrained but regularized fine adjustment of the SSM to account for an individual anatomical shape that is not captured by the SSM, yet. The SSM can be extended by each segmented structure to enlarge its shape space. To cope with age-related variation in size, the (adult) model of the temporal bone is initially scaled according to the pro- vided patient age employing prior knowledge of growth curves of the skull. After automatic segmentation of the temporal bone, the resulting triangular surface is decomposed into a mastoid bone region (patch) that is considered for automatic pre-positioning of the implant and remaining patches that will be used to compute distances and possible penetrations of the FMT with structures at risk. 2.2 Pre-computation of a Bone Density Map and Distances to Structures at Risk To select a suitable position for the bone screws (i.e., the implant fixtures) it is indispensable to assess the cortical densi- ty of the mastoid bone, which should be at least 3 mm. Therefore, a bone density map is computed for the mastoid bone patch of the adjusted SSM. Using the method presented earlier in [8], we densely sample the intensity values represent- ed by Hounsfield Units (HUs) at the inside of the mastoid bone in a 5 mm margin. At each surface location an averaged value is mapped to the surface and provides an estimate of the density of cortical bone in this region. For a selected implant position it is important to assure that no vulnerable structure is penetrated, neither by one of the screws nor by the cylindrical body of the FMT itself. A risk structure distance map is computed to efficiently query (1) the shortest (signed) Euclidean distance to the closest point of a structure at risk and (2) the patch id, or name of the vulnerable structures, corresponding to this distance. The distance map is generated by the vector-city vector distance transform (VCVDT) [9] introduced by Satherley and Jones. As an extension to the VCVDT, we do not only propagate vector components to compute the Euclidean distance (see [9] for details), but also the index of the corresponding sur- face patch. The resulting bone density map and the risk structure distance map (Euclidean distance and patch field) are stored for later use. 2.3 3D Visualization of Implant and Bone The 3D visualization is the core component of the implant planning tool. It provides an interactive environment for a targeted search for valid screw positions, i.e. with sufficient cortical bone, and an assessment of implant position and alignment w.r.t. surgical constraints, e.g. penetration of structure at risk. To identify valid screw positions, the previous- ly sampled bone density map is employed and visualized color coded. We map the averaged HUs on the mastoid surface utilizing a bone-like colormap that displays areas of low HU-values as dark semi-transparent regions, whereas dense cortical bone is displayed in an opaque beige color (cf. Fig. 3(b)). The choice of this colormap follows the natural intui- tion, where very thin cortical bone appears semi-transparent. Figure 3: Screenshot of the BoneBridgeViewer application in ZIBAmira (a). Examples of invalid implant positions: cortical coverage of one screw below 2 mm (b) and penetration of dura by FMT (red label (c)). CT intersection with im- plant (d). Besides the static display of the bone density map for each selected implant position the following parameters are eval- uated and displayed as labels: the distance (or penetration depth) to structures at risk for the cylindrical FMT body and the screws, as well as the cortical coverage of the screws. Note, that each parameter is only displayed if a critical value has been reached, e.g. the cortical thickness at a screw is smaller than 3 mm. The cortical coverage of each screw is sampled from the original CT data in real time while the screw is positioned. At each screw we use one line profile that follows the centerline of the screw. We densely sample the Hounsfield Units (HU) from the image data onto that profile. The profiles are then traversed to find a largest connected component, i.e. neighboring points, with a HU value above 250. If the cortical thickness falls below 3 mm a warning label is displayed hovering above the corresponding screw (see Fig. 3 (b) and (c)). 118 The third key aspect of the 3D visualization is the display of the penetration depth for the two screws and the cylindrical body of the FMT. To achieve an efficient distance computation we sample the distance map for each vertex of the three implant structures, i.e. cylinder and two screws. The signed distance map provides the smallest Euclidean distance to the structure at risk (note that negative values correspond to positions outside the skull bone, therefore, indicate penetra- tions). A look-up of the patch id allows for an efficient query of the corresponding structure. Again, a (yellow) warning label is displayed close to the respective part of the implant if it is reaching a critical distance to a vulnerable structure. If a penetration occurs, a red label is displayed including the penetration depth (see Fig. 3 (c)). 3 Results The framework described in the previous section has been implemented as an extension (BoneBridgeViewer) to the software ZIBAmira that already provides a DICOM import option, as well as 3D and 2D visualization. On start of the BonebridgeViewer application a simple user interface is presented that allows for import of 3D DICOM data. After ver- ifying patient information the user selects the desired laterality (left or right ear) before starting the automatic segmenta- tion process. Including the pre-computation of the distance map this process takes approx. 4 to 5 minutes. After success- ful anatomical reconstruction a quad-view or single-view mode is presented. In an interactive 3D viewer the user can manipulate the implant on the mastoid surface by only a few mouse interactions, e.g., dragging the cylinder to move the implant on the surface, dragging the wings to rotate the implant, or dragging the wing while pressing the CTRL key to lift the wings in order to simulate washers that might be used during implantation. At all time the user can verify the 3D position in the 2D slice viewers. The intersection lines of implant and CT data are visualized to allow for good assess- ment of the implant fit. Once finished, typically when no warning labels or only warnings within acceptable bounds (penetration) are displayed on the implant, screenshots can be generated for documentation. The BoneBridgeViewer is currently undergoing an evaluation in a clinical research study, where usability and manual effort will be assessed. First tests indicate manual manipulation times of less than 1 minute for easy cases, or up to 5 to 10 minutes for problematic cases (e.g. children with a very narrow sinodural angle). In a first leave-one-out study we evaluated the automatic segmentation framework on the 37 training datasets. We com- pared the automatic reconstruction results to ground truth data of manually segmented mastoid regions and structures at risk by means of a symmetric surface distance. Automatic segmentation results achieved an average mean distance of 0.68 mm (stddev 0.93 mm) and an RMS distance of 1.18 mm to the ground truth data. 4 Conclusion and Future Work We presented the BoneBridgeViewer, a software prototype that provides intuitive visual support for the positioning of hearing implants based on patient specific anatomical models derived from medical image data. To our knowledge this is the first tool that combines fully automatic segmentation of the temporal bone and interactive, visually guided im- plant positioning. Although the BoneBridgeViewer provides a full 3D visualization environment, valid implant posi- tions are restricted to the bony surface and therefore interactions are reduced to simple 2D dragging and rotation opera- tions. During this interaction all relevant implantation parameters are displayed as easy-to-grasp visual elements in a 3D environment. Assuming, that the manual effort won’t exceed 10 minutes, the BoneBridgeViewer adds only a small amount of manual effort for the surgeon (if any, because the CT scan has to be examined anyway). Additionally, various alternative implant positions can quickly be explored and assessed. In its current state the BoneBridgeViewer allows for exploration of different implantation scenarios. For a future clinical application it is necessary to transfer a found im- plant position to surgery. This could easily be achieved by measuring distances to temporal bone landmarks that are well know to the surgeon and can easily be identified during surgery. Until now the employed SSM of the temporal bone region does not contain anomalies, e.g. a missing ear canal. Future SSMs will include such anomalies to provide automatic segmentation capabilities for a wider range of patients. The re- sults of an ongoing clinical evaluation (involving four surgeons) will be used to further improve the usability of the ap- plication and to evaluate its benefit compared to the standard preoperative procedure. Future features might also include automatic implant positioning capabilities based on an optimization w.r.t. relevant parameters, like cortical coverage of the screws or the distance to structures at risk. By simply exchanging the implant geometry, our framework could easily be adapted to similar applications with different implant designs. 6 References [1] Vagli, P, F Turini, F Cerri, and E Neri. Temporal Bone. In Image Processing in Radiology, 137–149, 2008. [2] Dammann, F, A Bode, E Schwaderer, M Schaich, M Heuschmid, and M M Maassen. Computer-aided Surgical Planning for Implantation of Hearing Aids Based on CT Data in a VR Environment. Radiographics: a Review Publication of the Radiological Society of North America, Inc 21, no. 1 (2001): 183–191. 119 [3] Waringo, M, P Stolka, and D Henrich. First System for Interactive Position Planning of Implant Components. CURAC (2003). [4] Salah, Z, M Kastner, F Dammann, E Schwaderer, M M Maassen, D Bartz, and W Straßer. Preoperative Planning of a Complete Mastoidectomy: Semiautomatic Segmentation and Evaluation. International Journal of Computer Assisted Radiology and Surgery 1, no. 4 (November 23, 2006): 213–222. [5] Todd, C, M Kirillov, M Tarabichi, F Naghdy, and G Naghdy. An Analysis of Medical Image Processing Methods for Segmentation of the Inner Ear. In IADIS International Conference Computer Graphics, Visualization, Com- puter Vision and Image Processing, 213 – 218, 2009. [6] Lamecker, H, M Seebass, H-C Hege, and P Deuflhard. A 3D Statistical Shape Model of the Pelvic Bone for Seg- mentation. Proceedings of SPIE 5370 (2004): 1341–1351. [7] Seim H, D Kainmueller, M Heller, H Lamecker, S Zachow, H-C Hege Automatic Segmentation of the Pelvic Bones from CT Data Based on a Statistical Shape Model Eurographics Workshop on Visual Computing for Bio- medicine (2008). [8] Seim, H, H Lamecker, and S Zachow. Segmentation of Bony Structures with Ligament Attachment Sites. In Bild- verarbeitung Für Die Medizin (BVM), 207–211. Springer, 2008. [9] Satherley, R, and M W Jones. Vector-City Vector Distance Transform. Computer Vision and Image Understan- ding 82, no. 3 (June 2001): 238–254. 120