Three-Dimensional Catheter Tip Tracking from Asynchronous Biplane X-Ray Image Sequences using Non-Linear State Filtering Marcel Schenderlein1 , Volker Rasche2 , Klaus Dietmayer1 1 Institute of Measurement, Control, and Microtechnology, University of Ulm 2 Department of Internal Medicine II, University Hospital of Ulm marcel.schenderlein@uni-ulm.de Abstract. Image-based three-dimensional (3D) catheter tracking has the potential provide enhanced visualization and documentation means for cardiac catheter navigation purposes. However, if the biplane X-ray image sequences are acquired asynchronously, common stereo-geometric constraints are violated and simple 3D reconstruction introduces posi- tion errors. The presented work introduces an algorithm based on the unscented Kalman filter, which explicitly models this image acquisition situation. It updates the 3D catheter tip position and orientation with single-plane two-dimensional catheter tip and electrode position mea- surements. These measurements are derived from X-ray images, acquired alternately from two different orientations. In this manner, the proposed approach allows for 3D catheter tip tracking with a promising position accuracy. 1 Introduction Catheter ablation of cardiac arrhythmia is commonly carried out under guidance of fluoroscopic X-ray sequences. During the intervention, electro-physiological catheters, comprising cylindrical metal electrodes on the tip (Fig. 1d), are nav- igated inside the heart chambers. In order to infer the three-dimensional (3D) positions and orientations of the catheters, two image sequences from almost orthogonal orientation are acquired (Fig. 1a). In this biplane setting, the image sequence is generated by means of asynchronous acquisition in order to reduce the X-ray dose as well as cross-scattering effects (Fig. 1b). Such interlaced acqui- sition introduces errors into common reconstruction approaches which assume a stereo-geometric image pair. Instead, 3D positions have to be reconstructed from a set of asynchronous images, while dealing with the object motion between the acquisitions. In [1, 2] the interlacing is dealt with by reconstructing from an image triplet instead of an image pair. The drawback of these approaches is, that the re- construction is always delayed one time step. This in turn results in a tracking delay if applied in a real-time environment. Interlaced acquisition has also been mentioned by De Buck et al. [3] but not been dealt with. They reconstructed the 3D Catheter Tip Tracking 235 catheter tip from image pairs as if they were being acquired synchronously, ac- cepting the introduced errors. De Buck et al. also used a state filter for catheter tip tracking. They applied two independent standard Kalman filters for each image plane and reconstructed the 3D tip position from the two filter results. However, the independent filters may drift apart and loose correspondence. The main contribution of our presented work is a catheter tip tracking scheme, which uses the unscented Kalman filter (UKF) [4] to solve the delay problem and still provides good reconstruction accuracy. 2 Materials and Methods 2.1 Catheter Tip Measurement The catheter tip position is measured in the images by means of a masked normalized cross-correlation (MNCC) of tip templates within an image region of interest (ROI) around the previous tip position. The term masked denotes that not the whole rectangular template is correlated with the ROI, but only the pixels of the actual catheter image and a margin around it (Fig. 1c). This way, the sensitivity to background clutter is reduced. A set of multiple oriented templates is created by ray-casting the catheter model onto the image plane similar to [3]. Their correlation with the ROI leads to a set of potential 2D catheter tip positions with respective 2D orientations. Since the 3D orientation can not be fully inferred from the 2D orientation, the next electrode along the catheter is searched for. Together with its known 3D distance to the catheter tip its 3D position and thus the 3D tip orientation can be estimated. To this end, a search region is defined around the expected electrode position and again MNCC now with an electrode template is applied. The best correlation maxima are stored as electrode hypotheses. Out of the catheter tip hypotheses and its corresponding electrode hypotheses a set of measurements is created for the association with the UKF. IA (t) IB (t) t (a) (b) (c) (d) Fig. 1. (a) Schematic of the biplane x-ray system with the two image acquisition channels. (b) The asynchronous image (IA , IB ) acquisition scheme. Dashed rectangles depict the non-existing images. (c) A masked template. Only the pixels of the template which are inside the dashed contour are correlated. (d) Two views of a catheter from both orientations with tip position (◦) and next electrodes position (×) overlaid. 236 Schenderlein, Rasche& Dietmayer 2.2 The Unscented Kalman Filter The unscented Kalman filter [4] is a recursive state estimator just as much as the ordinary Kalman filter (KF). It uses a prediction phase to forecast its inner state by a state transition model and corrects this prediction by comparing an actual measurement to the predicted measurement in the update phase. In contrast to the KF the state transition model and the measurement model of the UKF may be non-linear functions. This fact is utilized in the proposed approach to apply the perspective projection formula as the measurement function. The UKF was preferred to the commonly used extended KF because of its simpler implementation and its higher accuracy. 2.3 Catheter Tip Tracking Since the 3D position and orientation of the catheter tip is to be tracked, the state vector of the UKF is defined as [ ] x = x, y, z, ẋ, ẏ, ż, θ, ϕ, θ̇, ϕ̇, r (1) where (x, y, z) is the 3D position and (ẋ, ẏ, ż) is the 3D velocity of the tip. The relative position of the next electrode to the catheter tip is characterized by the spherical angles θ, ϕ and the distance from the tip r. The angular velocities θ̇,ϕ̇ are also modeled. The prediction of the state is computed by a constant tip velocity and constant angular velocity model (CV) and a unit transform for r. However, the CV only approximates the complex catheter motion. The non-linear measurement model function is denoted by the two projections (u, v)tip = Pi (x, y, z)with i ∈ {A, B} (2) (u, v)elec = Pi (T (x, y, z, θ, ϕ, r)) (3) where (u, v)tip and (u, v)elec are the 2D positions of the catheter tip and the next electrode and T is a function, which transforms the spherical coordinates of the next electrode into Cartesian coordinates. The perspective projection function Pi alternates between the known acquisition geometries A, B for each time step. In the update phase of the UKF, a measurement of the catheter tip and the next electrode position is passed to the filter. This measurement is chosen from the multiple actually measured hypotheses as the one with the smallest Mahalanobis distance to the current predicted measurement. While the image frames from the two orientations are alternately processed in the prediction and update loop, the UKF implicitly reconstructs the 3D position (x, y, z) and orientation (θ, ϕ) of the catheter tip. 2.4 Experimental Setup In order to evaluate, if the proposed approach is able to track a 3D catheter tip and orientation, the following experiment is conducted. The X-ray images are 3D Catheter Tip Tracking 237 Table 1. Inter-frame min/mean/max 2D position and orientation changes. fps 25.0 8.3 5.0 3.6 ∆ pos. 2D [px] 0.0/7.5/67.1 0.0/17.6/156.2 0.2/24.1/162.7 0.4/29.0/195.6 ∆ orient. 2D [◦ ] 0.0/3.6/79.0 0.0/8.5/107.2 0.0/12.0/107.3 0.0/15.0/111.4 acquired at a resolution of 512 × 512 pixels with a pixel size of 0.36 mm. They are acquired alternately with an orientation difference of 70◦ and at a rate of 25 frames per second (fps) in total. The sequences were also subsampled in order to evaluate the algorithm performance with lower frame rates of effectively 8.3 fps, 5.0fps and 3.6fps. The UKF is initialized as follows: The process noise is set by a CV noise gain with a standard deviation (SD) of the tip position acceleration of 1500 mm/s2 and a SD of the angular acceleration of 35 rad/s2 . These values are derived from an analysis of labeled catheter motion data. The measurement noise is set by an assumed SD of 3 px and 10 px for the 2D position of the tip and the next electrode. The initial tip and electrode position is inferred from a manual labeling step and transformed accordingly into the initial state vector. The velocities are set to zero. The initial state variances are set by choosing ten times the process noise variances. No covariance is preset. We evaluate the tracking in four different patient sequences comprising a total of 936 frames (Tab. 1). The sequences contain active motion applied to the catheter by the physician beside the anatomical motion. A manual labeling of the catheter position in the 2D images serves as ground truth. Linear interpolation between consecutive label data of the same orientation provides the means for reconstructing the 3D ground truth. Although the process noise parameters were adapted accordingly, the ever higher inter-frame dynamics present in the 8.3 fps, 5.0 fps and 3.6 fps sequences caused several hypotheses association errors. This in turn led to temporarily longer mis-estimations of the UKF state. Thus we present only the results for the 25 fps sequences. 3 Results Fig. 2 shows the results of the 25fps sequences evaluation in the form of box plots with whiskers with a maximum length of 1.5 IQR. The median 3D tip position errors are 1.3mm, 1.6mm, 1.1mm, and 1.2mm, leading to a total of 1.2mm. The median 3D tip orientation errors are 28.1◦ , 13.1◦ , 30.0◦ , and 8.7◦ , leading to a total of 18.6◦ . Outliers are mainly caused by the algorithm temporarily loosing the catheter which leads to chaotic filter behavior and arbitrary maximum error values. For better illustration the plots do not show the outlier values. The outlier rates for e3D 3D tip are 15 %, 6 %, 8 %, and 7 %. The outlier rates for eor are 0 %, 0 %, 2 %, and 3 %. The 2D error plots serve as a reference for the tip and electrode detection accuracy. 238 Schenderlein, Rasche& Dietmayer Fig. 2. The 2D and 3D tip position error e2D 3D 2D 3D tip , etip and orientation error eor , eor . 5 100 10 60 80 4 8 tip [mm] tip [px] or [ ] or [ ] ◦ 3 ◦ 6 60 40 e3D e2D e2D e3D 4 40 2 20 2 20 1 0 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 sequence # sequence # sequence # sequence # 4 Discussion The decreased orientation accuracy and the outliers mainly caused by hypothe- ses association errors remain a challenge. One reason seems to be inaccurate measurements which occur in extremal cases like e.g. the 3D catheter pointing perpendicular to the image plane or due to bad contrast in general. Another reason is an insufficient approximation of the complex catheter motion by the CV process model. This fact becomes even more evident in the evaluated se- quences of lower frame rates. Nevertheless, our experience with these sequences is, that if the hypotheses association is correct, tracking remains possible. The accuracy decreases, but stays in acceptable ranges at least for the 8.3 fps and 5.0fps sequences. The results regarding the 3D catheter tip position accuracy for the 25 fps sequences are in the range of published results of other reconstruction approaches [2, 3]. Hence, if the detection did not substantially fail, our implicit reconstruction approach is able to provide clinically acceptable accuracy for the tip position. We have contributed an approach which explicitly models the ac- quisition situation and provides promising accuracy results for the catheter tip tracking. It has the potential to provide enhanced visualization and documenta- tion means for cardiac catheter navigation purposes. Motivated by the discussed challenges, future work includes the investigation of more accurate state filter modeling and higher electrode detection accuracy. References 1. Sarry L, Boire JY. Three-dimensional tracking of coronary arteries from biplane angiographic sequences using parametrically deformable models. IEEE Trans Med Imaging. 2001;20(12):1341–51. 2. Schenderlein M, Stierlin S, Manzke R, et al. Catheter tracking in asynchronous biplane fluoroscopy images by 3D b-snakes. In: Proc SPIE; 2010. p. 76251U. 3. De Buck S, Ector J, La Gerche A, et al. Toward image-based catheter tip tracking for treatment of atrial fibrillation. In: Proc MICCAI Workshop Cardiovasc Interv Imaging Biophys Modell; 2009. 4. Wan EA, Van Der Merwe R. The unscented Kalman filter for nonlinear estimation. In: Proc IEEE Adapt Syst Sig Process Comm Contr Symp; 2000. p. 153–58.