In this paper we propose a new method for stitching multiple fluoroscopic images taken by C-arm instrument. We employ an X-ray radiolucent ruler with numbered graduations while acquiring the images, and the image stitching is based on detecting and matching ruler parts in the images to the corresponding parts of a virtual ruler. To achieve this goal, we first detect and recognize numbered graduations on the ruler in the image. Then, we initialize the panoramic X-ray image with a virtual ruler, and we ''paste'' each image by aligning the detected ruler part on the original image with resepct to the virtual ruler on the panoramic image. Our method does not require overlap between the images as traditional methods. We tested our method on 8 different datasets of X-ray images, including long bones and a complete spine and our method achieves good results.
The X-ray images of this study are acquired using the Siemens SIREMOBIL IsoC3D C-arm (Siemens, Erlangen, Germany). During image acquisition, we utilize a radiolucent X-ray ruler as the image stitching reference. The ruler has a length of 125 cm and a major graduation interval as 1 cm. Moreover, numbers are present on the ruler for every 5 cm. During the acquisition, the ruler is placed approximately parallel to the anatomical structure being imaged, and we require that the relative position between the ruler and the imaged anatomical structure be fixed.
Note that our method does not rely on the pairwise stitching of successive images, therefore we make no requirement about the overlap between the images while acquiring images.
Ruler detection is performed in three steps: graduation detection, number detection, and graduation labeling. To detect the graduations in the image, we first detect the main line on the ruler using the Hough Transform algorithm as the strongest line response in the image. Then the image is rotated accordingly to make the main line horizontal. Thus the rotation variation is eliminated. Then, we take a parallel line which is slightly above the main line (we use gap of 5 pixels in our implementation). The image intensity varies periodically along this line due to the regular ruler pattern, and get local minima when we encounter the graduation. Therefore, the graduation locations are identified by nonmaximum suppression on the intensity curve over the line, as in Figure 2(a). To detect the numbers in the image, we first detect the digits from ''1'',''2'',...,''0''. To this end, we employ multi-level HoG (Histogram of Oriented Gradients) feature [5] and RF (Random Forest) classifier [6]. For each digit, we collect a set of samples by manually annotating the bounding boxes on a set of training images. In addition, we create a ''bg'' class which represents the non-digit backgrounds. Therefore, our training data contains 11 classes (10 digits + ''bg''. Then, for each number class, we train a ''one v.s. all'' binary RF classifier. During the testing stage, given a new image, we detect the digits using the sliding-window method, where we get the score map for each of the 10 digits on every location of the image. Based on these response images, we perform a thresholded non-maximum suppression which returns the bounding boxes of the detected numbers. After all the digits are detected, we merge the digits to numbers based on the location proximity in the image, and then the graduation which is closest to the center of the number is associated with this number, as shown in Figure 2 (b).
Finally, using the already associated graduations as seeds, all the other graduations are labeled with their corresponding numbers, as in Figure 2 (c).
After graduation detection and number detection, for every image, we know the location of every graduation and its corresponding number. To stitch the image, we first initialize an empty image with sufficient size. We establish a virtual ruler in this stitched image, and then each original image is ''pasted'' into this stitched image by aligning the detected graduations in the image and the corresponding graduations on the virtual ruler in the stitched image. The alignment is calculated by the optimal similarity (rigid+scale) transformation between the corresponding graduation coordinates in the original image and the stitched image. This process is visualized in Figure 1 bottom row. 3
We conduct our study using the following 4 structures: 1. (Cadever Femur) a cadaver femur after neck osteotomy 2. (Lower Extremity) a complete cadaver femur and tibia 3. (Plastic Femur) a complete plastic femur 4. (Spine) a complete spine phantom with deformity correction devices.
Each of the 4 structures are imaged in this way twice both in AP and LAT views, yielding 8 datasets (image sequences), based on which we perform our image stitching experiments.
Because our method does not require overlap between the images, for each datasets, we perform our image stitching method on different levels. The first level uses all the available images in the dataset which involves most overlap. The second, third and fourth levels use every two, three and four image, respectively. Therefore, in the higher levels the overlap is reduced, and often in the last (fourth) level, there is no image overlap at all.
The results on our datasets at different levels are shown in Figure 3. Due to the page limit, we only show 2 out of the 8 datasets. We observe similar results on the other datasets. Visually we can see that our method generates satisfactory results: the bone structures and the ruler show continuous contours. Also note that our approach generates good results on all levels, demonstrating that we do not require image overlap. The feedback from several orthopaedic surgeons about our image stitching results indicates that they are accurate enough for clinical usage.
To evaluate our approach in real clinical environment, we employed our method with our clinical partner Schulthess Klinik in a real scoliosis correction surgery. The X-ray images are captured by the Ziehm Vision FD Vario 3D system for intraoperative evaluation, and the panoramic image is constructed by our method. Figure 4 shows results on two image sequences. We can see that our method generates satisfactory results.
We have proposed a new method for stitching multiple fluoroscopic images acquired from C-Arm instrument. We employ an X-ray visible ruler. During image acquisition, the relative position of the ruler with respect to the imaged structure remains unchanged, and therefore the ruler can be used as the reference for image stitching. To this end, in each image, we first detect the main line and all graduations on the ruler. Then, we perform number detection to label and propagate the associated numbers of the graduations and image stitching is performed accordingly. Unlike many other methods, our method does not rely on image overlapping. We conducted experiments on 8 image sequences captured on different anatomical structures. Qualitative results show that our method generates satisfactory image stitching results. Distance and angle based quantitative measurements show that our method keeps the important geometric properties even with little or no image overlap.
Unlike the method such as [2], out method requires no modification of the fluoroscopic unit. We developed a robust method to detect the graduation and numbers on the ruler in order to achieve an accurate and robust image stitching. Furthermore, our method is based on aligning the detected ruler part on the original image with a virtual ruler on the panoramic image. We do not rely on pairwise image registration. Therefore, unlike most methods in literature, we do not require significant overlap between the images. 7