Minimally invasive procedures become increasingly popular due to their potential advantages. The correct choice of access to the operation area is essential for a safe and successful surgery. Besides traditional training options (e.g. cadaver-based,) virtual training systems gain importance due to their flexibility and cost efficiency. We developed a novel 3D interaction method to improve the understanding of the placement and orientation of the instrument for minimally invasive access (syringe needle or dilator) to the spine. In a virtual 3D scene of a patient's spine anatomy the trainee defines a straight trajectory, placing the virtual instrument with a haptic 6DOF phantom device in three steps. Haptic material properties and constraints support the single interaction tasks. We do not aim to simulate the handling of real instruments.
jectory bears a high mental effort. Thus, many adjustments are necessary to achieve the desired trajectory. Consequently, the user tends to perform a simplified planning, employing only a single image slice. Using a commercial 6 DOF haptic input device (e.g. SensAble PHANTOM - see Figure 1) the virtual puncture device can be manipulated more intuitive. If the given interaction task demands less than the available DOF, a reduction of the DOF is recommended to prevent unintentional actions [8]. The operational axes of haptic devices can be constrained for this purpose. In order to take advantage of the possibilities given by the haptic device, we separate the puncture task into three individual phases, whereas each phase contains a specific set of haptic constraints. The objective of our work is to provide a cost-efficient 3D interaction method that supports the virtual training of minimally invasive access planning (straight trajectory) regarding puncture point and incidence angle of the puncture device. 2
For a conscious learning process, we decomposed the virtual puncture of the instrument into three interaction subtasks. Hereby, the trainee concentrates on the placement of the puncture point and the angle adjustment separately. The straight-line puncture detects collisions of the resulting trajectory with tissues, but also trains the puncture depth itself. In the training setup, the trainee views the reconstructed polygonal 3D model of a patient’s anatomy (neck or lumbar region) on a 2D screen. In addition, this scene includes a virtual model of a needle or dilator that the trainee controls by a haptic 6DOF Phantom device (see Figure 1). The virtual puncture device, a cylinder with a cone as tip, corresponds to the stylus’ orientation and tip position. Mouse and keyboard inputs are used to adjust the viewpoint (rotate, pan and zoom). The trainee has to specify an appropriate access for an injection or a surgical procedure on the spine. He or she performs this in three successive interaction subtasks supported with haptic feedback: Positioning: In this task, the tip of the virtual instrument has to be placed on the virtual surface of the 3D skin representation. This placement is a 6DOF movement with simultaneous translation and rotation. Because of the lack of visual depth cues a proper positioning in z-direction is difficult. Therefore, the collision with the skin is indicated by haptic feedback, avoiding the tip to go through the skin. Furthermore, the tip will be haptically attracted to the surface within a certain distance to it. Thus, the user can move the tip along the skin concentrating only on 5DOF or in fact 2DOF.
Orientation: When the puncture point is specified, the incidence angle has to be adjusted. In order to prevent a shift of the puncture point, the tip of the instrument is locked. With the tip as pivot point, the user has 3DOF to orient the instrument appropriately.
Puncture: During this task, the user interaction is restricted to a translation of the instrument along its longitudinal axis. Thus, the puncture point and the incidence angle cannot longer be changed. The target point of the trajectory will be determined by the puncture depth. Possible injuries of vulnerable organs and collisions with impenetrable vertebra will be indicated by a high resistance force and a buzz effect by the phantom.
Switches on the stylus of the phantom device are used to change between the three tasks. Thus, adjustments of the previous settings can be done easily. Since some phantom devices have only one switch, the previous task can alternatively be selected by double-clicking the “main” trigger.
Our novel 3D interactive method can be used with default hardware setup for private use (PC + 2D screen and mouse) with only the phantom device as special hardware. We used a PHANTOM Omni® and PHANTOM Desktop® from SensAble for the implementation and evaluation of our three-step interaction method. We developed on an Intel® Xeon® quad-core Processor with 3.06 GHz, 8GB RAM and an NVIDIA GeForce GTX 460 graphics card with 768MB memory supporting 3D graphics. For development, the prototyping environment MeVisLab [9] was used, incorporating the visualization toolkit (VTK) for geometry handling and graphics rendering, and the Open Haptics Toolkit [10] for accessing the phantom device and haptics rendering. We derived a specific haptic mapper from vtkPolyDataMapper class. In [11], a similar integration of OpenHaptic Toolkit in VTK is explained.
Figure 3 illustrates the different constraints and the available DOF of the single subtasks. To support the positioning task, the stylus tip is attracted to the skin surface by rendering it as a constraint (Figure 3a). In this case, the proxy position is directly constrained to the skin surface and movements are restricted to 5DOF (see Figure 3a). The snap distance property determines the proximity in which the constraint will force the device to the skin surface. Beyond this proximity, all 6DOF are available to control the virtual puncture device. With the actual property value of 5 mm, the stylus could be pulled out of the attraction area (back to 6DOF) by applying force. When the tip touches the surface and the stylus button is pressed, a marker (3D point) will be placed at the current proxy position, specifying the puncture point. Rendered as a constraint, this 3D marker locks the position of the tip, as shown in Figure 3b. Via double-click, the marker and thus the constraint will be deleted.
With the tip locked, the puncture device and stylus respectively will be rotated around its tip (proxy) to determine the incidence angle. Rotation about the device axis has no effect on the virtual environment, but the handling of the stylus and the stylus buttons is facilitated. Pressing the stylus button this time disables further rotation locking the orientation. Since the less cost intensive PHANTOM Omni® and PHANTOM Desktop® only support 3DOF haptic feedback (green arrows in Figure 1), the rotation of the device stylus cannot be fixed in place. In this case, the orientation of the virtual puncture device will be released from the proxy orientation to avoid further changes in the virtual environment. More specifically, the new transformation of the puncture device contains the translation to the current proxy position and the proxy rotation at the press of the button. The PHANTOM Premium devices (e.g. 1.5/6DOF) offer force feedback in 6DOF. In this case, changes of proxy orientation will cause a virtual spring-damper effect (same principle as shown in Figure 2) that forces the stylus in a fix orientation corresponding to the incident angle. Additionally, a line will be generated as an extension of the instrument's center line by its own length (see Figure 3c). This line serves as constraint for the puncture task. In order to allow a penetration of the surface, the snap distance of the marker and skin surface constraint is adjusted to 0mm. Translating the virtual instrument along the line constraint determines the puncture depth. A second 3D marker specifies the target point of the access trajectory, after the stylus button is pressed again. When finished, the virtual instrument will be released from the input device.
In order to enable adjustments of the previous setting, the trainee could return to the previous task by double-clicking the stylus button. For correction of puncture depth, the target point constraint will be deleted with the marker. To enable a stable adjustment of the incident angle, the line constraint will be deleted and the first marker becomes a constraint again. The complete transformation of the proxy is used to control the virtual instrument. Thus, the trainee could resnap the tip of the stylus to the puncture point. Deleting this point in a further undo step leads to the initial state presented in Figure 3a. Besides those constraints, haptical material properties such as stiffness are used to support the trainee. Material properties are specified within the HLAPI. Principally, they are used to detect collisions with surfaces during puncture instantly even if the view is blocked. Thus, the trainee can quickly identify and correct errors during trajectory planning. Indirectly, this trains the trainee’s knowledge of the underlying three-dimensional spinal anatomy. A training case contains generally following 3D models according to the underlying 2D image slices (MRI or CT): vertebras, intervertebral discs, spinal cord, large vessels and nerves and muscles. Neck cases additionally contain tissues of the respiratory tract and the esophagus. Differences in the properties help to distinguish between critical and less critical tissues for minimally invasive access to the spine. This classification is performed by our medical experts during the examination of the 3D reconstructions. An injury of vulnerable risk structures such as nerves and large vessels has to be avoided, while impenetrable structures (e.g. vertebras) may serve as landmarks. Injury of fat and muscle tissue is unavoidable. Collision detection is realized with stiffness. The stiffness, or spring constant "k" (0,…,1), determines the resistance force F=kx (Hooke's Law equation) where "x" is the vector representing penetration depth [9]. Bone tissue will be indicated by a high stiffness of 0.98, thus it feels impenetrable without causing the device to kick or buzz. A high stiffness of 0.9 applies to vulnerable risk structures in order to achieve a clear tactile impression of their surfaces. An optional buzz effect alerts the trainee in case of a collision with them. The other tissue surfaces differ from those with a value of 0.7 for a more soft tactile impression. Penetrations of these surfaces are enabled by a value greater than 0 and less or equal 1 of the pop-through property. With a threshold of 0.18, a relatively low force of penetration is needed to pop through those surfaces. Thus, the penetration will be recognized without accelerating motion after being punctured. Friction provides resistance to lateral motion on an object. That property will be used to stabilize the hand as the trainee moves the puncture device along the skin surface. Friction consists of static and dynamic components. With a value of 0.5 for static friction (0,...,1) the surface feels slightly adhesive when the device initially starts moving along the surface. Dynamic friction (0,…,1) is relevant during the device movement along the surface. A value of 0.2 is applied in order to stabilize the motion, but not to restrict it. A dynamic (and static) friction value of 1 causes a lateral locking effect while the device is in contact with the surface. This feature is used as an alternative to the puncture marker constraint during the test runs. 3
We set up two neck training cases with soft disc herniation and five different training cases of the lumbar region. After a familiarization phase with the input devices, we compared our constraint-based interaction method with free-hand control. To examine the efficiency of the different constraints, the related single subtask was performed one time with and one time without constraint.
The positioning task with the skin surfaces as constraint was performed considerably faster and more accurate than with freehand movements. By use of collision control instead, the performance was also faster than free hand control. There was no significant difference to the constraint mode for trained subjects, but untrained subjects were better (faster and more accurate) with the constraint method. On closer consideration, they spent most of the time to get in contact with the surface. Even though friction property was used to stabilize the movements, attention was required to keep contact at irregular parts of the skin surface as well.
Similar observations were made during the orientation task. For untrained subjects, the employment of friction, as described above, to lock the device lead to less accurate results compared to the marker constraint. Performances without any stabilization by a marker constraint or friction property required the most time.
Constraining the instrument movement along the trajectory, defined by puncture point and incidence angle, prevents an unintended and intended change of the specified settings. Thus, the result is not distorted. A freehand movement enables the subjects to move and rotate the puncture device within the body to find a proper trajectory. However, this does not conform to the clinical situation.
The initial tests have shown that executing a puncture corresponding to a template trajectory (integrated 3D line) is performed faster and more accurate with our constraint-based method than with free-hand control. Furthermore, with our three-step method the puncture was performed faster than with the marker-based method from [3] when the trajectory was not in-plane. 4
We presented a novel 3D interaction method, which is designed to support the virtual training of positioning and orientating the puncture device for juxtaspinal puncture. The puncture task is separated into three subtasks, taking advantages by using a haptic 6DOF Phantom device to simplify their respective performance. The intention is to support the comprehension of the small scope for minimally invasive access by training trajectory planning within a virtual training system for spine surgery.The commercial Phantom device can be used for further interaction task, such as positioning and orientation of implants. Thus, several training tasks can be performed with one cost-efficient hardware setup and training system, before the acquired skills could be further improved with the help of more expensive and customized soft- and hardware (e.g. virtual or mannequin-based simulators). Our method is real-time capable on a current standard PC providing only haptic landmarks by tissue surfaces in combination with MRI or CT image slices. In contrast to the implicit marker-based approach in [3], our method provides an intuitive, direct way of defining the trajectory. Furthermore, injuries or collisions will be noticed instantly. Despite the sequential process, directing instrument and switching between three interaction tasks with the same input device, a fast performance is enabled. Integrating our 3D interaction method improves the non-simulating marker-based training system described in [3] regarding the puncture training for needle injections and minimally invasive access through a retractor tube (by dilator).
Informal interviews with two orthopedic experts confirmed the general usefulness of our constraint based interaction method. The feedback of the informal interviews and the findings of the initial tests will be used to further improve the approach and to design an adequate task-based user study.
The current default property values have not empirically been evaluated. Slight value variations did not show obvious differences. Nevertheless, further tests are necessary to specify exact values. The difficulty of perceiving spatial relations between interactively controlled puncture device and 3D tissue surfaces due to missing visual depth cues and occlusions is largely compensated by haptic feedback. Nevertheless, the initial contact to surfaces without additional visual depth cues is still difficult.
With gravity compensation techniques, such as described in [12], the interaction could be optimized. Actually, during the test runs with friction used as locking tool, the weight of the stylus and arm of the device had slightly influenced the interaction. However, with the constraint-based method no significant influence has been observed. Regarding the physical fatigue by using the device for a long duration, it could be reasonable to reduce its weight such that it would remain at its current position if it is released.
Training opportunities with special hardware are limited to the number of available workstations (e.g., simulators). For this reason we want to adapt our three-step approach for use without the Phantom device based on widgets for 3D manipulation, such as defined by [13], and pseudo-haptics [8] and compare them with each other and the marker based method presented in [3]. 5