According to literature evidences, simulation is of utmost importance for training purposes and for innovative surgical strategies assessment. Nowadays the market offers mainly two kind of simulators: rubber anatomies or virtual environments, each one with advantages and drawbacks. In this paper we describe a strategy to develop patientspecific simulators using a hybrid approach: silicone models of abdominal organs sensorized with electromagnetic coils, to acquire deformations, coupled with a virtual scene. As demonstrated, this approach allows to mix benefits of a real interaction with the physical replicas with the possibility to enrich the virtual visualization with add-ons and features difficult to obtain in the real environment.
Medical simulators are rapidly evolving from primitive plastic mannequins to machines with embedded technology and, recently, computer assistance capable of creating realistic physiological and patient scenarios. Consequently many types of simulators of varying complexity have been developed and marketed. The existing trainers can be essentially divided into two groups: virtual reality (VR) and physical simulators, while a third innovative approach to the simulation is now finding its space in market and research: hybrid simulation[2; 14].
Copyright © 2011 for the individual papers by the papers'
authors. Copying permitted only for private and academic
purposes. This volume is published and copyrighted by the
editors of EICS4Med 2011.
Virtual Reality (VR) simulators virtually reproduce the
surgical scenario and allows the user to interact with the
anatomy through different interfaces that could be
surgically realistic or not and that can or can’t embed some
kind of haptic feedback. Even if during last decade many
companies proposed virtual simulators, well described
technical challenges must be still overcome to permit
varied training in a realistic computer generated
environment. These challenges include the development of
realistic surgical interfaces and environments, and most of
all the modelling of realistic interactions between objects
and rendering of the surgical field [
Simulation using physical objects usually involves plastic, rubber and latex models arranged in boxes. These objects are used to render different organs and pathologies and allow to perform specific tasks such as cutting, suturing, grasping or clipping structures. The repetitive performance of a single task allows the trainee to develop the hand-eye coordination and the motor skills before entering the realpatient setting. The actual interaction with simulated anatomy can be considered the principal advantage of physical simulator that, on the other hand, are limited by being restricted to single or few standard anatomical structures and by requiring to buy a new phantom (usually expensive) for each destructive trial. Physical simulators can also be employed as testing environment for the invitro assessment and validation of innovative surgical technologies (like surgical instruments, robots or navigation) [4; 6; 8].
In the last years to overcome limits of the two former
described approaches a new concept of simulation has been
developed: hybrid simulation. It combines synthetic models
with VR, deploying for example mixed-reality, to bridge
the gap between the synthetic mannequin and the computer.
This avoids some of technical difficulties associated with
reproducing the feel of instruments and of human tissue in
a complete virtual environment, while still allowing access
to the advantages of computer simulation in particular for
the trainee performance evaluation, the possibility to enrich
the scene with virtual elements and to give instructions for
the surgical tasks execution [
This paper describes a fabrication strategy to build patientspecific hybrid simulators mixing patient specific synthetic anatomies with virtual reality features. The idea is to overcome the limit imposed by standard anatomy, starting from the elaboration of radiological images to develop a simulator including realistic synthetic organs paired with electromagnetic position sensors and enriched with consistent virtual model of the entire abdomen.
The goal of the present work is to define a strategy to manufacture patient specific silicone organs and pair it with sensors in order to build a physical test bed enriched by a virtual environment in the direction of an hybrid simulators for abdominal surgery.
The simulator is to be used for surgical training, with the chance of surgical performance evaluation, but also as testing environment to assess innovative surgical technologies like surgical robots or surgical navigators. The development of the simulator starts from the segmentation and surface extraction of anatomical components of interest from real medical image data sets. The obtained 3D virtual models are then employed on one side to build the graphic interface, on the other side as starting point to design the moulds for the silicone organs models.
A commercial torso phantom (CLA® OGI Phantom) is used to enfold synthetic organs models in a realistic environment (14). Moreover supporting structures are designed to guarantee the correct positioning of synthetic models inside the commercial mannequin and replicate space constraint and relationships between organs. In this work NDI Aurora® electromagnetic (EM) tracking sensors have been used (Aurora® 5DOF Sensor, 0.5 mm x 8 mm, 2 m) to sensorize organs[3; 16].
The fabrication steps is divided into two principal phases: Images acquisition and elaboration for the 3D virtual models extraction
Fabrication of the sensorized synthetic organs
The virtual environment is obtained through the segmentation of actual radiological datasets. In this first phase it lays the key to obtain non standard anatomies and to choose real anatomies to build up surgical theatre challenging for the trainee.
As first simulator we selected an healthy patient,
anonimized, dataset. The dataset has been segmented to
obtain organs frontiers. For this purpose we used a
semiautomatic tool previously developed in our lab: the
EndoCAS Segmentation Pipeline[
The class of silicone rubbers, which allows an easy
reproduction of objects with complex shape, and an agarose
hydrogel, which closely mimic the mechanical properties of
soft tissues [
More in particular the employed silicones are RTV-TIXO,
and GSP 400 from Prochima® while an agarose powder
from Sigma [
Regarding hollow organs, for example stomach and gallbladder, a process has been studied to embed sensors inside the organ wall, between two layers of silicone. In the following is detailed the procedure for fabricating a sensorized gastric model.
First the positions of 8 Aurora electromagnetic sensors have been identified on the 3D virtual model in function of the clinicians needs. Then it has been fabricated a mould replicating the gastric lumen, with holes in correspondence of planned sensors positions. Figure 2 shows the gastric mould with planned, in virtual Figure 2a, and actual screws positioning used for an exact sensors positioning Figure 2b. In Figure 2c, a first layers of silicone RTV TIXO has been applied on the gastric model; after the silicone curing, Aurora sensors have been positioned between each couple of screws; the thin screws have been removed from the rigid gastric model and a final layer of GSP 400 has been applied, Figure2d. the mould parts. Then, after silicone curing, Aurora sensors have been positioned in correspondence of the predisposed screws. A new layer of RTV TIXO silicone has been applied to properly cover sensors. When the silicone cured, after removing screws, the mould has been closed, ensuring the proper alignment of the two mould parts and using additional silicone to attach the two silicone shells.
RTV TIXO has been chosen to fine reproduce gastric folds, the outer layer of the model instead has been fabricated using GSP 400 that allows to obtain a more uniform and smooth surface.
The solid organs have instead been fabricated building
mould where to inject silicone or hydrogel. In the following
is detailed the procedure for fabricating a sensorized liver
model. In particular the agarose powder has been mixed in
water, heated until almost boiling, and then poured into the
designed mould. Since liver Young modulus varies around
20 KPa [
The process of fabrication started with the application of a layer of silicone RTV TIXO in the internal surface of both
Finally the prepared agarose gel has been injected into the closed mould. The final result can be seen in Figure 3d. In order to guarantee the correct positioning of synthetic organ models inside the commercial mannequin it has been decided to fabricate a supporting structure, that fits perfectly inside the commercial mannequin, and allows to insert synthetic organs models respecting their actual anatomical location in the patient.
At this aim, after positioning some radio opaque markers on the mannequin, another CT scan has been executed, then a registration between patient images and mannequin ones has been performed and finally the segmentation obtained from patient CT images has been loaded on the mannequin greyscale images.
This allowed to segment the empty space between the mannequin abdominal cavity and the organs models and thus to extract the 3D model of a supporting structure for patient silicone organs that fits perfectly inside the commercial mannequin abdomen.
Then the segmented model has been refined to optimize its shape and allow an easy positioning inside the mannequin and an easy insertion of the organs. Finally the designed supporting structure has been fabricated using the 3D printers.
A set of abdominal walls has been built to complete the
simulator. Such walls have been added in order to simulate
the
pneumoperitoneum
during
robotic
or traditional
The covers are fabricated in thermoformable plastic
material modelled in the right shape. They are provided
with some soft silicone windows in strategic positions to
allow the insertion of the instruments access ports.
In Figure 4 it is showed the mannequin with 4 organs
inside: liver gallbladder stomach and pancreas. The organs
are correctly arranged thanks to the supporting structure[
A software interface that acquires signals coming from the embedded sensors and emulates organs deformations on a virtual scenario (Figure 5) has been implemented to show the potentialities offered by hybrid simulation.
The software is written in c++ and deploys the openSG opensource libraries to deal with openGL window and the Qt libraries to build the interface. laparoscopic interventions.
The 3D model of the organs are visualized inside the software.
It is important to underline that the virtual environment is enriched respect to the real one by the possibility to add all abdominal segmented structures, i.e. vessels and kidneys. Color information are added to virtual model using vertex coloring techniques in order to increase the realism of the virtual scenario.
The physics mannequin is registered with the virtual anatomy with a point based registration algorithm. This is necessary to align the reference frame of the aurora localizer, that read the sensors inside the mannequin, with the CT reference frame in which the virtual anatomy is referenced.
The transformation between CT and Aurora reference frames is computed using the radiopaque artificial markers positioned on the commercial mannequin. Marker positions are acquired with the Aurora digitizer. Then the registration matrix is calculated through a least square error algorithm. Starting the simulation the Aurora localizer starts reading position information coming from sensors.
Each sensors position is registered to find its coordinates in the mesh reference frame; these coordinates are then considered as “control points” to apply the deformation function for reproducing the deformation actually imposed to the organs.
The class of Free Form Deformations methods are the most
spread methods to modify the shape of geometrical objects
when described with vertices and faces [
Below the mathematical description of the method is showed.
࢚࢞ ൌ ࢚࢟ ൌ ࢠ࢚ ൌ ࢚࢟ି ିࢠ࢚ ି࢚࢞ ൫࢙ ቀ࢙ ࢙൫ ୀ ୀ ૡ ૡ ૡ ࢚ ࢞ െ ࢙ ࢚ ࢟ െ ࢙ ࢚ ࢠ െ ࢙
࢞ ൯ ࢋ ି ࢊ ࣌
࢟ ቁ ࢋ ି ࢊ ࣌
ࢠ ൯ ࢋ ି ࢊ ࣌ ୀ ࢊ ൌ ቚ ሬ ሬ ሬ ሬ Ԧ െ ࢙ ሬ ሬ ሬ ሬ Ԧ ቚ Ԧ ௧ is the position of a mesh vertex at the instant t ݏ ௧ is the position of the sensor n at the instant t n is the sensor number (in our case from 1 8) dn is the Euclidean distance between the mesh vertex and the sensor n σ is the standard deviation of the distribution.
The latter parameter describes the amplitude of the gaussian bell and in this application it somehow reflects the material property of the organ describing how much wide the deformation is. The Gaussian distribution of the ʹ distances, ࢋ െ ݀ ࣌ ݊ ǡ is evaluated for each mesh vertex and each sensor “off line” when the mesh is loaded. So that, during the simulation, the amount of computational load to be done on the fly is reduced and the simulation is speeded up because it’s only needed to check precomputed values in a local area only.
Steering the σ parameter we obtained a simulator that reproduce virtually the physical interaction with the anatomy (Figure 6).
Moreover in order to add preliminary metric features to the simulator we inserted a visual effect that colours the deformed part in function of the deformation entity. This is to virtually transmit if a deformation is too strongly imposed and furthermore represent the first step to go towards bleeding anatomies and more complex virtual features.
In this work we describe how to develop surgical simulators using a new paradigm.
In particular it is shown a strategy to build up a complete hybrid simulator for surgical training.
Regarding the physical phantom the strategy easily allow to modularly build surgical scenarios. The mannequin was showed to clinicians that confirmed the high degree of realism and the correct arrangement of organs inside the abdomen.
Regarding the correspondence between real and virtual deformation real-time performances have been reached. At this moment only a simple deformation for the stomach is implemented but an integration of more complex functions is planned. The aim is to reach integration of enough functions in order to simulate a complete intervention.
For example next steps will regard the development of virtual deformation for liver and gallbladder in order to simulate a complete colecistecthomy.
This type of simulator overcomes the limits imposed by the use of standard anatomies and represents the first step for developing more complex hybrid platforms, that links benefits coming from having physical scenario to interact with (mostly in terms of force feedback) with virtual elements that enrich the realism of the simulation and can offer to trainee a complete environment to learn surgery from a single task to more complex ones.
While a complete evaluation as for this training purpose is currently underway, initial feedback from clinicians using the system has been positive. The winning strategy to build simulators not starting from standard anatomies but describing a wide variety of anomalies and pathological scenarios is very encouraged from surgeons.
The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement num. 224565 (ARAKNES Project)