Modern technologies give a great support to the minimally invasive surgical procedures through medical image processing and visualization, 3D organ's reconstruction and intra-operative surgical guidance. The practice of Minimally Invasive Surgery is becoming more and more widespread and is being adopted as an alternative to the classical procedure. This technique presents some limitations for surgeons. In particular, the lack of depth in perception and the difficulty in estimating the distance of the specific structures in laparoscopic surgery can impose limits to delicate dissection or suturing. The presence of new systems for the pre-operative planning can be very useful to the surgeon. In this paper we present a visualization and interaction system that allows surgeon to visualize the 3D model of the patient's organs built from the CT images. Different visualization modalities are available according to the surgeon needs in laparoscopy and an Augmented Reality application permits the choice of the best insertion points of the trocars on the 3D virtual model and the visualization of these points on the real patient's body. Two case studies have been considered. The system can be used as support for the diagnosis, for the surgical preoperative planning and also as visual support during the surgical procedure.
Minimally Invasive Surgery (MIS) has become very important and the researches in this field are ever more widely accepted because these surgical techniques provide surgeons with less invasive means of reaching the patient’s internal anatomy and allow entire procedures to be performed with only minimal trauma to the patients. As a promising technique, the practice of MIS is becoming more and more widespread and is being adopted as an alternative to the classical procedures.
The diseased area is reached by means of small incisions on the body; specific instruments and a camera are inserted through these ports and what happens inside the body is shown in a monitor. The surgeon does not have a direct vision of the organs and so he is guided by the camera images. This surgical approach is very different from the open surgery where the organs can be fully visualized and handled.
The advantages of using this surgical method are evident in the patient because the trauma is reduced, the postoperative recovery is almost always faster and the scarring is reduced. Despite the improvement in outcomes, these techniques show their limitations for the surgeons. In particular, the lack of the perception of the depth and the difficulty in estimating the distances of the specific organs in laparoscopic surgery can impose some limits on delicate dissection or suturing.
Anyway, the overall risk of complications is of 8.0% in laparoscopy versus 15.2% in laparotomy. Among these, more than 50% of laparoscopic complications occur during the initial entry into the abdomen.
The modern medical imaging acquisition associated to the
medical image processing could lead to an improvement in
patient care by guiding the surgeons. The medical image
processing allows detecting and identifying anatomical and
pathological structures and building 3D models of the
patient’s organs that could be used to guide the surgical
procedures. Many research teams have dealt with the task
of segmentation and have developed techniques that allow
automatic or interactive extraction of the patient’s organ
models from CT-scan or MRI [
The Augmented Reality (AR) technology can provide the
advantage of a direct visualization in open surgery also in
minimally invasive surgery and can increase the physician's
view of his/her surroundings with information gathered
from patient medical images [
Samset et al. [
Bichlmeier et al. [
Navab et al. [
De Paolis et al. [
In this paper we present an advanced platform for the visualization and the interaction with the 3D patient models of the organs built from CT images.
The developed application allows the surgeon to choose the points for the insertion of the trocars on the virtual model, to simulate the insertion of the surgical tools in order to verify the correctness of the insertion sites and to overlap the chosen entry points on the real patient body using the Augmented Reality technology.
The system could be used as support for a more accurate diagnosis, in the surgical preoperative planning and also for an image-guided surgery.
THE CASE STUDIES In MIS the use of the registered images of the patient is a prerequisite both for the pre-operative planning and the guidance during the operation. From the medical image of a patient (MRI or CT), an efficient 3D reconstruction of his anatomy can be provided in order to improve the standard slice view; colors associated to the different organs replace the grey levels in the medical images.
In our case study the 3D models of the patient’s organs
have been reconstructed using segmentation and
classification algorithms provided by ITK-SNAP [
The first case study, shown in Figure 1, is a two-year-old child with a benign tumor of the right kidney.
The second case study, shown in Figure 2, is a twelve-yearold child with a tumor of the peripheral nervous system (ganglioneuroma). THE USER INTERFACE The developed application is supplied with a specific user interface that allows the user to take advantage of the feature offered by the software. The application is provided of 4 sections with the aim to provide support to the surgeons in the different steps of the surgical procedure such as the study of the case, the diagnosis, the preoperative planning, the choice of the trocar entry points and the simulation of the surgical instruments interaction. Starting from the models of the patient’s organs, the surgeon can notice some data about the patient, collect information about the pathology and the diagnosis, choose the most appropriate positions for the trocar insertion and overlap these points on the patient’s body using the Augmented Reality technology. In this way it is possible to use this platform for the preoperative surgical planning and during the real surgical procedure too. In addition, it could be used in order to describe the pathology, the surgical procedure and the associated risks to the child’s parents, with the aim of obtaining informed consent for the surgical procedure. In the developed application, all the patient’s information (personal details, diseases, specific pathologies, diagnosis, medical images, 3D models of the organs, notes of the surgeon, etc.) are structured in a XML file associated to each patient.
A specific section for the pre-operative planning includes the visualization of the virtual organs and the physician can get some measurements of organ or pathology sizes and some distances; this section is shown in Figure 3. By means of a detailed view of the 3D model, the surgeon can choose the trocar entry points and check if, with this choice, the organs involved in the surgical procedure can be reached and the procedure can be carried out in the best way. THE AUGMENTED REALITY SECTION Sometimes, using the standard insertion points for the surgical tools, also a simple surgical procedure can be very difficult due to the specific anatomy of the different patients. The surgeon can have some difficulties to reach the specific organ or the interaction of the surgical tools can be very hard. In this case the surgeon has to choose another insertion point in order to be able to carry out the surgical procedure in the most suitable way.
Our aim is to avoid the occurrence of this situation during the real surgical procedure using the visual information provided by means of the 3D models of the patient’s anatomy.
In the developed application, in order to verify if the chosen insertion points allow properly reaching the specific organ interested by the surgical operation and permitting to carry out the procedure in a correct way, it is also possible to simulate the interaction of the surgical instruments. We also use the AR technology in order to visualize on the patient’s body the precise location of the selected points on the virtual model of the patient.
For the augmented visualization, in order to have a correct
and accurate overlapping of the virtual organs on the real
ones, a registration phase is carried out; this phase is based
on fiducial points. Using the augmented visualization, the
chosen entry points for the trocars can be visualized on the
patient’s body in order to support the physician in the real
trocar insertion phase.
In our application, we have used the Polaris Vicra optical
tracker of NDI [
Usually the tracking technology is already in the operating rooms and provides an important help to enhance the performance during the real surgical procedures. CONCLUSIONS AND FUTURE WORK The developed application offers a tool to visualize the 3D reconstructions of the patient’s organs, obtained by segmentation of a CT slices, and to simulate the placement of the trocars in order to verify the correctness of the insertion sites. A complete user interface allows a simple and efficient utilization of the developed application. Furthermore the system retains patient and pathology information that the surgeon can insert and includes an Augmented Reality module that supports the placement of the trocars on the patient’s body during the real surgery procedure. An accurate integration of the virtual organs in the real scene is obtained by means of an appropriate registration phase based on fiducial points.
The developed platform can support the physician in the diagnosis steps and in the pre-operative planning when a laparoscopic approach will be followed. This support could also lead to a better communication between physicians and patient’s parents in order to obtain their informed consent. The platform has been tested on study cases already operated by the surgeon; the future work will be the validation of the developed application on a new study case by following all the steps from the diagnosis to the preoperative planning and to the first phase of the real surgical procedure.
The building of a new Augmented Reality system that could also help the surgeon during the other phases of the surgical procedure has been planned as future work. ACKNOWLEDGMENTS This work is part of ARPED Project (Augmented Reality
Application in Paediatric Minimally Invasive Surgery) funded by the Fondazione Cassa di Risparmio di Puglia, Italy. The aim of the ARPED Project is the design and development of an AR system that can support the surgeon through the visualization of anatomical structures during a laparoscopic surgical procedure.