A GEANT4 Web-based Application to Support Intra- Operative Electron Radio-Therapy using the European Grid Infrastructure Carlo Casarino, Giorgio Russo, Giuliana Carmela Giuseppe La Rocca, Roberto Barbera Candiano Italian National Institute of Nuclear Physics, Division of IBFM CNR - LATO Catania - Italy Cefalù - Italy Department of Physics and Astronomy of the University of Catania - Italy Cristina Messa Giovanni Borasi IBFM CNR - LATO University of Milano Bicocca, Milan - Italy Cefalù – Italy University of Milano Bicocca, Milan – Italy San Gerardo Hospital, Monza - Italy Maria Carla Gilardi Gianluca Passaro IBFM CNR - LATO Consortium COMETA, Catania, Italy Cefalù – Italy University of Milano Bicocca, Milan – Italy San Raffaele Scientific Institute, Milan - Italy Abstract— Radiotherapy techniques permit to deliver ionizing This technique consists in delivering ionizing radiations (X- radiations (X-rays, photons, electrons, protons, etc) inside rays, photons, electrons, protons, etc) inside cancerous tissues cancerous tissues to kill the abnormal-cells. Radiotherapy related to control or kill abnormal-cell growth. Radiation therapy activities, as the optimization of the therapeutic radiation dose to normally has also several applications in non-malignant the patient, worker radioprotection, performance controls and conditions, such as the treatment of trigeminal neuralgia, technical innovations of linear accelerators, are strongly based on acoustic neuromas, severe thyroid eye disease, the prevention the ability to predict the dose distribution. Monte Carlo of keloid scar growth, vascular restenosis, and heterotopic simulations are the most accurate tools in this field but, ossification. The use of radiation therapy in non-malignant unfortunately, they require large computing power to achieve conditions is limited partly by worries about the risk of accurate results in reasonable times. In the last years, advanced radiation-induced cancers. In addition, radiation therapy can cancer treatment clinical and research communities have adopted e-Infrastructures to reduce this gap. The present paper also be used as part of curative therapy, to prevent tumor reports on the developments of a computing facility for helping recurrence after surgery to remove a primary malignant tumor physicians, radiotherapists and medical physical in using modern (for example, early stages of breast cancer). R&E networking and distributed resources to address some From the medical perspective, the radiation of the cancer technical and clinical Intra-Operative Electron Radio-Therapy cell aim to damage its DNA content and leads rapidly to its (IOERT) needs (e.g. the design of the linear accelerator death since no mechanisms to auto-recover from a damage are collimation system or the optimization of the patient therapeutic available in the cancer cells. Another important aspect which dose distribution). has to take into account when we talk about radiotherapy is the Keywords— IOERT, Monte Carlo simulation, GEANT4, radio-protection of tissues not affected by the cancer. To spare Distributed Computing Resources, Catania Science Gateway normal tissues, such as skin or organs, some different technics framework. are used such as: shape the beams during the irradiation, consider many entrance points, fluence modulation, or, in the intraoperative treatment, shift the normal tissue and irradiate I. INTRODUCTION the tumor directly. Thanks to this different technics it is Radiotherapy is one of the most successful non-invasive possible to guarantee a much larger absorbed dose level in the methodology for the treatment of different types of cancers. targeted tissues and protect the normal tissues. The rapid development and adoption of radiotherapy simultaneously allowing to process in parallel several histories techniques have been possible thanks to the introduction a new [6]. generation of linear accelerators (often shorted to linac) that greatly increases the velocity of charged subatomic particles or However, the best reliable and tested solution is to use the ions by subjecting the charged particles to a series of “raw” Monte Carlo method in distributed computing oscillating electric potentials along a linear beam-line. The environment. Parallelized computation, in fact, offers to serial design of a linac depends on the type of particle that is being processing a natural and more appropriate practical support. A accelerated: electrons, protons or ions. Linac range in size from time-heavy simulation (of N histories) may be splitted in N’ a cathode ray tube (which is a type of linac) to the 2-mile (3.2 smaller simulations each one executed on different CPU. For km) long linac at the SLAC National Accelerator Laboratory in example, a Grid infrastructure offering 200 CPU core for user, Menlo Park, California. permits then to reduce computing-times of a factor of 200 (N/200=N’). To address the problematic described above and Computer simulation of radiation transport and subsequent support a new and advanced radiotherapy technique, the dose distribution estimate is of fundamental importance, both iort_therapy application, has been developed. in clinical and research activities. In the clinical perspective some commercial software, such as Therapeutic Planning The outline of this paper is as follow: in section II we provide IOERT. In section III we will introduce some Systems (TPS), are commonly in use to plan, approve and in situ verify, patient radiotherapy sessions. TPS scope is to background information about the iort_therapy application optimize the dose distribution release in the treatment zone and developed by researchers of the IBMF CNR-LATO [7,8]. In implement the healthy tissues protection. Unfortunately these Section IV an overview of the reference model used for tools, often, offer unsatisfactory accuracy dose calculation implementing a Science Gateway, which is now used by methods, especially in inhomogeneous regions as soft tissue – physicians, radiotherapists and medical physical to address bone boundaries [1]. Moreover there are others clinical some technical and clinical Intra-Operative Electron Radio- activities, involving medical physicists, where simulations may Therapy (IOERT) needs, will be described. Finally, we will be a support in the development of procedures for the report about some outcomes and future prospective. verification of the linac specifications [2] or impractical activities where the radiation transport simulations are II. THE INTRA-OPERATIVE ELECTRON RADIOTHERAPY irreplaceable, as well adjustments in radiation shielding room The Intra-Operative Electron Radiotherapy (IOERT) is a design [3], or even impossible, when direct measurements of technique that allows treatment of the cancerous cells directly physical quantities are conceptually not executable as in the in the operating theatre after the surgeon have removed the case of energy of the particles escaped from the head tumor tissues [9]. The high ionizing beam is delivered through accelerators [4]. special cylindrical applicators positioned during the surgery. In radiotherapy research, linac manufacturer designers and The protection of internal normal tissue from radiation leakage medical physics study the best solution to generate and is a critical point [4]. In breast treatment, for example, it collimate the radiation beam and/or to optimize the patient involves the surgeon positioning a shielding metal disc radioprotection. In this context, Monte Carlo simulation between the deep face of the patient’s residual breast and the method offers the most exact software tool to calculate pectoral muscle. radiation transport and energy deposition but, unfortunately, For IOERT dedicated and mobile accelerators, such as require large computing power to achieve accurate results in NOVAC7 (NRT, Aprilia, Italy) [10], Liac (Sordina SpA, Italy) reasonable times [1]. In fact, particles transports (histories) are [11], and Mobetron (IntraOp Medical, Inc. Santa Clara, CA) processed sequentially and to achieve a correct statistical [2], are employed . The main characteristics of these machines uncertainty in dose values (in most case within 2% the relies on the capability to be placed near the patient’s bed, the experimental values) the number of histories may increase up orientation of the beam in different directions, and the delivery to 108. In these conditions, a single standard CPU (3 GHz) can of very high dose rates compared to traditional linear take even some months to complete a single simulation. accelerators. With the availability of these mobile electron On the other hand, useful (per simulation) times may vary linear accelerators, IOERT procedures have become from few hours, as in the case of a TPS, to some days for widespread, at least in major clinical centers, giving the company and research requests. possibility that greater numbers of patients can be treated. To improve simulation efficiency, in the last three decades, NOVAC7 produces electron beams of 4, 6, 8, and 10 MeV the direct synergy between scientific-medical community and nominal energies to perform treatments at different tissue linac manufacturers-commercial vendors has produced many depths. Applicators (collimators) with different diameters from alternative no MC solutions, as convolution/superposition 3 to 10 cm are available. The field of radiation is collimated by methods or approximate, as MC simulation associated with cylindrical perspex applicators with different diameters and tilt variance reduction techniques [1,5]. Another innovative and angles. Applicators are available with diameters ranging from 3 promising expedient it is represented by Graphical Processing to 10 cm and are either parallel (0°) or beveled (15°, 22.5°, 30° Unit (GPU) implementation of MC methods. Using the Single and 45°). The collimation system is of a hard docking type: the Instruction Multiple Data structure presents on the graphic applicators are rigidly attached to the accelerator. Each board, the same function may be execute many times collimator is made up of two parts: the upper applicator, that is fixed to the radiant head, and the lower applicator, that is placed on the surface to radiate. To perform the treatment, the radiant head is slowly moved to place the upper and lower part erroneous clinical set-up. In these situations the healthy tissues of collimator in line; a ring allows the two parts to be coupled. are exposed to dangerous ionizing radiations. Each collimator corresponds to a Source Skin Distance (SSD) of 80 cm except for the collimator of 10 cm of diameter that corresponds to a SSD of 100 cm. In figure 1, the NOVAC7 is shown in the surgery ready for operation. The collimation system starts from the end of the accelerator head. The latter acts as the primary collimation system. Connected to this is the secondary collimator, consisting of a larger PMMA cylinder that includes a fixed adaptor and a final collimator. Fig. 2. 2D dose distribution inside the water phantom (reproducing tissues) using a 10 MeV beam collimated with the collimator 10 cm diameter when the shielding disc is errouneously misaligned and tilted respect the beam collimation system . The ionizing electron beam comes from left Fig. 1. The mobile linear electron accelerator NOVAC7 in the surgery room. to right. III. THE GEANT4 - IORT_THERAPY APPLICATION Figure 3 shows the Graphical User Interface (GUI) of iort_therapy. The GUI is implemented with QT4 libraries. By To contribute to the study of intra-operative procedures and interactive windows (Help, Commands, Command outputs and to the optimization of the machine, we developed a specific History) the geometric and physical characteristics of the Monte Carlo (MC) application, “iort_therapy”, using the clinical setup are chosen and visualized. On the right side, the GEANT4 toolkit [12]. Today it represents one of the advanced smaller tube represents the accelerator head and includes the examples of the official GEANT4 release (9.5 version) and it exit window, the monitor chamber and the primary collimation can be freely downloaded [13]. system. The larger and longer tube is the second collimation The application simulates the electron beam and the system. The water phantom (simulating patient tissues) is collimation system of the NOVAC7 and addresses several represented by the larger box. It includes a smaller box technical and clinical issues related to the IOERT technique representing the sensitive detector (70 mm depth x 150 mm x such as: the design and optimization of the collimation system; 150 mm surface). The shielding disc is positioned in the middle the study of patient radio-protection aspects; the optimization of this structure. To provide a relative error comparable to that of the therapeutic dose distribution [4] and the development of obtained experimentally, 108 histories per simulation must be procedures for verification of the linac specifications [14]. generated. The application allows the calculation of dose distribution in water or in other materials. Moreover, it gives the possibility to choose between different clinical setups and to optimize radio-protection of normal tissues. Using macro commands, the user can easily select the appropriate collimator, the phantom and detector characteristics, the configuration of the shielding disc employed in breast treatments, the initial conditions of the electron beam (in particular, position and angular distributions), and the appropriate physics list, i.e., the command list for the physical processes. In Fig. 2 are shown typical critical situations when the shielding disc is incorrectly positioned (misaligned and tilted) Fig. 3. Graphical User Interface (GUI) of the Geant4 example iort_therapy respect the collimation system, to reproduce potential From a technical point of view the application is embarrassingly parallel and needs a prior-installation of Geant4 as well as some additional libraries. The typical output file produced by the tool is a dose distribution in a volume (sensitive detector) of 300 x 300 x 140 voxels. The size of output files vary from few MBs to tens of GBs. The application’s workflow is a highly computing demanding problem. On a single CPU (with 3 GHz core) it would require about 200 CPU days to produce the dose distribution with the required precision. The same Monte Carlo computation must also be repeated many times starting from the same input file which contains the macro. IV. THE REFERENCE MODEL Fig. 4. The reference model for the IOERT Science Gateway This section describes the present status of the IOERT The IOERT Science Gateway is an interdisciplinary work Science Gateway which has been developed by the INFN of that involves experts belonging to different scientific areas. The Catania for helping physicians, radiotherapists and medical main purpose of the Science Gateway is to build a unique entry physical to use the pan-European GÉANT network, which point for physicians, radiotherapists and medical physical operates at speed of up 100 Gbps, and the computing facilities needing to optimize the patient’s radioprotection and/or set up of the COMETA Consortium[15] to optimize the patient’s the best collimation system configuration to deliver a high dose radioprotection and/or set up the best collimation system to the target tissues. When users register to the Science configuration to deliver a high dose to the target tissues. Gateway, they are mapped to valid Grid users so that, through a single sign-on mechanism, they can use the portal services The IOERT Science Gateway is based on the Catania and the underlying computational and storage resources. In fig. Science Gateway framework [16] which has been successfully 5 is shown a standard JSP page and the relative logic portlet adopted in the context of several EU funded projects such as that has been implemented for collecting the input parameters DECIDE [17], EUMEDGRID-Support [18], GISELA [19], and send a bunch of Monte Carlo simulations on the computing CHAIN [20] and INDICATE [21]. In a nutshell, the Science facilities. Gateway uses JSR 286 standards (also known as “portlet 2.0”) to develop advanced tools that can be re-used and combined for resolving different and complex problems and Liferay as a portlets container. The access to the Science Gateway relies on the use of Identity Federations based on the SAML 2.0 standard and on its implementation done by Shibboleth [22] and SimpleSAMLphp [23]. A “catch-all” Identity Federation to gather all the users who do not already belong to any federations and a special IdP that allow people to get authenticated with the same credentials they already have with the most common and populated social networks (Facebook, Google+, LinkedIn, Twitter, Windows Live and Yahoo!) are also supported. The following authorization mechanism makes possible to simplifying the access to the infrastructure and widening its user base. The access to the underlying computing infrastructure relies on the SAGA standard [24] and on its JSAGA implementation. A specific JSAGA Adaptors is used to run Monte Carlo simulations on the COMETA computing facilities. For the authentication point of view, all the grid transactions are secured using a robot certificate installed on a eToken PRO 64KB USB smart card plugged on a dedicated server which is in charge to generate, using PKCS#11 and JAX-RS standards, proxy certificates for the Fig. 5. The IOERT input and job submission page users. Last but not least, the IORT Science Gateway has been implemented in order to be fully complaint with the strict rules The input page consists of several input fields with, at the of the EGI VO Portal [25] and EGI Grid Security Traceability bottom, a link to start the running of the Monte Carlo and Logging Policy [26] policies. The reference model of the simulations. Once the simulations have been submitted, users IORT Science Gateway is shown in figure 4. can follow its status and download its output at the end through the links of the MyWorkspace portlet which appears on the left side of the browser when users are signed in. The MyWorkspace portlet aims to handle users’ jobs and data through the respective and self-explaining functions MyJobs REFERENCES and MyData. As an example, figure 6 shows a typical view of MyJobs. [1] I. J. Chetty, B. Curran , J. E. Cygler, J. J. DeMarco, G. E. Bruce, A. Faddegon, I. Kawrakow, P. J. Keall, H. Liu, C. M. Charlie Ma, D. W. O. Rogers, J. Seuntjens, D. Sheikh-Bagheri, J.V. Siebers. Report of the AAPM Task Group No. 105: Issues associated with clinical implementation of Monte Carlo-based photon and electron external beam treatment planning. Med. Phys. vol. 34, pp. 4818-4853, December 2007 [2] Mills M D, Fajardo L C, Wilson D L, Daves J L and Spanos W J 2001 Commissioning of a mobile electron accelerator for intraoperative radiotherapy Am. Coll. Med. Phys. 2 121-130 [3] National Council on Radiation Protection and Measurement. Structural Shielding Design and Evaluation for Medical Use of X-Rays and Gamma-Rays up to 10 MeV (NCRP 49). Office of the Federal Register. U.S.A. 1976 [4] Russo G, Casarino C, Arnetta G, Candiano G, Stefano A, Alongi F, Borasi G, Messa C, Gilardi MC. Dose distribution changes with shielding disc misalignments and wrong orientations in breast IOERT: a Monte Carlo - GEANT4 and experimental study. Journal of Applied Clinical Medical Physics, 2012; 13(5):1526-9914 [5] P. Downes, G. Yaikhom, J.P. Giddy, D.W. Walker, E. Spezi and D.G. Lewis. High-performance computing for Monte Carlo radiotherapy calculations. Phil. Trans. R. Soc. vol. 367, pp. 2607–2617, May 2009. [6] L. Jahnke, J. Fleckenstein, F. Wenz and J. Hesser. GMC: a GPU implementation of a Monte Carlo dose calculation based on Geant4. Phys. Med. Biol. vol.57, pp. 1217–1229, February 2012. Fig. 6. MyworkSpace portlet showing the list of IOERT simulations [7] CNR-IBMF. Available from: www.ibfm.cnr.it [8] LATO HSR Giglio. Available from www.polooncologicocefalu.it/ V. OUTLOOKS [9] Veronesi et al 2001 A preliminary report of intraoperative radiotherapy Thanks to the advanced R&E networking and the (IORT) in limited-stage breast cancers that are conservatively treated computing facilities of the COMETA consortium, the Eur. J. Canc. 37 2178–2183. iort_therapy application produces several MB of compressed [10] Pimpinella M, Mihailescu D, Guerra A S and Laitano R F 2007 Dosimetric characteristics of electron beams produced by a mobile data for each simulation reducing the consumed computing accelerator for IORT Phys. Med. Biol. 52 6197–6214. time to 10 CPU hours per run. The raw data generated are then [11] Soriani A, Felici G, Fantini M, Paolucci M, Borla O, Evangelisti G, refined offline using some MATLAB sub-routines. In figure 7 Benassi M and Strigari L 2010 Radiation protection measurements is shown the Wall Clock time consumed by the application around a 12 MeV mobile dedicated IORT accelerator Med. Phys. 37 starting from Oct 2012 when the application started to run in 995-1003. production. [12] Allison J, Amako K, et al 2006 Geant4 Developments and Applications. IEEE Trans. Nuc.l Sci. 53 270-278. [13] Geant4.9.5. Available from: http://geant4.web.cern.ch/geant4/ [14] Bjork P, Knoos T and Nilsson P 2004 Measurements of output factors with different detectors types and Monte Carlo calculations of stopping- power ratios for degraded electron beams Phys. Med. Biol. 49: 4493- 4506. [15] The COMETA Consortium web site. Available from www.consorzio- cometa.it [16] Barbera R, et al 2012 The DECIDE Science Gateway. Journal of Grid Computing (2012), Vol10:689-707. DOI 10.I007/s10723-012-9242-3 [17] The DECIDE Project web site. Available from www.eu-decide.eu [18] The EUMEDGRID-Support Project web site. Available from www.eumedgrid.eu [19] The GISELA Project web site. Available from www.gisela-grid.eu [20] The CHAIN Project web site. Available from www.chain-project.eu Fig. 7. WallClock time consuded by the iort_therapy application from Oct to [21] The INDICATE Project web site. Avaialbe from www.indicate- Nov 2012 project.eu [22] The Shibboleth Project web site. Available from As it is shown by the figure the trend is increasing. With shibboleth.internet2.edu this data rate the application is now able to address some of the [23] The SimpleSAMLPhp web site. Available from simplesamlphp.org Companies and Research Community requirements. As a [24] SAGA Standard specifications. Available from future prospective, a new iort_therapy version is under www.gridforum.org/documents/GDF.90.pdf development for letting physicians to use this application in [25] https://documents.egi.eu/public/ShowDocument?docid=80 hospitals. [26] https://documents.egi.eu/public/ShowDocument?docid=81