The technologies of the augmented and virtual reality have a special role in medical education as an additional tool for training professional skills in pre-clinical practice. In the paper, we describe the development of a virtual reality simulator with immersion in VR scene for dentist ofice and simulation of tooth drilling. Such kinds of simulators would contribute to evolving capacities of motor skills and hand-eye coordination. The VR simulator for dental students training is developed for Oculus Quest 2 VR headset with six degrees of freedom. The Marching Cubes algorithm is chosen as an optimal decision for autonomous VR headsets, the computational power of which is much lower than PCs. The main stages of the development of tooth drilling simulation are considered. They include voxelization, marching cubes algorithm, collision detection, and detection of penetration depth of the dental drill. The experience of VR scene using for dental students training has been piloted at the Faculty of Dentistry at the V. I. Vernadsky Crimean Federal University. To evaluate the pilot study we used a satisfaction questionnaire, which evaluated the realism of tooth 3D model drilling and the realism of VR scene for the creation of a dentist's ofice atmosphere.
The augmented and virtual reality are considered as key technologies in education of the 21st
century and are used to study complex abstractions [
In recent years, the number of applications based on the augmented and virtual reality for education has increased rapidly. This is due to the capabilities of such technologies to immerse in nEvelop-O LGOBE
https://www.researchgate.net/profile/Yulia-Dyulicheva (Y. Yu. Dyulicheva); https://www.researchgate.net/profile/Yekaterina-Kosova (Y. A. Kosova)
CEUR
Workshop
Proceedings
htp:/ceur-ws.org
IS N1613-073
a fascinating world and demonstrate non-visible processes and phenomena through the
involvement of students in educational process. The teachers can create their own applications with
AR/VR thanks to special software such as constructors and visual development environments
[
Medical education is practice-oriented. So, medical education faced the challenge of creating
the conditions to acquire the skills of pre-clinical practice with the help of cutting-edge
technologies. The technologies of augmented, virtual and mixed realities can be considered as advanced
technologies for the creation of efective teaching simulators, but the teacher control on each
stage of virtual simulator usage by dental students remains very important with instructions
from a teacher about how to use special software for acquiring of manual and technical skills
[
The rapid development of AR/VR applications in medical education and in medicine, in general, is explained by the emergence of 3D scanners appearance allowed for the creation of fairly accurate and realistic digital models in diferent applied fields.
The technologies of augmented and virtual reality find more and more usage in the learning
of the maxillo-facial and dentistry disciplines on the stage of pre-clinical practice and in real
practice, for example, in dental implantology for coupling 3D models with real objects and
dynamic navigation [
The efectiveness of AR/VR technology in dentistry education is confirmed in many investigations as shown in table 1.
However, the use of virtual reality in predoctoral dental education is still limited [20] and needs to be explored further.
The aim of our research is to develop a virtual reality simulator for the teaching of dental students. The development process for a virtual reality simulator depends on the selected hardware and software. To implement the simulator aimed at dental students teaching, the Oculus Quest 2 VR headset was chosen by authors as the hardware and Unity 3D as the software.
This study was designed as a pilot study. As a testing result of VR scene usage, we studied
the opinions of two experts and three graduate students from the Faculty of Dentistry at
the V. I. Vernadsky Crimean Federal University. We used a satisfaction questionnaire, which
evaluated the realism of tooth 3D model drilling and the realism of VR scene for the creation of
a dentist’s ofice atmosphere (table 2).
Llena et al. the learning of the AR marker recognition Results of AR efectiveness are
[
OpenGL of anatomical structure
Zafar and head and neck mixed reality technology Results of AR app efectiveness are
Zachar [
At the stage of 3D models deformation for simulation of dental operation performance the mathematical and computational methods, image processing, and computer vision methods are used. Kim et al. [21] used a voxel-based collision model and spray efect based on particle systems to simulate tooth restoration and ultrasonic scaling. There are exist many collision detection methods between virtual objects in real time that, in general, are based on intersections of bounding volumes of virtual objects. One of the most popular algorithm for collision detection is Axis-Aligned Bounding Boxes (AABBs) and for improvement of collision performance – hierarchy of R-trees [22], BVH based method for detection of probabilities of interactions [23], apriori algorithm of collision detection [24]. Rhienmora et al. [25] propose to use AABB algorithm for detection of collision between virtual tooth and virtual cylindrical cutting bar. Wu et al. [26] considered criteria for dental drilling simulation such as fidelity of force, material removal, computational eficiency, and stability. They used voxel-based haptic rendering and real-time visual rendering for the handling of tooth model surface and collision detection with dental drill model.
The approaches to the detection of penetration depth of dental drill have an important role to simulate tooth drilling. Wu et al. [26] considered the sum of resistance forces at all boundary voxels of the drill, drill moving velocity, and material removing velocity. Rhienmora et al. [25] used the proportional dependence of the reaction force and penetration depth of dental drill and proposed force filtering method with possibility of large magnitude control of the force. Zhao et al. [27] proposed to evaluate the penetration force by integrating the unit force acting on the outer voxels of the tool.
For our purpose of tooth drilling simulation we search approach with the following requirements: the algorithm must have a suficiently high accuracy, which will be close to the capabilities of a real dental drilling; the algorithm would be able to edit meshes with diferent topologies, including meshes with “holes”, the algorithm has to be optimized for mobile processors and has to be parallel implementation. The last two points are due to the specifics of the platform for which the simulator is being developed: autonomous VR headsets have an integrated mobile processor and a graphics chip, the computational power of which is much lower than PCs. The most suitable algorithm is “Marching Cubes”. Oculus Quest 2 VR headset with six degrees of freedom is optimal decision for the requirements above.
Recently, the Oculus Quest headset is widely used and allows us to immerse in VR environments with high quality in medical education. Meese et al. [28] described the experience of Oculus Quest usage in the cardiopulmonary resuscitation training. Kang et al. [29] developed DeepHandsVR based on a hybrid approach that combined virtual reality with deep learning algorithms to improve interaction in an immersive environment for Oculus Touch and HTC Vive controllers.
The Oculus Quest 2 virtual reality headset and its usage for the development of VR simulator for dentist training is demonstrated in figure 1 (A). For the handling of 3D models of teeth the framework Unity was selected as shown in figure 1 (B).
Before starting to work with the Marching Cubes algorithm, the original 3D model must be voxelized, i.e. convert to scalar density grid. In addition to the density grid, a material map is also needed. It allows, for example, to assign the degree of tooth damage. The material map can also store meta-information about mesh editing under the influence of the drill for further analysis of the simulation results.
For the parallel implementation of the algorithm on the Unity engine, the Job System framework was chosen. This framework allows parallel implementation of many subtasks for a special engine, and the “native” representation of the algorithm for input and output data that is suitable for the chosen architecture. To implement the algorithm, many subtasks were formulated. The ifrst subtask deals with counting mesh density and materials. Each task is responsible for only one vertex of the part of the grid. The second subtask is aimed at the building of a mesh surface on their basis. In order to display the diference between the anatomical structure of tooth and the degree of tooth damage, the subtask aimed at painting of surface polygons with diferent colors using a palette texture and a uv-map, the positions on which are selected using the material index from the corresponding cell of the material map for the current cube. Another subtask solving is aimed at the generation of a set of sub-meshes, which then form the final mesh. Each subtask is responsible for only one cube within its region of interest. The edited surface of the sub-mesh is loaded from the bufer to graphics memory only when all the cubes in the sub-mesh have been processed.
The main stages of tooth drilling simulation development are presented in figure 2. They include voxelization, Marching Cubes algorithm, collision detection, and detection of penetration depth of dental drill as discussed above.
The stages of the handling of a tooth 3D model for marching cubes approach are demonstrated in figure 3: from 3D voxelization (to get a set of voxels containing material properties) to handling with marching cubes (tooth surface mesh) and tooth drilling implementation process with the penetration depth control based on force calculation as in [25].
For the creation of the realistic immersive virtual environment, the framework Unity XR and single-pass anticipatory rendering engine Universal Render Pipeline were used.
Since the resources of an autonomous VR device are very limited, the 3D models on the scene need to be simplified. For the smoothest display, it is recommended to keep the number of vertices on the scene in a volume of less than 300 000. To do this, you can perform the procedure of “decimate” polygons using diferent algorithms presented in graphic editors. One of the algorithms used from the Blender editor merged polygons if the diference between their normal planes was less than a specified threshold.
Wearing VR headset Oculus Quest 2, the students are transported to a virtual dentist ofice and can interact with virtual patient’s mouth and dentist’s tools doing a dental procedure with simulation of tooth drilling. The developed VR scene is demonstrated in figure 4.
The scene has been designed so that the operator does not have to move around using controllers. All necessary movements can be performed in a real room. Since the headset is autonomous, it does not restrict the operator’s movement and turns, and the sensors have no blind spots. Nevertheless, if the dimensions of the real room do not allow this, the ability to move with the help of controllers is provided. There are interactive instruments on the scene, the main of which is a drill. Taking the drill in hand (bringing the controller to its location on the tools shelf and pressing the pick-up button), you can drill the tooth of the patient, which is located on the dental chair. The keys on the controller allow you to put the drill into working state and change modes. During operation, the controller vibrates at diferent intensities, creating the feeling of a real tools. After working with the tool is finished, the operator can return it to the tool shelf.
Two experts and three students from the Faculty of Dentistry at the V. I. Vernadsky Crimean Federal University were participated in the testing of pilot version of VR simulator in dental education. They evaluated the realism of the tooth 3D model drilling simulation and the realism of VR scene for dentist’s ofice simulation based on questionnaire demonstrated in table 2. The results of VR environment satisfaction are presented below. 2. Did you think that illumination of VR scene was some diference compared with the real dentist/’s ofice?
The common questions about VR simulator 1. Are this kind of VR simutor useful in dental - yes (the choise of two experts and three students) education? - no 2. What kind of educational tool could be - main the VR simulator? - additional (the choice of two experts and three students) 3. Could such VR simulator be useful as a dis- - yes (the choise of two experts and three students) tance learning tool in general and in special - no conditions (for example, during a pandemic)?
The evaluation of the realism of the tooth 3D model drilling simulation. Two experts noted that 3D model of tooth had acceptable quality, three students noted the usefulness of 3D model of tooth, possibilities to explore tooth from diferent views and capacities to get first useful experience of tooth drilling.
The evaluation of the realism of VR scene for dentist’s ofice simulation. Two experts and one student noted that VR scene was enough realistic. Two students noted that they were needed some time to adopt and immerse in virtual environment and noted some diferences of illumination compared with the real dentist’s ofice.
All participants noted that such virtual experience was useful as an additional tool to improve quality of education and could be considered as one kind of tool in special condition, but never couldn’t replace real practice and could become a powerful competitor for physical templates usage in dental education.
Technological advances have made it possible to incorporate Virtual Reality simulation technology in dental education, and above all in the preclinical restorative dentistry. Virtual Reality simulators provide the ability to integrate clinical scenarios into the operating environment and facilitate tactile skills using haptic technology [30].
The advantages of 3D digital models’ usage in medical education lay in their simplicity of deformation and restoration without worrying that student breaks important sample, for example, plaster jaw model. The augmented or virtual realities create a realistic environment and feel of the realistic first experience of patient treatment. In particular, Procházka et al. [31] study the alternatives for replacing the plaster models of teeth with computer 3D models in AR/VR environments. They consider the problem of the dental orthopantomogram analysis with the usage of edge detection methods and 3D models of dental arches based on image segmentation methods for studying the special region of interest in the arches or teeth.
Training with simulators usage has an important applied role in medical education. In particular, Reznik et al. [32] studied the possibilities of VR simulators usage for first aid training and Pulijala et al. [33] investigated the experience of VR Surgery usage developed for surgical training for Oculus Rift and Leap Motion devices. So, the VR simulators usage can improve fine motor skills and hand-eye coordination when dentists and surgeons teaching in pre-clinical conditions [34]. The virtual reality simulator allows students to immerse in multi-sensory and three-dimensional environment with realistic patient “mannequins” and built-in 3D models of dental instruments as shown in our research. Such simulator acquaints students with the structure of the tooth and gains important clinical competencies in dental surgery.
The main problem is the lack of tactile feedback. So, due to the fact that the operator does not have an emphasis on the real dental chair and the patient’s body, the natural shaking of the hands does not allow performing delicate manipulations with the dental drill. Nevertheless, this problem can be solved by resorting to the use of special equipment in the form of a model of a chair and a patient, whose position is synchronized with the position in the simulator and usage of manipulators with tactile feedback instead of ordinary controllers.
Haptic devices create an illusion of “tissues” feeling and interactions with them in the virtual environment and play an important role to form skills in pre-clinical practice. For example, Hung et al. [35] developed the virtual reality training simulator based on a head mount display and haptic device to practice tooth preparation and noted the perspectives of such decision for training students in a realistic environment with the illusion of realistic dental operations. Anderson et al. [36] studied a haptic-based virtual reality models and their perspectives for dental education, in particular, to enrich practical skills on diagnosis and treatment of periodontal diseases, drilling operations training, head and neck anatomical structure learning.
Besides, AR/VR technologies have proved successful to prevent anxiety for people with diferent phobia and also during dental treatment [ 37], especially for children. The first negative experience of children teeth treatment may form a stable sense of fear and anxiety before each visit to the dentist and even further in adulthood. The usage of eyeglasses reduces stress level during teeth treatment confirmed by some vital signs as pulses and blood pressure [ 38]. The gamification approach with AR/VR is also produces significant results to decrease anxiety when virtual heroes immerse children in the atmosphere of teeth treatment and explain the necessity of dental manipulations [39].
The Oculus Quest 2 Virtual Reality dental training simulator is capable of simulating the visual experience in the simulation process. Further improvement of this simulator will be aimed at recreating tactile sensations and feedback. The VR simulator can be used in the educational process by dental students at the stage of acquiring primary skills in tooth drilling with immersion in the virtual environment.
Testing of the VR simulator on the pilot study has shown promising results in the field of practical skills improvements on the training of the dental students in pre-clinical conditions.
Thanks to the experts and students from the Faculty of Dentistry at V. I. Vernadsky Crimean Federal University that were participated in a pilot study of VR simulator testing in dental education. educational tool in dentistry, European Journal of Dental Education 24 (2020) 259–265. doi:10.1111/eje.12492. [20] A. Liebermann, K. Erdelt, Virtual education: Dental morphologies in a virtual teaching environment, Journal of Dental Education 84 (2020) 1143–1150. doi:10.1002/jdd.12235. [21] K. Kim, J. Cho, J. Kim, J. Park, A dental simulator for training of prevalent interventions: Tooth restoration and ultrasonic scaling, in: P. Isokoski, J. Springare (Eds.), Haptics: Perception, Devices, Mobility, and Communication, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 195–198. [22] M. Figueiredo, Surface collision detection for virtual prototyping, in: Y. Xiao, T. Amon, P. Kommers (Eds.), IADIS International Conference Computer Graphics, Visualization, Computer Vision and Image Processing, 2009, pp. 65–72. URL: http://www.iadisportal.org/ digital-library/surface-collision-detection-for-virtual-prototyping. [23] G. Zachmann, Virtual Technologies for Business and Industrial Applications: Innovative and Synergistic Approaches, IGI Global, 2011, p. 32. doi:10.4018/978- 1- 61520- 631- 5. ch003. [24] I. Alaytsev, T. Danilova, A. Manturov, G. Mareev, O. Mareev, A priory collision detection algorithm for 3-DOF haptic rendering, International Research Journal 12 (2016) 47–50. doi:10.18454/IRJ.2016.54.180. [25] P. Rhienmora, P. Haddawy, M. Dailey, P. Khanal, S. Suebnukarn, Development of a dental skills training simulator using virtual reality and haptic device, NECTEC Technical Journal 8 (2008). URL: https://www.cs.ait.ac.th/~mdailey/papers/Rhienmora-ToothHaptics.pdf. [26] J. Wu, G. Yu, D. Wang, Y. Zhang, C. C. L. Wang, Voxel-Based Interactive Haptic Simulation of Dental Drilling, volume 2: 29th Computers and Information in Engineering Conference, Parts A and B of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 2009, pp. 39–48. doi:10.1115/DETC2009- 86661. [27] X. Zhao, Z. Zhu, Y. Cong, Y. Zhao, Y. Zhang, D. Wang, Haptic rendering of diverse tooltissue contact constraints during dental implantation procedures, Frontiers in Robotics and AI 7 (2020) 35. URL: https://www.frontiersin.org/article/10.3389/frobt.2020.00035. doi:10.3389/frobt.2020.00035. [28] M. M. Meese, Z. Trost, T. P. Chang, Immersive virtual reality anestheology training: The next frontier in simulation training, Journal of Clinical Anesthesia and Pain Management 4 (2020). doi:10.36959/377/337. [29] T. Kang, M. Chae, E. Seo, M. Kim, J. Kim, DeepHandsVR: Hand interface using deep learning in immersive virtual reality, Electronics 9 (2020). URL: https://www.mdpi.com/ 2079-9292/9/11/1863. doi:10.3390/electronics9111863. [30] R. Mladenovic, D. Dakovic, K. Mladenovic, Development of simulation methods in biomedical sciences - from phantoms to virtual patients, Serbian Journal of Experimental and Clinical Research (2020). doi:doi:10.2478/sjecr- 2020- 0051. [31] A. Procházka, T. Dostálová, M. Kašparová, O. Vyšata, H. Charvátová, S. Sanei, V. Mařík, Augmented reality implementations in stomatology, Applied Sciences 9 (2019). URL: https://www.mdpi.com/2076-3417/9/14/2929. doi:10.3390/app9142929. [32] E. Reznik, I. Krasnopolskij, M. Potemkina, O. Prirodova, Using virtual reality technologies to practice the algorithm of emergency medical care, Methodology and Technology of Continuing Professional Education 2 (2020) 6–14. [33] Y. Pulijala, M. Ma, M. Pears, D. Peebles, A. Ayoub, Efectiveness of immersive virtual reality in surgical training – a randomized control trial, Journal of Oral and Maxillofacial Surgery 76 (2018) 1065–1072. doi:10.1016/j.joms.2017.10.002. [34] E. Roy, M. Bark, R. George, The need for virtual reality simulators in dental education: A review, The Saudi Dental Journal 29 (2017) 41–47. doi:10.1016/j.sdentj.2017.02.001. [35] H. Hung, H. Kim, S. Moon, Virtual reality training simulator for tooth preparation techniques, Oral Biology Research 42 (2018) 235–240. doi:10.21851/obr.42.04.201812. 235. [36] P. Anderson, M. Ma, M. Poyade, A Haptic-Based Virtual Reality Head and Neck Model for Dental Education, Springer Berlin Heidelberg, Berlin, Heidelberg, 2014, pp. 29–50. doi:10.1007/978-3-642-54816-1_3. [37] R. Mladenovic, F. Djordjevic, Efectiveness of virtual reality as a distraction on anxiety and pain during impacted mandibular third molar surgery under local anesthesia, Journal of Stomatology, Oral and Maxillofacial Surgery (2021). doi:10.1016/j.jormas.2021.03. 009. [38] E. Birang, J. Yaghini, R. Birang, M. Zohary, Efect of virtual reality during periodontal treatment of patients with anxiety, Journal of Research in Dental and Maxillofacial Sciences 4 (2019). URL: http://jrdms.dentaliau.ac.ir/article-1-225-en.html. doi:10.29252/jrdms.4. 1.9. [39] K. E. Jones, F. Loizides, P. Eslambolchilar, I. Johnson, S. Bhatia, O. Crawford, M. Beirne, R. Chand, L. Vuilleumier, I. Araneta, Reducing anxiety for dental visits, in: D. Lamas, F. Loizides, L. Nacke, H. Petrie, M. Winckler, P. Zaphiris (Eds.), Human-Computer Interaction – INTERACT 2019, Springer International Publishing, Cham, 2019, pp. 659–663.