New challenges in the form of a pandemic set complex tasks for society, including the task of providing quality education during lockdown periods when it is physically impossible to conduct contact classes and social distancing is needed. Medical education is impossible without obtaining the most important practical skills, which should be preceded by the stage of practicing the skills of performing manipulations on simulators before treatment of the real patients.
The development of trainers and simulators based on augmented (AR) and virtual (VR) reality technologies are the most promising direction for using these innovative technologies in the field of education [1]. In addition, simulators of this kind allow doctors to improve their qualifications, mastering new clinical procedures without risking the health and life of the patient [2].
A comprehensive study of the advantages and disadvantages of using VR/AR technologies in dental education contributes to the development of innovative curricula using new simulators aimed at increasing the academic performance and developing psychomotor skills of future dentists in preclinical trials.
Recent advances in the development of augmented and virtual reality applications indicate an increase in the realism of tactile sensations and an improvement in display and motion detection systems in virtual space, which allows future doctors and dentists, in particular, to gain important interactive experience when performing various procedures [3]. Tactile feedback jointly with immersion in a virtual environment creates a sense of the reality of simulation when teaching dental manipulation. Let's consider the most popular virtual simulators in dental education.
The VIRTEASY dental simulator allows realizing the realistic interaction between the future dentists and their virtual patients by immersing themselves in a virtual environment, using a tactile feedback device jointly with a tracking device to simulate working with virtual dental tools. The VIRTEASY system also contains an editor that allows you to download patient' images, diagnose caries, examine the anatomical structure of the tooth, and also has a feedback system with the teacher built into the software shell of the simulator, which produces the monitoring of the manipulation execution by the students and export models with the subsequent possibility of printing them on a 3D printer. With the help of the VIRTEASY simulator, students acquire skills in caries treatment, dental restoration, implantology, and planning further treatment of a virtual patient in an immersive environment [4].
The VOXEL-MAN dental simulator makes it possible to produce accurate models of teeth based on microtomography and use a tactile device with the ability to perform three-dimensional manipulations, which enhances the effect of realism when teaching manual skills to dental students through interaction with tooth tissues in a virtual environment. A built-in system for tracking and automatically assessing the quality of dental manipulations and learning progress provides feedback and recommendations for improving skills in a preclinical model, implementing problem-oriented learning [5].
Dental simulator Simodont allows students to examine a tooth using a virtual model of a dental mirror, see the measure of the virtual dental tool angles to teeth and gums, study the morphology of teeth, treat caries, and place virtual crowns and bridges [6].
The goal of the research is to analyze the usage of virtual headset Oculus in medical education and, in particular, in dental education, to describe the specialties of development of VR-scene with interactive dental tools for Oculus Quest 2 virtual headset.
AR/VR simulators are effectively used in medical education and real practice. MladenoviΔ et al. studied the effectiveness of using a simulator to perform blockade of the lower alveolar nerve in augmented reality mode. It was experimentally shown that a group of students who used an augmented reality simulator more successfully struggled to local anesthesia and spent, on average, less time for this manipulation, while all students had an increase in heart rate during the procedure (data were recorded using a Sony bracelet SmartBand2) [7]. Gorrea et al. described a virtual reality simulator with tactile feedback from needle insertion when performing a procedure for blocking the lower alveolar nerve, which increases the feeling of realism during manipulation. During the simulator usage, the students felt tissue resistance depending on the point and depth of needle insertion [8].
Juan et al. describe the development of an augmented reality application for the anatomical structure of the teeth and gums with detailed morphological features viewing in a real environment and the possibility of interaction with a 3D model of the tooth. Experiments have shown that the augmented reality application aroused interest, promoted the development of curiosity and practical skills of students aimed at understanding dental morphology. In addition, the authors note the prospects of using this kind of application in distance learning [9].
The simulators for virtual headsets such as Oculus Quest 2 and similar headsets could be interesting for teachers due to the ability of rapid development of application and wide areas of usage. They have already shown promising results in combating various phobias (for example, stomatophobia), physical rehabilitation, and local anesthesia performance. As known, stomatophobia is a fairly widespread phenomenon associated with dental anxiety. Such kind of anxiety interferes with the timely treatment and prevention of dental diseases. One of the approaches for the level of anxiety decreasing is cognitivebehavioral therapy [10]. Liu S. et al. immersed anxious patients in a virtual environment using the Oculus Go virtual helmet and IFGworld-based worlds to promote relaxation [11].
Madshaven J.M. et al. develop virtual simulators for Oculus Rift connected with the implementation of a rehabilitation program that patients will be able to fulfill at home [12].
Zafar S. et al. described the usage of the Oculus Quest virtual helmet in pediatric dentistry for local anesthesia performance [13]. 89.9% of the surveyed students noted an improvement in their practical skills after training with the help of a VR simulator. Sukotjo C. et al. developed VR-scene for the osteoplasty procedure performing with the Oculus Quest usage [14].
We developed a VR scene of a dentist's office with interactive virtual dental tools for Oculus Quest 2 usage. The Oculus Quest 2 virtual headset was developed so that virtual worlds adapt to user movements. Advanced touch controllers feature improved ergonomics and intuitive controls. The headset has the following characteristics: resolution 1832x1920 per eye, refresh rate 60, 72, and 90 Hz, RAM 6 GB, and tracking with outdoor cameras. The usage of the Oculus Quest 2 virtual headset and the development process with Unity in our research project is presented in Figure 1 on the left and right respectively.
The methodology in detail and pilot study are described in the first part of our research [15]. The main steps of the research are vocalization, marching cubes algorithm usage, collision detection between dental drill 3D-model and tooth 3D-model, detection of penetration depth of the dental drill.
The main results of the research include VR simulation of the teeth drilling process and VR scene of dentist's office development with the ability to interact with the virtual dental tools and virtual X-ray gallery.
When developing a virtual reality simulator for teaching dentists, it is important to create the effect of realistic interaction with the tissues of 3D dental models. The development of the virtual scene of the dentist's office was preceded by the creation of a 3D model of the teeth and their anatomical structure in the Blender editor as shown in Figure 2. After that, we use the voxelization of the 3D model of the teeth in Unity for the rendering process simplification.
Voxelization is the transforming of any object to volume data stored in a 3D voxel array [16]. The easiest way to render voxels is to represent each voxel from the grid as a cube that will be positioned in space according to its coordinates in the grid. The first problem that arises with this approach is the high load due to the rendering of numerous cube faces that are not visible to the observer. One of the most famous computer graphics algorithms is the so-called Marching Cubes algorithm [17] and we used it too. The underlying data structure is a 3D mesh, but the material is not stored in every cell of the mesh but at every vertex of the mesh. Therefore, each mesh cell has exactly 8 adjacent material mesh vertices.
The models of different tooth layers: enamel, dentin, pulp, and source of caries (footage from the Blender editor on the left) and the result of marching cubes algorithm for a mesh of materials obtained by voxelizing tooth layer models (footage from a running simulation in the Unity on the right)
The marching cubes algorithm continues to run throughout the simulation. When changes are made to the density map, the corresponding chunk that owns the modified vertex of the map is marked as edited. Changes are most often made during input processing (controller movements) or physics processing (for example, during collider collisions), the processing of Job System tasks is further in the queue, and is located right before the scene rendering stage. Thus, within one frame, information about changes in the density map is collected, then the algorithm rebuilds the mesh of the changed chunks, and at the end, the changed mesh is drawn.
The next stage for the simulation of the teeth drilling process is collision detection of two 3D objects (3D model of tooth and 3D model of turbine handpiece). Tzafestas et al. considered the Octree method for the collision detection of two virtual objects. This method is based on the recursive division of the virtual environment into octants followed by division into octants of the detected virtual object. A collision is detected when the minimum distance between two objects is reached, taking into account their polygons [18].
In our research, we considered two approaches. The first, more optimal, involves firing rays from the base of the drill to a length equal to the length of the drill. The rays are fired along the entire circumference of the drill with a certain step. If the ray collides with the mesh collider of one of the chunks, a point is projected from the center of the drill to the same length that the original ray passed, then the coordinates of this point are transformed into the local coordinates of the mesh. In the neighborhood of these coordinates in the grid, the density indices decrease in direct proportion to the distance from the center of the neighborhood. The radius of the neighborhood and the intensity of the density decreasing are set according to the size and type of drill.
The second way utilizes the capabilities of the Unity physics engine. The drill consists of several capsule colliders, one of which is set for the drill. Non-drill colliders are used to indicate the correct center of mass of the drill. The drill itself acquires a Rigidbody component, and its position and rotation are controlled not by changing the parameters of the Transform component, but by using the Velocity tracking method, i.e. by giving physical impulses of acceleration and rotation in the direction of the target position. Since the drill is a collider attached to the Rigidbody, collisions of the drill with the mesh colliders of the tooth chunks can be tracked by subscribing to the corresponding event in the event dispatcher. The event is called with the collision details (including the points of contact), so further processing is done in the same way as in the first approach. This method is convenient because it provides a fairly realistic behavior of the drill as a physical object, and does not allow the drill to penetrate the tooth, ignoring the shell. To reach the center of the tooth, even under conditions of strong pressure of the drill on the tooth, it is necessary to consistently drill a tunnel to the center. It gives the realistic feeling of the resistance of the dental tool.
The strength of resistance to provide tactile feedback is often determined based on Hooke's Law or its modifications. So in research [8] the resistance force during the interaction of the needle with the anatomical material is determined according to Hooke's law F=kβx, where kthe coefficient of resistance, determined by the simulated tissue structure, βx is the displacement of the position of the 3D model of the needle. In [19], the force of tactile feedback is also described using Hooke's law in the form πΉ = πβππ β + ππ ΛπΆπ , where βd is the shortest distance from the tactile device π πΆπ to the bounding plane, π ΛπΆπ is the speed of movement of the tactile device. The proposed rendering of a simulation of tooth preparation with a turbine handpiece based on the view from two cameras: distant and close is shown in Figure 3 on the left and right, respectively.
Consider the development of a virtual scene for teaching future dentists using the Oculus Quest 2 virtual headset, which determined the choice of software and optimization procedures, aimed at achieving high performance. In particular, Unity was chosen as the engine with the ability to use the Universal Render Pipeline (URP) single-pass forward rendering. Forward rendering renders each object in one or more passes, depending on the number and nature of light sources acting on the object. To optimize the number of displayed vertices, a preprocessing stage was performed, including the procedure for reducing the number of polygons and generating UV maps to implement the correct "baking" of lighting.
Interaction with VR devices is implemented based on the experimental framework XR Interaction Toolkit, which allows using a special object XR-rig to track the position of the head of the user wearing a helmet and transform it into space coordinates inside the engine.
The developed virtual scene of the dentist's office allows the user to interact with dental tools, in particular, with the dental turbine handpiece and the X-ray gallery. The hand models have individual finger bending and straightening animations that match the touch sensors on the controller to realize the actual position of the fingers. When the user clamps the trigger of an object holding, the virtual dental tool moves into his hand and is ready for use. The user-friendly windowed interface integrated into the virtual scene allows the user to interact with the main menu using a translucent pointer that is projected from the hand.
To open the main menu, use the corresponding key on the right controller, as shown in Figure 4 on the left. Pressing on the icon with the help of the interaction trigger on the device controller launches a virtual window, for example, a gallery window with X-ray images, as shown in Figure 4 The main menu appears in the center of the user's area of view and is directed at right angles towards the user. Double pressing the key on the controller closes the menu. After opening the menu, you can call a specific window by clicking on the corresponding icon. Interaction with icons occurs using a translucent pointer. The pointer is visible only if the ray hits the interface elements. The point at which the pointer' ray touches the interface is the position of the virtual cursor in the window system of the simulator. Pressing the trigger of interaction (with the help of the index finger) on the controller, the interface interprets as the cursor clicking. So, the user-friendly interface with windows and clicks was realized in the virtual environment.
The use of VR simulators in dental education is aimed at gaining important practical skills in a preclinical model. Future dentists will be able to discover new possibilities for interacting with 3D models of teeth, tissues, and dental tools with the help of the Oculus Quest 2 virtual headset through immersion in a virtual learning environment. We proposed the interaction with virtual X-ray gallery organized with the helped of a user-friendly windowed interface and working with dental tools in VR scene of dentist's office. Further research will be aimed at the approbation of the developed VR-simulator for training future dentists at the Department of Propedeutics and Dentistry of the V.I. Vernadsky Crimean Federal University.