The paper deals with the description of a project of designing a training complex based on virtual reality technologies and that can be regarded as an e ective means to solve the problem of high cost content and renewal of training grounds for railway wagons maintenance workers due to constant-growing railway freight service operation needs. The use of VR training in teaching methods might be appropriate and e ective tool in support and assessment system of railway workers. The requirements for the training simulator and the challenges of its design are considered in this issue. Currently obtained results are under discussion - a prototype of the simulator based on the HTC Vive for testing the skills of a 12-position inspection of an open freight car with a bypass in accordance with the instructions illustrated by the example of a 3D model inspection of a four-axle freight car standing on the railway tracks with malfunctions (widening of the car wall, absence of a hatch, absence of a door, loss of a mechanism's part, a wheel crack, an empty axle box) and with the inspector's tools such as a hammer, a chalk and an absolute template. Peculiarities of implementing the prototype on the Unity platform are in the focus of this research. The feasibility test of the simulator's prototype proved its su cient ergonomic relevance to real maintenance process and revealed a number of implementation di culties to be considered in further research and development.
Due to Russian Railways data, at the beginning of 2019, the total number of
freight car eets on the Russian Railways network was 1,114,322 units, including
open cars - 526,628 units [
One of the problems of supporting the required professional knowledge and skills of railway wagon maintenance workers is the equipment of training grounds with samples of real wagons, which causes their high cost, as well as imposes restrictions on training non-standard situations and remote users.
To solve this problem in the framework of the integrated scienti c and
technical project "Digital railway", JSC Russian Railways plans to design training
complexes based on virtual and augmented reality technologies [
Abroad and in Russia, the railway industry has similar demonstration solutions for individual car maintenance tasks, as well as for various trains and infrastructure in general. Consider the following cases.
VRTraining { AirBraketestforRailroad [
VRTruckInspectionTeachingSystem (HTCVIVE) [
Railroad operationsinVR (HTCVIVE) [
EVE-Interactive 3D&VRlearningapplications [
Digitalizingtrainingfortrainoperator [
VRSimulators [
Referring to the literature survey results, the following conclusions can be drawn: { data are not available on the use of VR technologies for training and, most importantly, for supporting and evaluating the skills of servicing freight cars in full on the basis of regulatory documentation; { most implementations are related to solving individual tasks, applicable to a single user (collaboration is not supported), demo-based, and not used in the actual production process; { mostly VR applications are realized as the desktopVR to support interactive 3D product designs without special equipment (helmets, controllers, etc.), and those VR applications using special equipment model operations of \hand { tool { object" are not based on direct manipulation because of the complexity, but on creating an animated 3D video, which violates the ergonomic adequacy of VRmodeling real process of the wagon maintenance. 2
In the laboratory of computer graphics, virtual and augmented reality of the
Department of computing systems and networks and the engineering school of
SUAI [
The results of the work can also be used in specialized educational institutions, at the children's railway, in scienti c and technical museums and parks for career guidance and demonstration of the use of modern computer technologies in the eld of railway transport.
The simulator must meet the following requirements [
Broadly, the simulator consists of three main subsystems: { modeling environments, objects, and characters { creates a realistic representation of them (geometry and hierarchy, physical properties, and material properties) and simulates their behavior in time and space in real time based on physical models or arti cial intelligence models; { user interface - provides a display of environmental conditions; registration of human actions performed using motor skills, and impact on human modality in accordance with the state of the environment; { management - includes analysis of the actions performed by the subject; forming a reaction to the actions of the subject in di erent scenarios; determining changes in the state of the environment in time and space.
In view of the implementation of the simulator based on virtual reality
technology and its broad interpretation, we will clarify the terms used by the authors
[
{ Immersiveness - sensorimotor involvement of the user in the simulated
environment.
{ Interactivity - the user's ability to interact with environment objects and
modify them to get feedback on their actions (for 3D environments -
navigation, selection, manipulation and management [
The simulator is based on the HTC Vive virtual reality system (Pro and ProEye, including the wireless adapter) using Blender, 3ds Max, Unity, C#, and other tools. To design a simulator, you need to meet the following challenges: { Analysis of the inspector's activities according to the instructions of 12position freight car inspection with the aim of de ning requirements for the objects and processes modeling with constraints used VR equipment, including the formation of the list of positions of inspection, inspect elements of the carriage at each position, the standard fault elements, variants of de nition of serviceable/damaged by visual or instrumental inspection with tools and actions, types of chalk marking's information, working positions of inspector's body parts, methods of interaction with other inspectors and personnel, time of operations, etc.). { Development of 3D models of the car to the level of veri able elements (according to drawings or imported from CAD if possible) and types of faults, the location of the car (landscape, railway tracks, house for instruction), tools and inspector's bags, avatars of the inspectors, as well as recording sounds of serviceable and faulty equipment. { Development of typical actions with 3D models of tools and objects at each of the 12-position inspection using Vive HTC controllers. { Implementation of a 12-position inspection of a freight car in accordance with the instructions using the Vive HTC. { Development of interfaces of instructor (desktop, monitoring and recording the actions of the operator, setting of faults, navigation/map) and of inspector (HTC Vive, inspection of the wagon). { Development of a module for recording and evaluating students' actions, storing of results, and printing them out. { Implementation of a group (multi-user) mode of work of inspectors. { Development of a management system for individual and group classes in di erent scenarios of training, practice and certi cation. 4
Initially, a prototype of the simulator was developed to test the feasibility of the decisions taken, in condition of limitations of the speci ed VR equipment: 1. A library of 3D models of a landscape fragment with railway tracks, a freight car with the required detail for 12-position inspection, a hammer, a chalk (see Fig. 1), and an absolute inspector template(see Fig. 2, 3). 2. Library of fault models - widening of the car wall, absence of a hatch, absence of a door (see Fig. 4), loss of the mechanism's element. 3. Library of sounds corresponding to serviceable and faulty wheel pairs and axle-boxes of the railcar when they are tapped. 4. A library of typical actions with the 3D model elements of the car and the instructor's tool using HTC Vive controllers, including moving in the scene (around the car with a pass) or teleporting on the surface around the car, highlighting signi cant objects of the car with the illumination (faulty with a chalk, removing the illumination by repeated application in case of an error) (see Fig. 5, 6), manipulating the hammer (tapping the wheel pairs and axleboxes) (see Fig. 7) and the template (changing the location in the leading hand, taking/putting actions). 5. The instructor's interface, which displays the scope of the inspector from the 1st person viewpoint; a mini-map with the location of the inspector; the area of installation of faulty elements of the open car in positions before the start of the inspection or during the inspection of positions not yet passed; the scope of the scene of the instructor from the 1st person viewpoint (see Fig. 1- 6). 6. A control system that supports arbitrary by-pass of car inspection positions.
Control of the car by-pass is currently assigned to the instructor, who monitors the actions of the inspector. 1. The interaction of the Unity editor scene and the HTC Vive virtual reality complex is implemented using the SteamVR plug-in. In the VisualStudio development environment, C# programming language used for the scripts responsible for the logic interaction of a simulator - user with scene objects in Unity by means of selection and manipulation; the scripts responsible for the manipulations between objects (feedback, reaction the sound of the collision objects, the reaction of the render settings for the collision objects); the scripts responsible for the elements of two-dimensional interface of the instructor. 2. The terrain tool was used to create a landscape for the environment. The highlands are arranged so that the railway runs through the valley, and the edges of the map cannot be seen. Since the inspector's travel area is limited by the size of the SteamVR game room and the teleportation platform in the project, it is not possible to see the edges of the landscape at this location. After that, some of the hills were painted gray, and the rest - in green. The result is a kind of imitation of a valley or canyon. 3. The following simpli ed 3D models were implemented from the drawings: { railway, that was created and textured in the 3ds Max editor; { railway wagon, that was created in 3ds Max editor and then nished in
Blender editor.
Models [
Two \Empty" objects were used to specify the points, from which and towards which, the wall of the car extends, in order to create a widening e ect. { Absence of a hatch. The oor of the freight car has 18 hatches, the display of any of which can be disabled in Unity if necessary, which will mean its absence, and, consequently, failure. { Absence of a door. The car door display is disabled in Unity in the same way as the hatches. { Loss of a mechanism's element. When creating this breakage in Blender, the model of the hatch closing mechanism was modi ed, namely, one of the parts of the closing mechanism was selected and then removed. Then, in Unity, you can enable the display of the model with or without damage. 9. The following typical actions are implemented: { Moving the inspector around the map is performed by physically moving the inspector in the designated room. Moving the instructor is performed at the computer by using the mouse to view and the WASD keys to move. { Since the size of the room does not always allow the inspector to move on his feet and get where he needs to, the project uses the teleportation element. By clicking on the round button on any of the controllers, the inspector is indicated by a dotted pointer that draws a parabolic trajectory, the place where he wants to nd himself, and by releasing the button, he moves. This is a standard feature of the SteamVR plugin used for Unity. { Highlighting of faulty objects with a chalk is implemented using a Shader that applies a glow e ect when applied to a separate object. Initially, some parts of the car have a special tag \can be marked", which means that this part may be broken. When the chalk Collider touches the Collider of such an object, this Shader is superimposed on it, and the tag changes to"marked". If a part was selected accidentally by mistake, the inspector selects this part again, the Shader is removed, and the tag is returned to the original one. { The work of a hammer is similar to that of a chalk. If a collision occurs, the condition of the part is checked, and if no damage has been detected, the sound corresponding to the serviceable part is played. If the part was faulty, the sound corresponding to the damaged part is played. { The work with the template can be divided into 4 parts - it must be taken out of bag and returned to its place, the location in the leading hand, interaction with the non-leading hand and interaction with the elements of the car.
When the hammer and chalk tools were developed, the Interactable and Throwable scripts were used to con gure them. The Interactable script allowed you to perform actions such as selecting an object with your hand, and the Throwable script was responsible for manipulating the object, holding it in your hand, etc. However, when working on the template, it was decided to use a di erent approach. The ItemPackageSpawner script was used in conjunction with the ItemPackage script. The rst script allows you to create a certain \item distributor", from which you can take the \item" created using the second script only once before it is returned to its place. While the item is missing, you can leave a visual hint (in the form of a model or other) in its place, which will help you to understand where the object is expected to be returned. In addition, using this method allows you to interfere with the interaction of the inspector's hand models with tools. This is necessary for managing the template, since we hold the template in one hand and manipulate its elements with the other (moving the measurement lines).
In order for the template to t correctly in the inspector's hand, it was necessary to con gure the locations of the hand models relative to the template. To do this, the SteamVR Skeleton Poser script was applied to the template model. This script adds a window for editing the PoseEditor hand skeleton relative to the object to which the script is applied. Using this editor, has been con gured with two poses of the hands that held the template in di erent ways, depending on the purpose of the current usage of the template: position for inspection of the wheels a measuring template rulers, and the second to check the thickness of the wheel ange slot of the template. In addition, this script adds a BlendingEditor window for mixing hand poses, which allows you to mix or replace one hand pose with another, creating a \posture behavior". Next, a script was written that controls the \posture behavior", the task of which is to switch the original hand pose to the second one (and back).When using ItemPackageSpawner, we change the logic of detach the tool from the hand so that the item will be returned to its place only when we bring it back to the place where it was taken, i.e. we do not need to hold the trigger of the controller clamped. Instead, using our \posture behavior" control script, we use the trigger of the leading hand as a counter that is used in the script, which changes the location of the template in the inspector's hand. 10. Inspector's and instructor's interfaces.
The inspector-repairer is immersed in a virtual environment and its interface is designed as a direct manipulation of its tools and the interaction of these tools with the elements of the environment.
The instructor's interface consists of four main elds: a mini-map, a eld for displaying the view of the inspector-repairman, a eld of view of the instructor, and a list of switches for car breakdowns. In the Unity development environment, you created a Canvas element that contains elements of the instructor's interface. The instructor's interface is created by placing the rest of its elements on the Canvas element in the Unity development environment. The mini-map and instructor's visibility elds are implemented using the interaction of the camera component and video text elements. The fact is that there is an additional camera in the scene, which is located high above the car and monitors what is happening below it. Above the instructor and the inspector are marks that are implemented in the form of two di erent color bitmaps, in the form of a drop, the sharp end indicating the direction in which they are looking. Since the normal of these marks is located vertically up, and they themselves are located horizontally, only the camera located at the top can see these marks. The video texture captures the image that is currently being displayed on this camera, which allows you to use it as a mini-map. The eld of breakdowns switches is implemented and placed on the Canvas element using a script developed in the VisualStudio environment. 11. Joint inspectors' work.
An environment has been prepared for the Unity multiuser mode using Photon. The scene objects are divided into network and non-network. Objects that change their position in space, appearance, etc. must be synchronized between users, otherwise their changes will only be visible on the client's side. To do this, the following list of objects was compiled, in which one part of the objects was assigned to the network, and the other to the non-network: { Networked objects: parts of the freight car that were damaged; tools of the inspector-repairman; avatar of the inspector-repairman; mini-map. { Non-networked objects: a fragment of the landscape with railway tracks; parts of the freight car that do not provide for damage analysis; global lighting. 6
An experimental evaluation of the usability of [
As a test task, we used a 3D model of a four-axle freight car standing on the railway tracks with malfunctions (widening of the car wall, absence of a hatch, absence of a door, loss of a mechanism's part, a wheel crack, an empty axle box) and tools of the inspector: hammer and chalk.
Main features of hardware and software con guration - HTC Vive PRO Eye virtual reality system, Intel Core i9 { 9900K 3600 MHz, motherboard MSI Z390 MPG GAMING PRO CARBON, MSI GeForce RTX 2080 Ti LIGHTNING 11GB, memory CRUCIAL Ballistix Sport AT BLS2K16G4d32aest DDR4 { 32GB, SSD SAMSUNG 970 EVO Plus MZ 1TB.
Participants are a car inspector and an instructor who have previously been trained to work with a prototype based on HTCVive.
The following methods of analysis were used [
The following indicators of usability were taken as measured characteristics: { e ectiveness - the percentage of goals achieved and the accuracy of the operation; { e ciency { task-timing; { satisfaction{ rating scale of assessment and visual discomfort, complaints.
Based on the results of the experiment, the following conclusions can be drawn: { As a rule, the inspector only reached the goal after 2-3 familiarization passes (passing all 12-position inspection) and performed quite accurately both moving and teleporting in the scene according to the inspection positions, and operations for identifying and highlighting faulty objects visually and instrumentally by tapping with a hammer and marking with a chalk (highlighting). { The inspector's interface is mostly intuitive, but requires getting used to both orientation and movement in space, as well as interactivity using HTCVive controllers. { The instructor's interface is well supported for setting car faults, the inspector's location on the mini-map, the scope of the inspector from the 1st person view, but there are problems in the scope of the instructor from the 1st person view when monitoring the actions of the inspector because of its size and the choice of the necessary angle in a limited time. { The average passing time was about 4 minutes, which is currently used as a reference for comparing passes for this prototype con guration. { On a ten-point scale, the average ratings of inspectors are 6 and instructors are 7. Complaints were mostly reduced to the discomfort of working in a helmet due to poor xing and adjustment of lenses, control of controllers, etc., which were minimized as experience gained.
Outside of the main experiment, a test was performed using a 3D model of the absolute wagon template to identify wheel faults that require interaction between the two hands of the inspector with the template and the object of inspection, which revealed problems due to the design features of HTCVive controllers.
In the future, the following is planned:
1. Increasing the number of car malfunctions and tools for various elements of
the car inspection positions in order to Re ne the simulator prototype to the
minimum viable product and the ability to assess the ergonomic adequacy
of VR modeling to the real car maintenance process.
2. Use of sensors for some tools (hammer, etc.) in order to increase the realism
of simulated situations.
3. The implementation of various test scenarios skills check and assessment of
the actions of employees, store the results and print them.
4. Support for collaborative work of inspectors, including the development of
inspectors' avatars.
5. Integration with interactive electronic technical manuals that are used to
describe the composition and operation of the product, its intended use,
maintenance, troubleshooting, training and testing during the after-sales service
phase of the product [
A large eet of railcars on the Russian Railways network, the high cost of training ranges equipped with real equipment for training and evaluating car maintenance skills, and restrictions on training in non-standard situations lead to the search for new approaches to solving these problems, one of which is the use of virtual and augmented reality technologies.
However, today there is no data on the use of VR technologies for training and evaluating the skills of servicing freight cars in full on the basis of regulatory documentation.
In the article, the authors propose a virtual reality-based trainer to support and evaluate the skills of the inspector based on the instructions for maintenance of cars in operation, the requirements for it and the main tasks for its design are considered.
To test the feasibility of the simulator, a prototype based on HTC Vive and Unity was developed. An experimental test of the suitability of using a prototype simulator showed su cient ergonomic adequacy of working on the simulator to a real object, increased motivation and emotional involvement of employees, and the ability to model various situations in ways that are not available in the real world due to their high cost, danger, or impracticality.
The bene ts of designing this type of simulator will be granted as: { improving the training skills of the railway workers, reducing the cost and time of their training, and, as a result, increasing productivity and reducing downtime of cars, reducing injuries; { signi cantly reducing the cost of creating and maintaining training sites due to the partial elimination of real equipment and reducing the required space, the ability to quickly update the simulator with 3D models of new cars.