The research provides a review of applying the virtual reality (VR) and augmented reality (AR) technology to education. There are analysed VR and AR tools applied to the course “Development of VR and AR software” for STEM teachers and specified eficiency of mutual application of the environment Unity to visual design, the programming environment (e.g. Visual Studio) and the VR and AR platforms (e.g. Vuforia). JavaScript language and the A-Frame, AR.js, Three.js, ARToolKit and 8th Wall libraries are selected as programming tools. The designed course includes the following modules: development of VR tools (VR and Game Engines; physical interactions and camera; 3D interface and positioning; 3D user interaction; VR navigation and introduction) and development of AR tools (set up AR tools in Unity 3D; development of a project for a photograph; development of training materials with Vuforia; development for promising devices). The course lasts 16 weeks and contains the task content and patterns of performance. It is ascertained that the course enhances development of competences of designing and using innovative learning tools. There are provided the survey of the course participants concerning their expectations and the course results. Reduced amounts of independent work, increased classroom hours, detailed methodological recommendations and increased number of practical problems associated with STEM subjects are mentioned as the course potentials to be implemented.
The technology of AR is well-known for most of people. It was under the close attention of Gartner analysts for many years. On July 2020, the Gartner Hype Cycle for Emerging LGOBE (I. S. Mintii)
https://kdpu.edu.ua/semerikov (S. O. Semerikov); https://www.scopus.com/authid/detail.uri?authorId=57205438593 (M. M. Mintii); https://kdpu.edu.ua/personal/ismintii.html (I. S. Mintii) © 2021 Copyright for this paper by its authors.
CEUR
Workshop
Proceedings
htp:/ceur-ws.org
IS N1613-073
Technologies reflected the increasing attention for such technologies as Augmented Design and
Augmented Development (figure 1) [
There are high expectations for AR and VR in education. AR possesses a significant
educational potential, first of all, due its ability to visualize objects. In Ukraine, the number of
researches into AR/VR application to education has greatly increased in the last few years. Let us
mention only publications of 2019-2020 dealing with simulation of ultrasonic wave propagation
in the heterogeneous medium using coarse grids (Morkun et al. [
There is also a review of the content of massive open online courses associated with AR
and its application to education in order to create a special course for the professional
development system for the research and teaching personnel of postgraduate educational institutions
(Panchenko and Muzyka [
To study a natural cycle, a free mobile application LiCo.STEM is developed (Midak et al. [
There are also investigations into AR/VR application to general technical disciplines including
development of a mobile application for accomplishing tasks on a projection drawing (Kanivets
et al. [
As for teaching foreign languages at university, attention is drawn to the use of AR elements
in order to support students with diferent learning styles (audio, visual, kinesthetic) (Tarasenko
et al. [
There are researches into vocational training of future transport specialists (Lavrentieva et al.
[
AR tools are used in studying vehicles, both civil and military: a new teaching tool containing
a spherical (360° or 3D) photographic panorama and a VR device are presented (Barkatov et al.
[
In natural and mathematical disciplines, the AR application is developed by means of Android
Studio, SDK, ARCore, QR Generator, Math pattern. A number of markers of mathematical
objects are developed relevant to the school mathematics course (the topic Polyhedra and
Functions, their properties and graphs) (Bilous et al. [
The possibilities of using AR technology are analyzed and the software model of the solar
system is created by Hordiienko et al. [
To study the subjects of the astronomic cycle at primary school, a mobile application (on the
Android platform) is designed to visualize the solar system by using the AR technology as well
as the alphabet study with astronomic definitions (Midak et al. [
In education and therapy of people with special needs, AR application is aimed at dealing
with cognitive disorders and providing them with communication skills through associations
(Dyulicheva et al. [
AR tools are also used to provide children with autism spectrum disorders with psychological
and pedagogical support (Osadchyi et al. [
Specific attention is paid to the usage of AR books in education (Panchenko et al. [
These and other issues were discussed at the 1st [
Based on the current experience of using VR and AR technologies in education and the prospects for their release into the masses in next 5-10 years, it is necessary to think about the problem of preparing for the use of these innovative technologies in the professional activities of future STEM teachers. After all, in just a few years, today’s students will have to manage this process: both as software engineers and as teachers. Therefore, the main purpose of our research is to develop a training course on designing VR and AR systems for future STEM teachers, adapted to Ukrainian users and to the current level of technology development.
The object of the research is the professional training of future STEM teachers for the design of VR and AR technologies.
The subject of the research is the learning resources for the design of VR and AR systems for future STEM teachers.
The purpose of the research is to develop the learning resources for the design of VR and AR systems adapted for diferent types of learners.
To achieve the purpose of the research such tasks were solved: 1) an analysis of the experience of using AR tools for the development of educational materials was done; 2) the software for the design of AR tools for educational purposes were identified and the technological requirements for the course “Development of VR and AR software tools” were characterized; 3) individual components of the training complex for the design of VR and AR systems for future teachers majoring in STEM disciplines were developed.
Hereafter, we treat AR as an ability of a device (a mobile device or a web-browser) to track an image or display a 3D object over this image. The main idea of AR involves displaying a computer model in real-time and real-space in order to establish interaction between a user in real-space and a 3D model in the virtual one.
AR can be both marker and marker-free. In case of the marker AR, the device tracks a 2D-marker: when it is detected, it actually displays a 3D object. In the marker-free version, a device searches for a flat surface (a table, a floor, etc.) and places a 3D object there.
Using a device camera, AR enables displaying a computer-generated objects in game, marketing and other programmes, for example to arrange furniture in a living room or to try clothes on before buying them. It is really a great opportunity for business, as AR displays products before a consumer actually buys them.
There are specific devices developed for AR including AR helmets and sets that enable users’ immersion in the simulated environment.
AR provides the real-life world with 3D models controlled by mobile devices any place. Virtual
Reality (VR) plunges a user into a simulated world by using head mounted devices (HMD). Both
AR and VR provide interactivity in a similar way. For example, VR actually uses controllers
and in some cases, hand tracking that enable a user to interact with 3D objects inside a scene
they are located in. Main threats of HMD application to working with VR include eye strain,
dizziness and headaches. Unlike VR, AR is not noted for such serious health risks. Nevertheless,
users’ long concentration on their actions while using AR is causing some disturbances as it is
not quite safe [
While solving the first problem, it was found out that at the present stage of development of information technologies, the leading means of implementing AR are mobile Internet devices – multimedia mobile devices that provide wireless access to information and communication Internet services for the collection, systematization, storage, processing, transmission, presenting all kinds of messages and data.
While solving the first problem, it was found out that at the present stage of development of information technologies, the leading means of implementing AR are mobile Internet devices – multimedia mobile devices that provide wireless access to information and communication Internet services for the collection, systematization, storage, processing, transmission, presenting all kinds of messages and data.
The use of AR technology in a mobile-oriented learning environment of higher education institutions: • expands the capabilities of laboratory facilities used to prepare students for work with real systems; • makes complex and expansive systems available; • contributes to the improvement of vocational training by providing laboratory simulators with AR; • motivates students for experimental and educational research work.
The creating of interactive training materials with the use of AR systems can be done in two main directions: 1) the use of utilities or linking markers with user-developed models; 2) the development of VR and AR software for educational purposes.
In the first direction, the developer does not require good programming skills, however, the functionality of the created tools is significantly limited by the relation to proprietary software. In the second direction, the developer needs develop tools himself. However, the functionality and adaptability of the developed tools are significantly increased.
To solve the second purpose of the study, we reviewed the tools for developing VR and AR
and chose the most suitable for achieving the goal of the study. Among the reviewed, we note
both the “old” Wikitude SDK [
The current version of ARKit allows you to develop multiplayer games with AR ARCore
[
ARCore comes with three main possibilities of combining VR and real worlds: 1) tracking the position of the phone in the environment; 2) “recognizing the environment” provides the ability of the phone to determine the size and location of horizontal surfaces; 3) lighting assessment allows the phone to evaluate the actual lighting conditions.
ARtoolKit [
Maxst [
Vuforia [
In our opinion, in the process of preparing future STEM teachers for the use of AR systems
for developing interactive teaching materials it is advisable to use an integrated approach.
The design with use of standard objects can be performed in a visual design environment.
Providing standard objects with new properties and creating new ones can be performed in
an object-oriented programming environment. At the present stage of ICT development, it is
advisable to use the Unity environment [
AR programming has become innovative (trendy, interesting, useful, etc.) in the recent 60 years, usage of JavaScript as a programming language being an exclusively recent trend. However, language choice also determines choice of development tools, the most eficient of which nowadays being the following: a) A-Frame and AR.js are application programming interfaces (API) and unique tools of fast prototyping. The HTML-like code, which uses JavaScript on the server, constitutes the major part of their application software. A-Frame is applied to creating scenes, objects, animation and other 3D elements in the web-browser. AR.js enables tracking a marker and displaying a scene designed by A-Frame on the marker; b) Three.js and ARToolKit are a so-called backbone applied by many JavaScript libraries.
Three.js uses WebGLRenderer to create high-quality 3D scenes directly in the browser. Unlike A-Frame, Three.js is mostly used to create web-programmes under Google Cardboard control and requires JavaScript application.
AR software tools developed with JavaScript and WebGL can be placed on the Internet at one of cloud services like Heroku.
Everything concerning JavaScript/ECMAScript syntax can be found in numerous sources, for instance, video-lectures by Douglas Crockford (https://youtu.be/playlist?list= PLEzQf147-uEpvTa1bHDNlxUL2klHUMHJu).
A-Frame uses the ECS pattern for designing computer games, Entity-Component-System being its basic concepts. The Entity is a container for components. Entities are the basis of all the objects in a scene, yet without components, entities are unable to do or provide anything. The Component is a small object that implements a certain data structure and is responsible for a separate part of software logic. Each component type can be attached to the entity to provide the latter with some property. Systems control sets of entities combined by some components, which are not obligatory. In A-Frame, this design template is realized by attributes.
Any primitives of A-Frame (a-scene, a-box, a-sphere, etc.) are used as entities. a-entity is of special importance being self-explanatory. All other primitives are actually covers for components and developed for convenience sake as any element can be created by means of a-entity. A-Frame is constantly being supplemented with new components designed by users and packed in npm, which is available at https://www.npmjs.com/search?q=aframe-component. To apply a component from the list, you should go to its repository, download and install a corresponding file. Another option is to use the unpkg service to download any npm file following the link unpkg.com/:package@:version/:file.
8th Wall is one of the commercial suppliers of AR services that can be used under some functional conditionally free mode. 8th Wall Web is a JavaScript library that implements the technology of simultaneous localization and mapping (SLAM). It is widely used in unmanned vehicles, flying devices, autonomous underwater apparatuses, rovers, household robots, etc. 8th Wall Web easily integrates with such libraries as A-Frame, Three.js, Babylon.js (https: //www.babylonjs.com/) and Amazon Sumerian (https://aws.amazon.com/sumerian/). In the same way as awe.js, 8th Wall Web requires the HTTPS browser to access a web-camera. Mobile browsers require support of: • WebGL (canvas.getContext('webgl') || canvas.getContext('webgl2')); • getUserMedia (navigator.mediaDevices.getUserMedia); • deviceorientation (window.DeviceOrientationEvent); • Web-Assembly/WASM (window.WebAssembly).
The elective course “Development of virtual and AR software” has resulted from solution of the third research problem. The substantive basis this course is the open course from the University of San Diego on the EdX platform [39] and the book by Jesse Glover [40].
The course consists of two substantive modules.
Topic 1.1. VR and Game Engines Topic 1.2. Physical interactions and camera Topic 1.3. 3D interface and positioning Topic 1.4. 3D user interaction
Topic 1.5. VR navigation and introduction Content module 2. Development of AR tools
The tasks vary in complexity, its level increasing step-by-step. For instance, the first task includes creating the scene described theoretically and disposing its file at students’ own site section (e.g., https://playground2.ccjournals.eu) as well as sending hyperlinks as answers to check and creating a scene with Ukrainian texture and signs and students’ own names (figure 2).
Next, students are to create an AR scene described theoretically (figure 3) and a VR model supplementing it with all the Solar system planets (figure 4).
The 5th week task includes attaching three arbitrary A-Frame objects to markers Hiro, Kanji and one’s own marker, placing required files at one’s own site section at https://playground2. ccjournals.eu and sending a hyperlink for the scene and the designed marker (figure 5) as an answer.
The task of the 6th week is to modify the scene described theoretically to build a pentagon on the markers with letters A, B, C, D, and F (figure 6).
The task of the 7th week involves creation of the empty scene described theoretically by using Three.js (figure 7). After that, students should perform Laboratory Work 1 – create a scene in VR and AR.
During the 8th week, students are to create a dynamic scene described theoretically (figure 8).
Next, students create a dynamic scene using users’ shaders as the 9th week task (figure 9) and then the dynamic scene is disposed in the files by using at least three camera helpers (figure 10) as the 10th week task.
During the 11th week, students develop a 360-degree scene (figure 11) and put the model in it (figure 12) during the 12th week.
The 13th week task includes: a) realizing a pattern from theoretical materials using the marker with the number of the two latest last numbers of the hyperlink for the personal section of https://playground2.ccjournals.eu site; b) downloading the model of the previous task into the scene from awe.js (figure 13).
The 14th week task implies creation of a scene from the laboratory work 1 using the materials of weeks 7-13 and one’s own data of the laboratory work as well as Three.js and awe.js (figure 14).
During 15-16 weeks of study, students fulfill the tasks of laboratory work 2 including transferring the scene from laboratory work 1 to 8th Wall using the 8-frame library.
During 2020 80 students were involved in the experiment: POKT-18m and PO-16 groups of Faculty of Information Technology of the Kryvyi Rih National University and MIM-14 and FIM-14 groups of Faculty of Physics and Mathematics of the Kryvyi Rih State Pedagogical University.
So, among the participants in the experiment, a survey was conducted on the formation of competence in the design and use of innovative learning tools.
Survey results are shown in figure 15.
A survey was conducted after the course to obtain feedback on the impressions of the participants. The research data were collected using interview techniques in qualitative data collection method. The survey was attended by 23 participants. It should be noted that at the time of the interview all of them were no longer students, but STEM teachers, which allows us to conclude that the responses received are independent. Gender distributions of the interviewees were three men and twenty women.
Interview questions: 1. Have you had any experience with AR before studying the course? 2. What was the most interesting thing to know?
the editing, structuring, and interpretation of collected data.
Initially, the following answers to this question were supposed: • Yes, I have developed applications myself. • Yes, I used training applications. • Yes, I used in everyday life (advertising, entertainment, etc.).
• No, I haven’t.
As a result, only 4 of those respondents used AR earlier, and only in everyday life, the remaining 19 before the course had no idea about AR (figure 16).
The meaning of 100% of respondents’ answers was either to the process of development or the result of application development or the practical application of these applications.
All received answers were the application development; process of reviving pictures; convert 2D images to 3D; 3D modeling; practical application; visualization.
The most impressive answer was: “Results exceed all expectations”.
To this question 2 respondents gave a negative answer, 1 was dificult to answer, 15 answered in the afirmative, and 5 said that they have already improved their knowledge. Figure 17 shows the distribution of responses as a percentage.
All the received answers are: • I’m already using it. • I’m going to use. • I think I will use it.
• No, I don’t (figure 18). Respondents identified areas for using AR, such as a master class on the use of augmented reality for school teachers, when learning to program with high school students, when learning mathematics in specialized classes.
One respondent immediately after the course during preparation for the state exam. One of the questions on the exam was “Demonstration of a fragment of a non-standard lesson”. The theme of the lesson was “Creating logos. Brandguide. Brand book”.
The answers we’ve received: • Reduce independent work. • Increase classroom activities (lectures, labs, consultations). • Extend the course for 2 semesters. • Detail the methodological guidelines. • Increase the number of practical tasks connected with STEM courses. • The idea of conducting a survey using AR markers was interesting.
To get a complete picture of students’ impressions of the course, it is necessary to reproduce exactly the answers of some students.
Thus, the course “Development of VR and AR software” promotes the development of competence in the design and using innovative learning tools. The research is not completed, the implementation of the developed course and experimental verification of its efectiveness has been continuing. [39] Virtual Reality (VR) App Development (UC San Diego, 2019), 2019. URL: https://www.edx.
org/professional-certificate/ucsandiegox-virtual-reality-app-development. [40] J. Glover, Unity 2018 AR Projects: Build four immersive and fun AR applications using ARKit, ARCore, and Vuforia, Packt Publishing, Birmingham, 2018.