AR-technologies can be widely used in the educational process using ICT. Especially valuable is the use of AR-based applications for the visualization of illustrations of models of objects and processes in school books. The use of electronic educational resources using AR-technologies can improve the quality of students' knowledge, as well as provide teachers with a wide range of new opportunities. Using AR technology allows you to design and create effective learning resources. This develops students' creative thinking and increases their motivation to learn. In this paper, we consider a model of a learning system using augmented reality technologies for visualizing illustrations in school textbooks. A survey of secondary school teachers showed the possibility, interest and effectiveness of using e-learning tools based on AR technology. Pupils can use this electronic resource both in class at school and at home during independent study. Students of Kherson State University, interviewed in the STEM education system, showed a willingness to work with augmented reality technologies. To assess the prospects of using AR-based applications in the educational process, an expert method was used. Experts evaluated the prospects of using AR technology to visualize 2D and 3D models of educational objects. For evaluation, a five-point Likert system was chosen. The developed model can be used as a means to create the basis for future research, development and dissemination in the system of educational institutions. The proposed model of the learning system was tested in the classrooms of students in the learning process at STEM.
New information technologies are widely used in the development and use of electronic educational resources (ESM). Virtual and augmented reality (VR, AR) technologies are actively used to improve teaching methods using information technology.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
user sees in their real surroundings with digital content generated by computer software.
The additional software-generated images with the virtual scene typically enhance how
the real surroundings look in some way. AR systems layer virtual information over a
camera live feed into a headset or smart-glasses or through a mobile device giving the
user the ability to view three-dimensional images [
expanding the boundaries of reality, which in turn helps to increase students' interest in the subject of the study area, better learning new information. The difference between
The relevance of using AR in high and secondary school is significant when studying a wide range of disciplines for visualization of models of systems and processes, detailing the presentation of properties of complex objects, visualization of abstract objects, conducting virtual excursions and travels, etc.
The subject of the study is the software and hardware of the training system using interactive augmented reality technologies.
The purpose of this work is to design a system and develop software for visualizing processes and models of objects in a school textbook using interactive augmented reality technologies.
We list a number of works that summarize the experience in the development and use of training tools using AR in educational institutions of Ukraine and abroad.
The authors of the Institute of Information Technologies and Learning Tools of
NAES of Ukraine” in their article describe the main problems of modern education in
higher educational institutions of Ukraine and their solution using AR [
A detailed description of tools for developing applications using AR can be found in
the article [
Tony Liao identifies key themes and areas of focus for AR technology use: AR
users/nonusers, AR devices, AR content, and AR industry. By organizing these lines of
research, manuscript serves as a call for specific future areas of research, suggests new
approaches that researchers could take to explore interrelationships between these
areas, and advocates for the necessity of research that examines different levels
(micro/meso/macro) of analysis within AR [
The article [
In the article [
Article [
The work [
The work [
Article [11] describes a typical AR system with software and hardware architecture based on modern technological advances. The use of AR in various areas, especially in the field of the military system, equipment support, and simulation training, has been investigated.
A smartphone can be used as a mobile device for the tasks of visualizing objects and processes in a school textbook. Use Case Diagram (Diagram 1) shows the options for user actions with a smartphone [12]. Aspects of AR-based applications programming technology were kindly provided by Nicolò Carpignoli [13].
Diagram 1 contains the following main elements.
1. Scan QR Code can be a marker of the first interaction of the user and the system. It is assumed that each textbook has its own QR Code. This user case is called when the user wants to scan the QR code on the front of the AR popup book. 2. Scan AR Markers: This interaction is called when the user wants to scan the various AR markers in the AR popup book after successfully scanning the QR code of the AR popup book. 3. View Overlaid Information: This interaction is called when the user wants to view information overlaid after successfully scanning the AR markers.
information. It is a simple call that terminates after the information is displayed.
When developing software using VR and AR-based applications, the platforms on which computer games are created are usually used (Unity [14], Unreal Engine [15], etc.). At the same time, such tools as Steam VR [16], Google VR [17], Oculus [18], Windows Mixed Reality [19], Google ARCore [20], Apple ARkit [21], Google Tango [22], Vuforia [23] and others can be used.
Consider several examples of the use of these programs and technologies: Authors of article [24] presents the experience of developing 2D and 3D virtual laboratories based on the cross-platform game engine Unity for studying physical phenomena and processes.
Expeditions [25] is an application developed by Google. Using Google Cardboard, you can go on an archaeological expedition and see the excavations, take an underwater trip along the ocean floor or swim with sharks. This application is already used in their lessons by many teachers around the world.
The mobile application "Mars is a Real Place" was developed using Oculus technology [26]. The musical slide show consists of stereoscopic 3D photographs of Mars. 80 fascinating high-resolution 3D images are presented, carefully thought out for easy viewing. Images are a combination of views from the orbit of Mars and panoramas taken on the surface.
Sky Map [27] is an Android planetarium software application. Sky Map was designed and developed by a group of Google engineers in Pittsburgh, Pennsylvania. This application used to identify stars, planets, nebulae and more. 2
Description of the learning system model using AR
Visualization in a Schoolbook First of all, let's consider the definitions of the basic concepts. By a learning system using AR, we mean a hardware and software environment in which the states and processes of functioning of complex systems are visualized, and aimed at improving the efficiency of student learning. Interactive technology enables the information and communication training system to variably respond to user actions in an active mode. The main goal of such a learning model is the active involvement of students in the educational process.
The term “trigger image” will be understood as any image on the pages of an electronic textbook that has the properties of a trigger for a mobile application and allows displaying elements of augmented reality on the device.
A virtual object is a 3D object that is displayed and used in a mobile application for demonstration in AR.
Consider a learning system using interactive augmented reality technologies, consisting of a regular paper school textbook containing textual information of a specific subject area, which is illustrated by drawings or photographs.
A model of this system is shown in Fig. 1.
School textbook is the main learning resource in the learning system using AR. Images in the school textbook serve as a trigger for visualizing models of existing objects that the user sees on the smartphone screen as a result of the application. An electronic book may also be used, which is the publication of a book in digital form, consisting of text, images, or both, which can be read on a flat display of computers or other electronic devices.
Smartphone is a mobile device with the necessary OS parameters (version of Android or IOS, camera availability, Internet access) and with the installed AR application.
Database is used for external storage of training resources and is an important element of the training system. All mobile devices have memory limitations, on the basis of this, a system should be developed, part of the resources of which will be stored in the database.
Before creating a learning system using AR, a scenario plan is required, which should include the following components: name of e-learning resources; class e-learning resources: multimedia electronic educational resource; presence of interactivity and multimedia: This system contains multimedia and interactive elements - tests, videos, simulators; description of the user interaction with the content: User interaction is carried out through data exchange with the system server (the User can download the necessary resources and also enter data during the test); indication of the software required to work with the e-learning resource; 3D objects to create models; convenient user interface for using the application.
When creating an e-learning resource, the following tools are used: Multimedia technologies and platforms, that used to create AR applications (3ds
UML modeling tools (www.draw.io), when designing the program. 3
Software modeling and design The application for learning with AR usually consists of the following components: “Help” (instructions for this application), “Exit” (exit from the application), “View object” (the camera screen for trigger scanning appears). “levels” (opens the screen to select the level of the scenario), “level” (opens the selected level).
The diagram of use cases for interactive training application with augmented reality is shown in Fig. 2.
The sequence of user actions when working with the application for learning augmented reality and virtual reality is shown in Fig. 3.
An example of designing classes in the development of virtual and augmented reality applications is shown in Fig.4.
To use the AR environments, the student needs to install two corresponding applications on his smartphone.
When application for AR launched, the student gets to the main menu of the program: by clicking on the “help” button you can find out the instructions for this application, by clicking on the “exit” button it will close the application, by clicking on the “levels” button, a screen appears to select the level of the game that corresponds to the lab number, by clicking on the “level 1” button, the user is inside the virtual space, organized according to the requirements put forward to conduct this virtual laboratory work.
After the camera screen appears, you must aim the camera lens on the page with the task. The program recognizes the image trigger. Each trigger is unique. He links the 3D image and page of the book with the corresponding task. As a result, a moving 3D object will appear on the screen against the background of the corresponding page of the book.
After the user opens a certain level, he gets the opportunity to study in detail all the 3D objects of the scenario in virtual reality and their interaction according to the learning objectives. Since the inclusion of a certain level of the program is in standby mode, the user can perform control and navigation actions. At the moment of performing a certain action with objects, the user in the background can see the corresponding contextual help and training information.
For a better understanding of the technical characteristics inherent in the objects used in this model, consider the description of meta-objects, objects in the real world and objects used in laboratory work.
Consider a model for constructing a learning system using AR (Fig. 4). Unity 3D allows to develop applications for Android.
Vuforia is an augmented reality mobile platform supported on Unity 3D. o AR camera initializes Image with the image that the camera transmits. o Image is a graphic object from the database, which is a trigger for the AR application. 3D models – multimedia 3D objects in the public storage for users (Asset Store, Google search) or developed own models (3D Max). o Static elements – elements that the user cannot move.
o Dynamic elements – elements that the user can move. Lean s a class library of the Unity 3D platform for managing objects. o Rotate - rotate an object. o Scale - resize an object.
o Move - move an object. Database – Vuforia platform database, which stores images for later use in the application. Book – school textbook with images that serve as triggers in an AR application. PNG, JPG images – images and other multimedia resourses, that are added to the database.
To use the software for its intended purpose, you must have a textbook with triggers and a smartphone with an AR application. The application will display a scene with a 3D model depending on the trigger found, each trigger has its own scene. The trigger is the image in the textbook. The camera initializes the image and displays a model that is attached to this image. Since Vuforia is supported only by mobile devices running Android 4.0.3 and iOS 7 and higher, this is a prerequisite that must be observed (Fig. 5).
The interaction of objects occurs after the user presses certain control buttons that trigger a trigger set for interaction between objects. 4
An example of using a learning system using AR We will consider in more detail the process of modeling and designing AR educational objects using the example of visualizing models of physical processes in a school physics course.
As an example, consider a school textbook in physics for grade 8 edited by V.G. Baryakhtar, S.O. Dovgy [28], topic “Thermal Conductivity”, § 2. Temperature dependence of the size of physical bodies, page 15. An example of illustration of the property of thermal conductivity of bodies is considered in the textbook (Fig. 6).
In Unity 3D, Vuforia has developed a 3D model for trigger # 1. It consists of such elements: a book, a glass, half a carrot, a metal dart, two whole carrots (these elements are static), a candle (this is a dynamic element).
The user can move the candle along the diagonals x, y, z. If the candle is under the dart, after some time, due to the temperature change, the dart will change its angle of inclination, if after that the candle is removed, then the dart tilt will return to its original position due to the temperature change to the original one. This is confirmed by the laws of physics. Students can take part in this experiment themselves (Fig. 7).
Consider the following example, demonstrating the thermal conductivity of metals, paragraph 5, figure 5.1 from the school textbook [28] (Fig. 8). This figure acts as a trigger in the AR application.
Unity 3D with Vuforia has developed a 3D model for the trigger Fig. 8. It consists of the following elements: a tripod, a metal dart, nails (these elements are static), a candle (this is a dynamic element).
A candle is a moving object, when the fire comes in contact with a metal dart and a certain time passes (2 seconds), the nails begin to fall in turn, thus confirming the experiment (Fig. 9).
The “Repeat” button updates the application to its original characteristics. All objects fall into their original places. 5
Prospects for using AR in the educational process
process. 16 experienced teachers of secondary schools of Kherson were interviewed.
of educational objects. For evaluation, a five-point Likert’s system was chosen. Table
Classroom use Use in independent work The interest of teachers in the use of learning resource I plan to use such a training resource The results of expert’s assessment showed on Diagram 2. 4,9 3,7 3,4 9 , 3 8 , 3 4 7 , 2
G R A D E 1 , 4 3 , 4 6 , 3 8 , 4 9 , 4 2 , 3 7 , 3 4 , 3
An expert assessment of the quality and prospects of using training resources can be considered sufficiently reliable only if the expert evaluations are in good agreement. Therefore, statistical processing of the results of expert evaluations should include an analysis of the consensus of experts. The Concordance method is used to assess the degree of consensus of experts on options for assessment [29].
The results of the expert survey are presented in Table 2. -56 -37 14 9 11 16 15 16 15 47 (1) 1 1 1 1 1 1 1 -92 -20 -42
-39 -34 -21 W
12S m2 (n3 n)
. 10 7 7 6 5 2 5 15 14 16 13 6 6 11 44 13 15 14 9 11 10 9 47 16 16 15 12 10 8 10 62 11 13 12 15 9 13 16 43 12 6 10 14 16 12 14 17
Concordance coefficient W is calculated according to the formula proposed by Kendall [29]
Here S in1 2i in1 mj1 xij 12 m(n 1)2 , m – number of experts, n – the number of objects of examination, xij – assessment of the i-object by j-expert. Coefficient of concordance may vary between 0 and 1. If W = 1, all experts gave the same evaluations for all objects, if W = 0, the evaluations of experts are not coordinated.
Using the formula (1) calculated coefficient W = 0, 65 is significantly different from zero, so we can assume that among experts there is objective concordance. Given that the value of m(n – 1)W is distributed according to χ2 with (n – 1) is the degree of freedom, then W2 12S = 117,1. Comparing this value with the tabulated m n (n 1) value T2 for n – 1 = 15 degree of freedom and significance level α = 0,01, we find W2 = 117,1 > T2 = 30,578. Therefore, the hypothesis of consistency of expert evaluations confirmed according to Pearson.
Thus, the results of a pedagogical experiment can be taken as a basis for the design and creation of learning resources using AR-based applications. 6
Conclusion and future work
We described the training methodology and proposed a model of the training system using augmented reality technologies.
Currently, such training systems are in demand both in higher and secondary schools. Teachers and students can use this electronic resource both in the classroom at school and university, and at home. A survey of secondary school teachers and students of KSU showed the readiness of this category of users to work with augmented reality technologies.
The proposed model of the training system has been tested in the process of STEM education.
The developed model can be used as a means to create a basis for future research, development and dissemination in the system of educational institutions. We plan to introduce a system using virtual and augmented reality technologies in the school educational process of the STEM school of Kherson State University. 7 11. Jun He, Peng Han, Huan Liu, Shiying Men, Lu Ju, Pu Zhen, Ting Wang. The research and application of the augmented reality technology. IEEE Xplore (2018). online https://ieeexplore.ieee.org/document/8284781 12. Oluwaranti, A.I., Obasa, A.A., Olaoye, A.O. and Ayeni, S. Architectural Model for an
1977 (2015), online http://www.jmest.org/wp-content/uploads/JMESTN42350957.pdf 13. Nicolò Carpignoli. Web XR Expert. Maintainer of AR.js, online
https://medium.com/@nicolcarpignoli 14. Development environment Unity, online https://unity3d.com 15. Development environment Unreal Engine, online https://www.unrealengine.com/en
US/what-is-unreal-engine-4 16. Development environment SteamVR, online
https://developer.valvesoftware.com/wiki/SteamVR 17. Development environment Google VR, online https://vr.google.com/ 18. Development environment Oculus, online https://developer.oculus.com/ 19. Development environment Windows Mixed Reality, online
https://developer.microsoft.com/en-us/windows/mixed-reality 20. Development environment ARCore, online https://developers.google.com/ar/ 21. Development environment ARKit, online https://developer.apple.com/arkit/ 22. Development environment Tango, online https://developers.google.com/tango/ 23. Development environment Vuforia, online https://developer.vuforia.com/ 24. Gonzalez, J.D., Escobar, J.H., Sánchez, H., De la Hoz, J., Beltrán, J.R. 2D and 3D virtual interactive laboratories of physics on Unity platform. Journal of Physics: Conference Series 935(1) (2017), online https://www.researchgate.net/publication/322106253_2D_and_3D_virtual_interactive_lab oratories_of_physics_on_Unity_platform 25. Expeditions, online
https://play.google.com/store/apps/details?id=com.google.vr.expeditions&hl=en 26. Mars is a real place. Virtual Engineering by DrashVR LLC., online http://www.titansofspacevr.com/marsisarealplace.html 27. Sky Map, online
https://play.google.com/store/apps/details?id=com.google.android.stardroid&hl=en 28. Physics: 8th grade textbook ed. Baryakhtar V.G., Dovgy S.O. Kharkov: Publishing House "Ranok", 240 p. (2016) 29. Kendall M. Rank Correlation Methods, Charles Griffen & Company, London (1948)