=Paper=
{{Paper
|id=Vol-3099/paper11
|storemode=property
|title=VirtuaLecto Storytelling Content Creation Platform Using Augmented Reality and Virtual Reality
|pdfUrl=https://ceur-ws.org/Vol-3099/paper11.pdf
|volume=Vol-3099
|authors=Julio Vera-Sancho ,Christian Condori Mamani,Gustavo Suero-Soto,Juan Callacondo-Velarde,Wilber Anibal-Valdez
}}
==VirtuaLecto Storytelling Content Creation Platform Using Augmented Reality and Virtual Reality==
https://ceur-ws.org/Vol-3099/paper11.pdf
VirtuaLecto Storytelling Content Creation Platform
Using Augmented Reality and Virtual Reality
Julio Vera-Sancho1[0000-0001-5526-5223] , Gustavo Suero-Soto1[0000-0002-6479-4810] Christian
Condori-Mamani1[0000-0002-1810-978X], Juan Callacondo-Velarde1[0000-0002-4576-5889] and
Wilber Valdez-Aguilar1[0000-0002-3527-6766]
1 Universidad Nacional de San Agustín de Arequipa
{jveras,gsueros,ccondorimama,jcallacondo,wvaldez}@unsa.edu.pe
Abstract. In recent years, technologies such as augmented reality and virtual re-
ality have not only bridged the digital and real world, but have also become in-
volved in education as part of immersive experiences, supporting the student’s
learning process. This research work shows the advances in the development of
a digital platform called VIRTUALECTO, in which teachers can create content
with augmented reality and virtual reality to reinforce learning activities that al-
low students in an interactive, educational way and immersive to improve reading
skills, through the use of mobile devices. The results, so far, indicate that the
proposal guarantees a virtual learning environment (VLE), within the ISO 25000
[1] quality standards, in addition to having some improvements compared to
other similar proposals for storytelling.
Keywords: Augmented Reality · Virtual Reality · Virtual Learning Environ-
ment · Micro-services · Web Server · Application Server.
1 Introduction
Currently, technologies have become a very important tool in teaching and learning
processes, as part of their advantages of modern education, and students compared to
traditional education are no longer mere recipients of information [2]. At an early age,
children interact with new technologies in their learning, as devices are increasingly
used in their education and make this more funny with the help of videos, slides, online
platforms, virtual and augmented reality [3].
Augmented reality (AR) and virtual reality (VR) are one of the emerging techno-
logical tools that have been gaining popularity in recent years. Its trend is increasing in
relation to its application in the educational field, since they allow users to have an
immersive experience, visual three-dimensional simulations combined with audio, su-
perimposing virtual elements on reality, allowing them a greater understanding of an
specific topic [4] [5]. There are some tools for Storytelling, the best known case is Sto-
ryBird. In addition to the fact that the vast majority of this type of tools are focused
only on narrative texts, using only audio-visual resources, but now they are being sup-
ported with technologies such as augmented reality [6].
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
1
2
In the present research work is shown the progress of creating a platform for story-
telling, and interactive Virtual Learning Environments for content creation with aug-
mented reality and virtual reality.
2 Related works
Recent studies aim to used virtual reality and mobile applications to improve the expe-
rience in education. ARVEE seeks combining traditional and immersive education in
an interactive, fun and educational way, encouraging students to learn astronomy and
motivation to pursue a career similar to the one they are learning. The project was tested
in a group of students between 7 and 14 years of age, where a great acceptance was
observed for this technology. In addition, 90.9 % of them consider that virtual reality
can be of great help in the learning process and make some school subjects more inter-
active and easy to understand
[2].
Previous studies investigated about the use of augmented reality Game-Based
Learning, that enriching students, experience during reading comprehension activities,
its purpose is the design a game using Augmented Reality in order to promote under-
standing in primary school students, the tests included students from third to sixth grade,
between the of 8 and 12 years old. With the help of teachers, a game called AR Ole
Cierraojos was developed for tablets and smartphones with Android, the game is in-
spired by pop-up books and works using AR markers represented on each page of the
book. Hendrys concluded that the results of reading comprehension with the game show
no difference with the results of traditional approaches, however, the students show a
greater motivation and interest in the activity [7].
at present there are investigations that Comparing reading comprehension between
children reading augmented reality and print storybooks, measured the reading compre-
hension of children who read a storybook in augmented reality and They compared it
to others reading the traditional print version of the same story. To animate the book
with augmented reality, the HP Reveal Studio platform was used to create 2D anima-
tions with the help of images and audio materials, and show them on the pages of the
book. Delneshin, concluded that the application with AR content has a positive influ-
ence on readers, and observed or a significant difference compared of the traditional
method [8].
Other studies investigated about Virtual Reality, and the develop of an integration
in the learning environment for education, which was developed a simulation platform
to teach the water cycle to children, using COSPACES VR and its MERGE Cube com-
plement. With the CoSpaces creation tools, it is easy to build 3D scenarios, students
interacted with 3D objects and models. Ahmed concluded that using VR enables the
user to feel more personally connected to their environment and helps them learn more
effectively [9].
2
3
3 Proposal
3.1 Theorical framework
Augmented reality (AR) It is a way of Augmenting the real world with virtual ob-
jects, it consists of combining real and virtual objects in a real environment and in real
time. AR technology has been applied to a wide range of fields: tourism, entertainment,
marketing, surgery, logistics, manufacturing, maintenance, and others [10].
Virtual reality (VR) refers to an entire simulated reality, which is built with computer
systems using digital formats. Building and visualizing this alternate reality requires
powerful enough hardware and software to create a realistic immersive experience (for
example, virtual reality headsets or dedicated glasses and 3D software) [11].
Storytelling Digital storytelling has been widely adopted in teaching and learning in
educational sectors around the world. Digital storytelling is regarded as the art of telling
stories with digital elements including text, graphics, sound and video. A typical digital
story is a few minutes long narration with video and music to present ideas on a certain
theme. [12] .
React Native React Native is a native scripting framework for cross-platform mobile
development. The framework was originally developed by Facebook and released in
2015 for iOS only, but an active community has since added Android support and done
many other contributions to the open source project. [13].
3.2 Objective of the platform
The objective of the platform is the creation of educational storytelling content, with
augmented reality and virtual reality technologies, to improve reading comprehension,
as a motivating effect, our target audience is teachers, who will be able to create content
with AR, VR and activities for student evaluation from a web platform, the other target
audience is the students, who will be able to consume these resources, through a cross-
platform mobile application.
3.3 Platform architecture
Due to the special characteristics of the proposal, there are two client applications, both
for content generation and for consuming mobile learning objects. An architecture is
proposed that finds a balance both in design and the development of the different com-
ponents that compose it. This work propose to divide into two parts, server and client
side.
Server Side Considering the demand for data storage and transfer in the flow of the
platform, it is necessary optimal requirements of the server side. That is why the trends
in architectures based on microservices are taken as a base, which, unlike monolithic
3
4
architectures, allows a better use of the services provided by the cloud [14]. This model
considers two main modules, a database module with PostgreSQL as the main database
and DigitalOcean Spaces as the file storage system, and a Services module in DJango
on Docker containers as see Fig. 1.
Fig. 1. Communication. Between the web server and the application server
– Database PostgreSQL is a relational and object-oriented database manager. It is
considered among the most powerful relational database management systems,
provides simple accessibility, is cross-platform, and is available for use on all major
operating systems without compromising its performance [15]. However, it should
be considered that it is easy to violate without the appropriate protection settings,
another important point to consider is its small amount of data types, as see Fig. 2
– File system DigitalOcean spaces is a file storage, service that allows you to store
large amounts of files. The data transfer is protected by HTTPS, and the scalability
of the storage is simple, so its consumption is optimal and safe.
Fig. 2. Communication. Between the web server and the application server
Web services application : Django is a very powerful and popular web frame-
work, which helps to effectively launch web applications implemented on Python. Fea-
tures like its relational object mapper, Python API, and admin interface make it securely
efficient in web development [16].
Usually, applications running on a platform, either on production or development,
use a server to respond to a request of the client, the Django framework uses its own
execution server to deliver content whenever a request is made [17].
4
5
Although this works good, it is not advisable to use this development server to run
a production environment, so for this proposal Green Unicorn was thought for the com-
munication between the application server and the web server, as shown in 2.
Client Side Client-side applications are those that can be operated by both web brows-
ers and applications on mobile devices, generally are not well compatible with current
technology. These applications not only need to share data remotely, but also cache and
update data locally to continue responding to user requests through their interaction
[18]. Existing cloud databases only provide remote data that is shared and the need
arises to implement caching and buffering to application level [19]. In addition, on the
client side, information processing is quite fast, allowing very complex tasks to be car-
ried out, allowing applications to be very flexible and personalized.
In Fig. 3, the client-side architecture proposal for the mobile application is shown,
which has several modules that help the generation of 3D digital scenarios, to create
both Augmented Reality and Virtual Reality.
– Object Learning Reader Module : This module is responsible for reading the
learning objects, which come as a query from each user in JSON format, which
contains a set of metadata, which provides us with information such as: Title, De-
scription, Creation Date, Object Type learning (AR or VR), Positions of the 3D
models, Path of the 3D models, Keywords. All this information will become object
data, which can be interpreted and processed by the client application.
– 3D Scenes Creation Module : This module is in generates the different scenarios,
based on the learning objects that were read in the previous module. This module
is responsible for assembling resources that can compose a 3D scene, and prepare
them both to become available in cache and buffer, to allow subsequent rendering,
and it is very important the role of the graphics processor of each mobile , to opti-
mize its correct operation and display.
– AR and VR Rendering Module : The most important part for the visualization of
the 3D models is the present module, since from the creation of the 3D scenarios
and already pre-loaded in cache memory, it is that we work with different technol-
ogies that have the same objective, create real or virtual situations, such as ViroRe-
act open source library, which allows to create augmented reality such as AR Core
for android and AR kit for iO’s, and for virtual reality creation Google Virtual Re-
ality SDK [20].
Fig. 4 shows the client-side architecture of the scenario creation platform and learn-
ing objects.
5
6
Fig. 3. Client side architecture proposal for mobile
– Creation Object Learning Creation Module This module allows us to create
learning objects, which is made up of resources for immersive activities such as
augmented reality and virtual reality, in addition to creating a set of questionnaires
to verify the student’s learning process. This module is supported by JavaScript
technologies such as Three JS and WebGL, which allows us to build 3D scenarios.
– 3D Models Library This library contains 3D models that allow different 3D
scenes to be assembled with them, these models are categorized into: People, Na-
ture, Buildings, Animals, Objects, which allows to help in the scene assembly pro-
cess.
– Audio visuals resources library This library contains resources such as audio,
videos, 360 videos, as part of the modeling of Storytelling scenes.
3.4 Features
Mobile application To measure reading comprehension, the following functionalities
were proposed for the application:
– Login: The application has a validation to save the information and the user’s pro-
gress Fig. 5
– Library: It is a view where all the stories available to the user are shown, each
story has a list of chapters and each chapter has a questionnaire about that chapter.
– Questionnaire: It is made up of questions of different kinds. There are four types
of question, simple selection, multiple selection, true or false question and drag-
gable questions. as seen in Fig. 5
– Reward: If the questionnaire is successfully completed, a virtual reality view will
be shown Fig. 6 or augmented reality Fig. 5 with 3D objects in relation to the se-
lected story.
6
7
Fig. 4. Client side architecture proposal for mobile
– Workgroups: The user will be able to enroll in a work group and thus access new
stories and see objects in augmented or virtual reality
Fig. 5. Validation and simple selection question
7
8
Fig. 6. Virtual Reality.
Web application One of the main functionalities of the platform’s client, we have the
creation and display of content, and user interaction follows the same pattern as the
mobile application. In the Fig. 7 show the Scene Editor
– User login and registration They can register and login in the web interface.
– Contents library In this module users can browse and view descriptions of learn-
ing objects.
– Content creation module Allows users to create their own storytelling material,
as well as providing the ease of using objects, stages and sounds available in the
platform’s database, as well as their own that have the possibility of being im-
ported.
– Rendering module allows to visualize and evaluate the quality of the content
created, this feedback allows to have a better perspective of the material that is
being built.
4 Evaluation of the platform
The ISO 25000 standard is called Software Quality Requirement Evaluation
(SQUARE), it provides a guide for the use of the new series of international standards,
called Requirements and Quality Assessment of Software Products (SQuaRE); which
constitute a series of standards based on ISO 9126 and ISO 14598, and its main objec-
tive is to guide the development of software products with the specification and evalu-
ation of quality requirements, allowing to define the quality model and the process to
follow to evaluate said product, establishing criteria for its specification, its measure-
ment and its evaluation [21]. The usability of VLEs can be evaluated in two different
contexts: usability during the development process and usability evaluated by end users,
that is, when the VLE is found as a finished product. This article will focus on deter-
mining the characteristics or aspects necessary for the evaluation of VLE as a final or
8
9
Fig. 7. Scenes Editor.
finished product, for which some characteristics are defined that an VLE must have
according to product-oriented standards [1].
We use ISO 25000 as measurement tool, for which it will be applied in the Virtual
Learning Environment called VIRTUALECTO, for this we will give a star-based eval-
uation where the minimum is 1 and the maximum 5, these evaluations were carried out
by experts in software development and specialists in educational technologies.
Table 1. VIRTUALECTO platform evaluation under ISO 25000
Cod Feature Point
1 Ease of Learning *****
2 Ease of Understanding or Understandability ****
3 Ease of use and operability *****
4 Help facility ***
5 Technical accessibility *
6 Degree of attraction or attractiveness *****
7 Adherence to standards or conventions or complicance ****
As we can see, the proposed VIRTUALECTO, largely complies with ISO 25000
standards, it should be noted that this standard is based from the point of software en-
gineering.
In the analysis and research carried out on the most similar platforms, which create
both augmented reality and virtual reality content, and which are focused on education
with the area of communication, the platforms, CoSpaces, Augmented, and Storybird
9
10
were found, below The comparative table of the characteristics of the platform proposal
is shown, with respect to the other content platforms. In the Table 2 show the compar-
ative.
Table 2. Comparison of VIRTUALECTO with other platforms
VIRTUALECTO CoSpaces Augmented Storybird
Augmented Reality Yes No Yes No
Virtual Reality Yes Yes No No
Education Yes Yes No Yes
Reading Yes No No Yes
Recommendation systems Yes No No No
Questionnaires Yes No No Yes
Cross-Platform Yes Yes No Yes
Web Editor Yes Yes No Yes
5 Conclusions
This research works on the storytelling content creation platform called VIRTUALECTO,
developed with a set of tools that allow creating a scene editor with 3D models with
technologies such as Threejs, React, WebGL. In addition, the proposal also shows us a
robust architecture for the rapid creation of this AR and VR content with the use of
cross-platform technologies such as ViroMedia. The proposal uses in its development
standards for software such as ISO 25000, focused on educational platforms, from a
product perspective, as well as where the strength lies in the ease of learning, operabil-
ity, attractiveness and ease of understanding. In addition, it shows the set of architec-
tures and technologies to give you the support and high demand for information ex-
change when using big data such as 3D objects, and audio-visual resources.
6 Acknowledgments
Special acknowledgment to the Universidad Nacional de San Agustín de Arequipa,
UNSA that through the Grant contract IAI-005-2018-UNSA of the project “Animación
a la lectura con M-Learning, creando situaciones reales y virtuales”, is that it was pos-
sible to carry out the investigation of the proposal proposed in this article.
10
11
References
1. 25000, I.: Iso 25000 calidad de software y datos. urlhttps://iso25000.com/index.php/nor-
mas-iso-25000 (2019)
2. Dascalu Maria Iuliana, Dragoi George, S.I.C.U.T.M.B.R.: Arvee - an innovative applica-
tion based on virtual reality for immersive education. In: The 15th International Scientific
Conference eLearning and Software for Education Bucharest. (2019) 19–26.
https://doi.org/10.12753/2066-026X-19-139
3. Akc¸ayır, M., Ak¸cayır, G.: Advantages and challenges associated with augmented reality
for education: A systematic review of the literature. Educational Research Review 20
(2017) 1 – 11. https://doi.org/https://doi.org/10.1016/j.edurev.2016.11.002
http://www.sciencedirect.com/science/article/pii/S1747938X16300616
4. Tobar-Muñoz, H., Baldiris, S., Fabregat, R.: Augmented reality game-based learning: En-
riching students’ experience during reading comprehension activities. Journal of Educa-
tional Computing Research 55(7) (2017) 901–936.
https://doi.org/10.1177/0735633116689789
5. De la Gala Quispe, K., Vera, J.: Uso de la realidad aumentada mejora los niveles de com-
prensión lectora en estudiantes de quinto grado del nivel primario. CEUR (2018)
6. Yilmaz, R.M., Goktas, Y.: Using augmented reality technology in storytelling activities:
examining elementary students’ narrative skill and creativity. Virtual Reality 21(2) (2017)
75–89
7. Hendrys Tobar-Muñoz, Silvia Baldiris, R.F.: Augmented reality game-based learning: En-
riching students’ experience during reading comprehension activities. In: Journal of Edu-
cational Computing Research, Los Angeles, USA, SAGE journals (2017) 1–36.
https://doi.org/10.1177/0735633116689789
8. Delneshin Danaei, Hamid R. Jamali, Y.M.H.R.: Comparing reading comprehension be-
tween children reading augmented reality and print storybooks. In: Computers Education.
vol 153, Computers Education (2020). https://doi.org/10.1016/j.compedu.2020.103900
9. Ahmed Al-Gindy, Chema Felix, A.A.A.M., Alkhidir, M.: Virtual reality: Development of
an integrated learning environment for education. In: International Journal of Information
and Education Technology. ijiet.2020 (2020) 171–175.
https://doi.org/10.18178/ijiet.2020.10.3.1358
10. Riccardo Palmarini, John Ahmet Erkoyuncu, R.R., Torabmostaedi, H.: A systematic re-
view of augmented reality applications in maintenance. 49 (2018) 215–228.
https://doi.org/10.1016/j.rcim.2017.06.002
11. Jorge Martín-Gutiérrez, Carlos Efrén Mora, B.A.D.A.G.M.: Virtual technologies trends in
education. 13 (2017) 469–486. https://doi.org/DOI 10.12973/eurasia.2017.00626a
https://www.ejmste.com/download/virtual-technologies-trendsin-education-4674.pdf
12. Banny S. K. Chan, D.C., Chiu, T.K.F.: Digital literacy learning in higher education through
digital storytelling approach. 13 (2017) 1 https://files.eric.ed.gov/fulltext/EJ1144564.pdf
13. Kuitunen, M.: Cross-platform mobile application development with react native. (2018) 23
https://trepo.tuni.fi/bitstream/handle/123456789/27139/Kuitunen.pdf?sequence =4isAl-
lowed=y
11
12
14. Vayghan, L.A., Saied, M.A., Toeroe, M., Khendek, F.: Kubernetes as an availability man-
ager for microservice applications. arXiv preprint arXiv:1901.04946 (2019)
15. Makris, A., Tserpes, K., Spiliopoulos, G., Anagnostopoulos, D.: Performance evaluation
of mongodb and postgresql for spatio-temporal data. In: EDBT/ICDT Workshops. (2019)
16. Mel´e, A.: Django 3 By Example: Build powerful and reliable Python web applications
from scratch. Packt Publishing Ltd (2020)
17. Marani, F., Marani, F., Karkal: Practical Django 2 and Channels 2. Springer (2019)
18. Zawirski, M., Preguic¸a, N., Duarte, S., Bieniusa, A., Balegas, V., Shapiro, M.: Write fast,
read in the past: Causal consistency for client-side applications. In: Proceedings of the 16th
Annual Middleware Conference. Middleware ’15, New York, NY, USA, ACM (2015) 75–
87. https://doi.org/10.1145/2814576.2814733
19. Attiya, H., Burckhardt, S., Gotsman, A., Morrison, A., Yang, H., Zawirski, M.: Specifica-
tion and complexity of collaborative text editing. In: Proceedings of the 2016 ACM Sym-
posium on Principles of Distributed Computing. PODC ’16, New York, NY, USA, ACM
(2016) 259–268. http://doi.acm.org/10.1145/2933057.2933090
20. Boutsi, A.M., Ioannidis, C., Soile, S.: Hybrid mobile augmented reality: Weblike concepts
applied to high resolution 3d overlays. International Archives of the Photogrammetry, Re-
mote Sensing & Spatial Information Sciences (2019)
21. Cocunubo-Suárez, J.I., Parra-Valencia, J.A., Otálora-Luna, J.E.: Propuesta para la evalua-
ción de entornos virtuales de enseñanza aprendizaje con base en estándares de usabilidad.
TecnoLógicas 21(41) (2018) 135–147
12