=Paper=
{{Paper
|id=Vol-2733/paper24
|storemode=property
|title=An Augmented Reality Information System Designed to Promote STEM Education
|pdfUrl=https://ceur-ws.org/Vol-2733/paper24.pdf
|volume=Vol-2733
|authors=Maria Cristina Costa,António Manso,Paulo Santos,João Patrício,Francisco Monteiro Vital,Gonçalo Manuel Martins Rocha,Bernardo Marques Alegria
|dblpUrl=https://dblp.org/rec/conf/siie/CostaMSPVRA20
}}
==An Augmented Reality Information System Designed to Promote STEM Education==
https://ceur-ws.org/Vol-2733/paper24.pdf
An Augmented Reality Information System
Designed to Promote STEM Education
Maria Cristina Costa Paulo Santos
Unidade Departamental de Matemática e António Manso Techn&Art - Instituto Politécnico de Tomar,
Física Techn&Art - Instituto Politécnico de Tomar , Portugal
Instituto Politécnico de Tomar Portugal psantos@ipt.pt
CICS.NOVA - Universidade NOVA de manso@ipt.pt
Lisboa Goncalo Manuel Martins Rocha
ccosta@ipt.pt Licenciatura em Engenharia Informática-
João Patrício
Ci2 - Smart Cities Research Center, Instituto Instituto Politécnico de Tomar
Politécnico de Tomar aluno20614@ipt.pt
Francisco Monteiro Vital jpatricio@ipt.pt
Licenciatura em Engenharia Informática- Bernardo Marques Alegria
Instituto Politécnico de Tomar Licenciatura em Engenharia Informática-
aluno20472@ipt.pt Instituto Politécnico de Tomar
aluno20620@ipt.pt
Abstract—The PlanetarySystemGO is an information Mobile Augmented Reality (MAR) applications in formal
system that is being designed to promote STEM education in the learning environments such as schools [6,7].
context of planetary systems of the Universe. Besides providing
information about the celestial bodies of the Universe, it also This paper presents an information system that is being
provides information about its dimensions being possible to designed to promote STEM education in the context of
explore different scales. The information system includes a planetary systems of the Universe, such as our Solar System.
mobile augmented reality location-based game and a web The PlanetarySystemGO information system architecture
application with a back-office. The game that can be played with includes a MAR location-based application and a web
a smartphone consists in a sort of planet hunt in an outdoor application with a back-office. The MAR application can be
space. The players need to walk in the real world to find virtual used on mobile devices such as smartphones or tablets that
objects, which are celestial bodies such as stars or planets that have camera, Global Position System (GPS), gyroscope and
appear on the screen of the mobile device. During the several accelerometer. It consists of a serious game that is a sort of
stages of the game, information about the celestial bodies is planet hunt in an outdoor space. The players need to walk in
provided and a set of multiple-choice questions needs to be the real world to find virtual objects, which are orbits of
answered. The player gains points as he succeeds by either planets and celestial bodies such as stars, planets or satellites
finding the planets’ orbits, the celestial bodies or answering the that appear on the screen of the mobile phone. The back-office
questions correctly. The back-office is responsible for
provides the necessary definitions required for the mobile
administering learning objects and data management through a
device is responsible for managing learning objects and data
web application running in a web browser. Through the back-
office, teachers can introduce contents about the celestial bodies
management through a web application running in a web
that they intend their students to learn and also the set of browser. Through the back-office, teachers may introduce the
multiple-choice questions that they find appropriate in order to contents about the celestial bodies they intend their students
evaluate their students’ knowledge about these contents. to learn about and also the set of multiple-choice questions
Therefore, the PlanetarySystemGO may be implemented in any that they find appropriate in order to evaluate their students’
level of school curricula to promote STEM learning. knowledge about these contents. Therefore, the
PlanetarySystemGO may be implemented by teachers in
Keywords – Mobile augmented reality; location-based schools according to the grade level they teach.
games; serious games; STEM; planetary systems.
The paper is structured as follows: section 2 presents the
I. INTRODUCTION background and context of the study, section 3 the
functionalities of the PlanetarySystemGO information
There is an increasing call to promote STEM (Science, system, section 4 its architecture, and finally conclusions and
Technology, Engineering and Mathematics) education to future work are presented.
better prepare students to the increasing challenges of the real
world [1]. In this regard, the integration of the subjects II. BACKGROUND AND CONTEXT OF THE STUDY
included in the STEM acronym is defended in the literature
and is part of the school curricula in several countries [2]. The Academy of Science, Arts and Heritage
(AcademySAH) is a pedagogical intervention project that
Augmented Reality (AR) is an emerging technology that occurs at the Instituto Politécnico de Tomar (IPT) since 2013,
permits to combine the real world with virtual objects and runs and focuses on establishing a constructivist approach of
interactively in real time [3]. In addition, AR games have the knowledge (www.academiacap.ipt.pt). The team members are
potential to engage students in practice-based activities [4]. higher education professors in the areas of electrical and
Based on a literature review on the use of AR technology to computer engineering, mathematics, biology, physics and
support STEM learning, Ibáñez and Delgado-Kloos [5] refer chemistry, graphic arts, archeology, amongst others. Besides
that few studies provided students with assistance in carrying several activities in the community targeted to students and
out learning activities. The same authors sustain that teachers, the AcademySAH also promotes projects of higher
researchers need to design features that allow students to education students from IPT, under the supervision of the
acquire basic competences related with STEM disciplines. team’s project staff in order to develop hands-on experiments
Moreover, there is a gap in the literature about the use of and prototypes, including mobile games, amongst others.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Some examples intended to promote learning about STEM are Learning objects are celestial bodies, orbits of planets,
“Sonicpaper” related to sound [8], and “SolarSystemGO” information about the celestial bodies, sets of multiple-choice
related to our Solar System [9]. In a preliminary work related questions, amongst others. An event is characterized by a
to the implementation of the SolarSystemGO game, which planetary system parametrized with the following definitions:
was a version hard coded on the app, the authors concluded
that AR games catch children’s attention and promotes - GPS coordinates of the star in the real world,
learning of interdisciplinary subjects related to our Solar - scale that defines the game’s arena and the size of the
System. celestial bodies including the orbital radius of each planet
The PlanetarySystemGO is an upgrade of SolarSystemGO circulating around the star,
to an information system that includes new functionalities - information about the celestial bodies included on the
such as the possibility of including other planetary systems of chosen planetary system,
the Universe [10]. With game based-learning purposes, the
PlanetarySystemGO information system is being designed in - multiple-choice questions about the celestial bodies
the framework of Problem-based Learning. This is an active included on the planetary system.
engagement pedagogy that is very relevant for engineering There are two types of game modes: autonomous game
education because it better prepares the engineering students and game provided by a previous planned event. In the first
for their future professional practice, since it promotes one, when the player starts the game (e.g., holding a
students learning and leads them to achieve 21st century skills smartphone) his location in the real world (gathered by GPS
such as problem analysis, problem solving, communication, coordinates) will be the location of the star of the planetary
teamwork, interdisciplinary knowledge, participant‐directed system in the virtual world. Moreover, the game arena must
learning, critical thinking and creativity, amongst others [11]. be defined by choosing a certain scale according to the
Every school year, a new group of computer engineering player’s preferences and the outdoor space available to walk
students takes the challenge of keeping upgrading the in the real world. The scale corresponds to the orbital radius
information system, in order to improve its performance and of the last planet. In the case of our Solar System it
to include new functionalities. corresponds to the Neptune’s orbital radius.
In the school year 2019/2020, the main challenges The second type of game mode is defined by an event that
proposed to the computer engineering students were to was previously prepared by the teacher through the web
introduce more features in order to: application. In this case, the teacher choses the GPS
- increase the MAR game’s immersive experience coordinates of the star of the planetary system in the real
world, that may be for example the center of the school
- make the back-office more user-friendly to teachers campus where the students are inserted in. Moreover, the scale
- take advantage of the web application as a resource to be arena and the dimensions of the planets, their velocities
used by the teachers in the classroom. around the star, the information about the celestial bodies that
teachers intend to transmit to the students and the set of
In this regard, several tasks were proposed to computer multiple-choice questions to assess students’ knowledge about
engineering students in the context of their final project before the characteristics of the celestial bodies, amongst other
graduation. Among them are the inclusion of asteroids and definitions. Each predefined event is associated with an
moons, new multimedia contents related to the celestial bodies identification code that will be used in the mobile device to
and the development of the web application to provide an download the application, in order to play the game according
interactive modelling of the Solar System on a georeferenced to the teacher definitions (Figure 1). Before starting the game,
map. Three groups of computer engineering students choose students choose a nickname that will serve to catalogue data
the PlanetarySystemGO as their final project in order to collected during the game, which will be available to the
improve its performance and include more functionalities. The teachers through the back-office. In this mode, the star and the
students were supervised by higher education teachers from planets are in the same position for all the players. Students
IPT (four first authors of this paper). should start the game as close as possible to the position
defined for the star of the planetary system in the real world.
III. FUNCTIONALITIES OF THE INFORMATION SYSTEM
In this section, we describe the functionalities of the The game may be played in online or offline mode.
PlanetarySystemGO information system by highlighting the Internet is only necessary to download the autonomous game
MAR application, and the web application that includes a or the event predefined by the teacher. The offline mode
back-office. All those components work as a whole and minimizes problems related to lack of internet in outdoor
interact with each other, in order to comprise the goals of the spaces and performance issues related to interconnectivity.
information system, which is to provide learning about When the game, provided by an event predefined by the
planetary systems in an interdisciplinary way. teacher, is played in offline mode, the information is stored in
the device and sent to the back-office when it gets online.
A. The location-based augmented reality application Therefore, teachers will have access to all the information
The location-based MAR application can be used on about the performance of their students during the game. If the
mobile devices (e.g., smartphones or tablets) that have a game is played online, the teacher will have information about
camera, GPS, gyroscope and accelerometer. Because it the position of the students and their performance in real time
requires GPS it is an outdoor game. Players need to walk in through the web application.
the real world to find virtual objects that consist of orbits and
celestial bodies such as stars, planets, satellites or asteroids,
amongst others. The real world is captured by the mobile
camera and the user’s location is tracked by GPS technology.
Fig. 3. Sequences of the game.
Figure 4 shows the Moon circulating around the Earth and
the respective question. Moreover, it is possible to see its
shadow on the Earth. Once the orbits are found, they turn their
color from yellow to green, and the celestial body gets bigger
Fig. 1. Starting the game. as the player gets closer to it. Finally, after correctly answering
the question, the body is identified with a flag containing the
The game should start with the player's position in the real AcademySAH logo. With this information, the player knows
world representing the star. In this case, the star will be the which celestial bodies he already captured and the ones he still
first virtual object that appears in the screen of the mobile needs to find.
device. After capturing it, a question about its characteristics
appears with 4 answer options (Figure 2). When pressing the
wrong answer, the option is blocked and turns its color to red.
When the correct answer is selected, it turns its color to green
and the player receives points. If he chooses the right answer
in the first choice, he receives 4 points, 3 points in the second
choice, 2 points in the third choice and 1 point in the fourth
choice.
Fig. 4. Question and the Moon circulating around the Earth.
The MAR game has been implemented in informal and
formal learning contexts, including several primary schools,
and has proven to engage the students to play it and also to be
very effective in promoting students' interest in learning about
the Solar System [10]. The in-service teachers of the classes,
where the game was implemented, also considered that it was
Fig. 2. Multiple-choice questions about the Sun and Mercury.
adequate for the school context and that this was a more
After answering the question about the star, the goal is to effective way of teaching students about these contents in a
“travel” through the planetary system to find other orbits and much more motivating way.
celestial bodies such as planets or moons and answer the B. The web application
respective questions. Moreover, if the player touches the
The web application has evolved with the addition of tools
celestial body, information about its characteristics appears in
to allow the planetary systems modelling over any part of the
the mobile device screen. The player gains points when he
world on a georeferenced map. In this regard, a given point on
finds the orbit, captures the celestial bodies or answers
the map is defined as the star and the application provides the
correctly the questions about them. Figure 3 presents some
planetary system modelling on the selected location.
sequences of the game when the planetary system is our Solar
Moreover, it can be used by teachers in the classroom without
System. In this sequence, the player accumulated 37 points.
having to go outdoor, as is the case of the MAR game that
needs GPS. In this regard, it is possible to view the celestial
bodies (which are in motion) with different levels of detail
through a menu and by manipulating the map zoom (Figure
5). The inclusion of celestial bodies and planetary systems that
group them together in repositories allows teachers to prepare
their own lessons or sharing pedagogical contents among teach contents of the Solar System based on previous
them. In this regard, we define three types of repositories in knowledge of their students related to their community. For
the information system: The public repository that is example, the Sun may be placed on the school and the planets
accessible to all users; the repository of an organization (e.g., may circulate around the Sun overlaid on the map of their
school or group of schools) that is accessible only by the municipality. Thus, the teacher may discuss with the students
teachers of that organization and the private repository that is where the orbits of each planet are placed on the map
accessible to an individual person. according to a chosen scale. For example, research if it is near
their homes or heritage monuments that they recognize, and
also relate the size of the planets on the chosen scale with the
size of real objects that they recognize.
Fig. 5. Interaction with the Solar System on a Map of Tomar, Portugal.
Next, we summarize some examples of functionalities that
may be provided by the back-office:
- clone celestial bodies (includes physical characteristic
such as textures, orbital radius, velocities, and also
information about the celestial bodies and questions to
assess students’ knowledge)
- clone planetary systems already parametrized in the
repository
- introduce new planetary systems and celestial bodies Fig. 6. Modelling the Solar System on Malaga city (Spain).
- introduce information about the celestial bodies With the described functionalities, teachers can provide
different parametrizations of the planetary systems just by
- introduce multiple-choice questions
pressing buttons, which will be automatically visualized on
- provide information about the results of the game the simulation provided by the web application. Moreover,
they can create an event with a chosen parametrization that
Events are created by the web application that gather all they intend to explore with their students, which afterwards
the information provided by the back-office. After playing the will run on the MAR application.
game, all the information such as scores, the questions with
more correct answers and number of attempts to answer C. The back-office
correctly a question, if the players searched for information With the inclusion of an information system that
about the celestial body, etc, will be available in the back- communicates with the MAR application, it is possible to
office. Therefore, the teachers can evaluate the performance provide the player with different experiences every time he
of their students during the game. For example, Figure 11 plays the game. In this regard, the back-office is responsible
shows the graphics that resulted from an implementation of for managing the information that will be made available in
the game that took place in a primary school, where the the MAR application. Moreover, teachers may introduce the
application was played with 6 teams, with each team using a contents that they find adequate to teach their students.
mobile phone. Furthermore, all data collected by the MAR application during
the game will be send to the back-office, in order for teachers
By using a menu, different interactions are provided, such to assess students’ performance when playing the game.
as changing the scales of the planetary system by using
“Expand” and “Diminish” buttons, and also moving the Solar Learning objects, associated to celestial bodies, provide
System centre (the star) with the planets to another place on a information about their characteristics through multimedia,
map (e.g. school), which is achieved by using the “Change such as text, images, or animations. In addition, they include
Centre Button”. Figure 5 shows the Solar System on Tomar sets of multiple-choice questions to be answered by the player.
city (Portugal) and Figure 6 shows the Solar System on Celestial bodies and the planetary systems that group them
Malaga city. together are stored in repositories included in the back-office
(Figure 7). By pressing the celestial body, it is possible to see
Moreover, it is possible to change the planet’s speed by or to introduce information about its characteristics (Figure 8).
using “More Speed” and “Less Speed” buttons. This feature
allows an interactive way to model the Solar System on a map In the menu of the celestial body (Figure 7) it is possible
in a web environment with georeferenced data. Besides to visualize the existing multiple-choice questions (Figure 9)
exploring science contents related to the Solar System, we or to edit and insert new sets of multiple-choice questions
propose to explore mathematics. Teachers may use different (Figure 10).
scales to model the Solar System. Furthermore, they can use
the map scale to ask students the distance of the orbits from
the Sun on the map, amongst other possibilities. In addition, if
the map represents their local community, the teacher can
Fig. 11. Team results obtained from the back-office.
As can be seen in the figure, the winning team (Os
Planetários) won 32 points and the worst ranked team (Os Via
Láctea) got 16 points. By placing the mouse cursor over the
Fig. 7. Part of the repository that includes the Trappist System.
graphic, the exact punctuation is revealed, as can be seen in
the team “Os Galáxias” that scored 26 points.
IV. ARCHITECTURE OF PLANETARYSYSTEMGO
In this section, we describe the PlanetarySystemGO
architecture. The PlanetarySystemGO information system is
mainly composed by two components, android application
and a web application, that share data on a common server.
The MAR game is played on the Android application
developed in Unity. The web application runs on a WAMP
server (Windows, Apache, MySQL and PHP) and includes a
back-office and front-office accessed by several web browsers
(Figure 12).
Fig. 8. Information about the Trappist star.
Fig. 9. Questions about Trappist star.
Fig. 12. PlanetarySystemGO architecture.
The server provides an API (Application programming
interface) through REST (Representational State Transfer)
web services to be consumed by the Android application. All
communication in this API uses JSON (JavaScript Object
Notation) standards for data representation.
V. FINAL CONSIDERATIONS AND FUTURE WORK
This paper presents an information system that is being
designed to promote STEM education in the context of the
planetary systems of the Universe, such as our Solar System.
Through the back-office, the information system allows the
introduction of school contents, for example related to the
Fig. 10. Edit question about the Trappist star. Solar System. Therefore, teachers can introduce information
about the celestial bodies of the Universe and also a set of
multiple-choice questions in order to assess students’
knowledge about the contents they intend them to learn.
The contents introduced in the back-office will be played in an outdoor space with mobile devices, but the web
available to be used in the MAR application. After playing the application can be used in the classroom. Finally, we argue
game, all the data collected by the MAR application is that teachers may use the different components of the
accessible to the teachers through the back-office, namely the PlanetarySystemGO information system, that interact with
scores of the players, the answers that students gave to the each other, to prepare interdisciplinary classes related to
questions including number of attempts to get the correct planetary systems, according to the school level they teach.
answer and the time it took to answer correctly, and also if
they consulted the available information about the celestial ACKNOWLEDGMENT
bodies. *This work is supported by national funds through FCT -
Furthermore, the web application can provide the Foundation for Science and Technology, I. P., in the context
modeling of the planetary systems on a map by placing the of the project PTDC/CED-EDG/32422/2017
star on a certain coordinate. Moreover, it parametrizes the
planetary system with a chosen scale by giving dimensions to REFERENCES
orbital radius and diameters of the celestial bodies. By just [1] European Schoolnet, “Science, Technology, Engineering and Mathematics
Education Policies in Europe”. Scientix Observatory report. October 2018,
pressing a button from the menu of the web application it is European Schoolnet, Brussels, 2018.
possible to change the location of the planetary system to
[2] D. Kim, M. Bolger, “Analysis of Korean elementary pre-service teachers’
another location of the map. Teachers can use this application changing attitudes about integrated STEAM pedagogy through developing lesson
in the classroom to explore different scales for example in the plans,” International Journal of Science and Mathematics Education, vol. 15, no.
map of the region where the school is inserted in. Therefore, 4, pp. 587-605, 2017.
students may have an idea of the dimension of the planetary [3] R. T. Azuma, "A survey of Augmented Reality," Presence. Teleoperators and
system comparing for example the orbital radius of planets Virtual Environments, vol. 6, pp. 355-385, 1997.
with distances on the map that they recognize because it [4] P. Fotaris, N. Pellas, I. Kazanidis and P. Smith, "A systematic review of
belongs to the environment where they live including the Augmented Reality game-based applications in primary education," in
Proceedings of the 11th European conference on games based learning
school where they go five days in a week. (ECGBL17), Graz, Austria, 2017.
Because of the COVID 19 pandemic, it was not possible [5] M. Ibáñez, C. Delgado-Kloos, “Augmented reality for STEM learning: A
to perform implementation tests with the whole information systematic review”, Computers & Education, no. 123, pp. 109-123, 2018.
system in schools. For example, we intend that teachers use [6] F. Saltan and O. Arslan, "The use of augmented reality in formal education: a
the PlanetarySystemGO to introduce contents and to use them scoping review," Eurasia Journal of Mathematics, Science & Technology in
Education, vol. 13, pp. 503-520, 2017.
to teach their students in order to assess its impact on schools.
If this scenario continues, we will prepare an online [7] G. Koutromanos, A. Sofos and L. Avraamidou, "The use of augmented reality
games in education: a review of the literature," Education Media International,
intervention because the web application can be used in vol. 52, pp. 253-271, 2015.
videoconference by sharing the screen of the teachers’
[8] C. Ferreira, P. Neves, C. Costa and D. Teramo, “ Socio-constructivist teaching
computer with the students. powered by ICT in the STEM areas for primary school,” in 12th Iberian
Conference on Information Systems and Technologies (CISTI), IEEE, pp. 1-5,
However, several tests of the MAR application with our 2017.
Solar System where successfully implemented in primary
[9] M. C. Costa, J. M. Patricio, J. A. Carrança and B. Farropo, "Augmented reality
schools [10]. Because of these good results already obtained technologies to promote STEM learning," in 13th Iberian Conference on
during the implementation of the MAR application with Information Systems and Technologies (CISTI), Cáceres, Spain, IEEE, pp. 1-5,
previous versions, we believe that the PlanetarySystemGO 2018.
information system will be successfully implemented in [10] M. C. Costa, A. Manso, J. M. Patrício, “Design of a mobile augmented reality
schools. In fact, teachers can use the PlanetarySystemGO to platform with game-based learning purposes”. Information, vol.11, no. 3, 127,
2020.
teach school contents and also to assess students’ knowledge
about those contents at any level of school curriculum. The [11] Z. Huang, P. Hui, C. Peylo and D. Chatzopoulos, "Mobile Augmented Reality
Survey: a bottom-up approach," Cornell University Library, 2013.
MAR application is a location-based game that needs to be