It is both expensive, dangerous and partly impossible to practice fire safety scenarios in real environments. Playing a digital game provides us with a view of pupils´ behavior in case of the emergency. In this paper we discuss the pedagogic principles and design approaches followed to develop an AR-based serious game for fire safety. Based on educational sciences, we consider learning as constructed and, as a combination of knowledge, skills and attitudes, which were designed into game mechanics and its pedagogic flow. In the empiric part, we describe the learning outcomes of school aged children before and after the game play. After the game play, school aged children knew the meaning of safety signs better, and they also knew where the signs were located at the school. With the results of this study, we conclude that game-based mobile AR technology can help pupils to learn fire safety issues, increase finding and observing different fire safety signs in their own built environment. As conditions for gaming of this kind are co-development procedures, exact concepts and visuals, authenticity and curriculum-based content of the game.
Earlier studies revealed [
Various strategic documents in Finland state
that learning fire safety is important. The focus is
in children and youth, because their attitudes
toward fire safety are still developing. First,
according to the Basic Act for Education,
everyone participating in education is entitled to a
safe learning environment. [
CEUR Workshop Proceedings (CEUR-WS.org)
safety competence should be promoted among
children and young people by means of regularly
repeated training in various learning and
operating environments. According to this target
program, new learning materials should be
developed. [
For these reasons and purposes, it is well
argumented that there is a need for digital learning
material in fire safety. Compared to widely used
standard and passive training practices, such as
lectures or videos, digital games provide
immersive and engaging experience for learning.
[
Fire in the school environment is the most
common target for safety related VR and AR
experiments [
The objective of this paper is two-fold. Firstly,
the paper discusses the pedagogic principles and
approaches followed to develop serious game for
fire safety [
Some results of the earlier studies show that to
design a successful learning game, the approach
needs to lie on pedagogy [
The educational background of this game
design lies on a few basic principles. Vygotsky´s
[
Authenticity was the third principle applied in
the design. Despite the game having a cartoonish
visual outlook, all fire safety related objects had
to be well placed. The game environment was
created following a co-developing setting [
The pedagogic solutions in the game were created so that teachers are able to use this game as the study material during the lessons. Therefore, it was important that the grammar, definitions and concepts were exact and correct.
The National core curriculum for basic education determines what is taught in Finnish schools, yet the Finnish teachers hold their autonomy how they teach and what kind of learning material they want to use. Before the game design process started, we checked that the cognitive content of the game was connected with the learning content of the National core curriculum for basic education. We decided to provide additional material for teachers, such as slides about fire safety, lesson plans and some additional information about the game metrics.
Universality is the fifth principle that we applied. Despite fire safety being of interest at a global scale, not all the safety signs are equal, neither is the emergency phone number which in Finland and in most of the European countries is 112. Nevertheless, the game was published for iOS and Android mobile devices in the two most popular app stores at a global scale. The game is free to download and play. Most of the schools have hired a teacher, who is responsible for the hardware and software issues as well as for the use of the digital learning material. They tend to check regularly the digital content and evaluate whether the material is suitable to be downloaded to the devices owned by the school. To ensure that the schools were able to use the game during the lessons we decided not to accept any purchasable content or commercial cooperation to keep the game clean for any kind of advertising.
The aim of Virpa - Fire Expert game is to teach
school children, 7–13 years old, fire safety signs
and fire safe behavior. We started the design
phase by informal brainstorming with game
developers, researchers, and fire department
personnel. The aim was to find out a suitable
virtual environment for such emergency
scenarios, and what kind of game (inter)actions
the children and youth engage more with. Based
on our pedagogical aims and principles about
fostering knowledge, skills, attitudes, and
behavioral change i.e., competence [
Fire safety signs and the use of other items like floorplans, fire alarms, fire alarm buttons and fire extinguishers were carefully modeled into the game to improve player´s awareness towards these objects existing in the real world. Furthermore, a set of minigames and actions were designed to improve player´s knowledge and skills and playing experience.
The virtual game environment represents a school building with three floors. (Fig. 1). Each floor has classrooms to unlock (Fig. 2) and a set of hazards, newspapers and minigames to play and discover.
The virtual game environment integrates with the real world via an augmented reality functionality using machine vision algorithms. One of the early brainstorming decisions was to build the game around these technologies. The technology enables the game design to employ this dichotomy to a significant degree, while players are allowed to divide their attention between these two modes of gaming largely as they wish. The scanning taking place in the real world was designed to engage players to interact with real-world environments to enable learning outcomes that differ from any mobile game not utilizing the same dichotomy. This approach to game mechanics were hypothesized to yield more holistic learning outcomes, possibly including changes not only in players’ knowledge, skills and attitudes towards fire safety, but also in the attention they pay to fire safety signs in their every-day environment. Scanning the signs in the real world occurs by activating the mobile phone camera and using it to catch a sign in real spaces, for instance, own school or near-by public spaces (Fig. 3). The machine vision algorithm recognizes the sign that together with a dedicated neural network informs the game of the scanned safety sign (Fig 4).
The technology used in Virpa Fire Expert functions well in most of the buildings, also when the illumination of the building is not very effective. The algorithm was set to identify the following signs: exit, fire alarm button, fire extinguisher, fire hose reel, assembling point and defibrillator. It also could identify actual fire alarm buttons and extinguishers.
After scanning a specific sign, the player could open the door of one of the three rooms reserved to teach the concepts related to specific sign. In the first room the player meets a fire officer (Fig. 2) who makes a question regarding the knowledge about the scanned sign (e.g., do all extinguishers have the same chemical product inside?). The player will be granted with a bronze star linked to that sign if the question was answered right (the player could change the answer infinitely).
Another scan of the same sign and the bronze star will grant the player rights to open the second room, where a skills related question is posed (e.g. how is a fire extinguisher used?). A right answer to that question grants the player a silver star. The silver star and a third scan of the same sign give rights to open the third room, which poses an attitude related question (e.g., how important is this sign?). Any answer in the attitude question grants a gold star and an access to a final exam.
Pedagogically the game flow is that the teaching material presented to the player after each question (e.g., knowledge of extinguishers) provides an answer to the next question (e.g., skills in using extinguishers). The attitude-related question has no correct answer, and therefore the teaching material of the skills question prepares the player to the final exam. Notable is that the question will not immediately follow its corresponding teaching material as the rooms become accessible in a partly non-linear fashion. Together these questions and their corresponding teaching material prepare the player for the final exam.
All in all, player can collect six gold, six silver and six bronze stars that in turn will grant access to the room of the final exam. The final exam includes 18 questions to evaluate the achieved skills and knowledge. The number of the right answers in the final exam determines again the type of diploma awarded to the player: gold, silver or bronze.
With knowledge and skills related questions, the verification of player’s learning is based on a repeated measures research design implemented into the game. Twelve out of 18 questions (excl. attitude questions) form a baseline, a comparison point for each player’s personal learning on their way to the end of the game. The first questions of each sign/room, i.e. 6 questions in total, represent most genuinely each player’s baseline in understanding fire safety signs since these questions are asked before any treatment of that specific topic. Same 12 questions are repeated in the final exam, which allows a comparison of answers in the beginning and at the end of the game play. The underlying assumption is that the treatment – the information the player is exposed to during the game play – will increase the number of correct answers in the final exam compared to the baseline answers. In addition, the final exam consists of six new questions about same topics. That is to confirm the correctness of the learning measurement (i.e., to avoid confirmation bias). This comparison of pre- and post-intervention answers produces a learning rate for each player.
Behavioral learning is built on the AR features, which are to encourage players to move, search, identify and scan fire safety signs in the real world. In contrast to knowledge, skills and attitude related questions discussed above, the measurement of the behavioral change of the player lacks a similar, in-game determined baseline: We do not know how the player has acted before the game play. Thus, the verification of learning must assume that the player has not observed any fire safety signs before the gaming experience. The assumption is that each scanned sign is a positive signal towards the behavioral change, and the larger the personal sign collection and the number of scans is, the more the player has changed one’s behavior in real life.
In addition to the number of scans and signs, behavioral learning and its verification is based on the number of visited areas. Area means the player’s geographical (GPS) location during the sign scan. In the game back-end system, the earth is divided into areas (squares) of 200*200 meters. To increase scans and players’ behavioral change, the game rewards the player who conquers the area first. On the other hand, the player gets less and less points in the game, if the sign scans are taken in the same area. The points decrease very rapidly, which motivates players to move to new areas. Again, we must assume the baseline behavior: The players would have not observed any signs in a certain area without the motivation and intervention given by the game.
The collected metrics related to number of areas, scans and different signs will provide us understanding about the behavior of the player and its assumed level of change due to the game. However, we also conducted an external learning verification with a pre- and post-test questionnaire during a pilot test to get more objective understanding about where, how much and often players observed fire safety signs before playing the game (see Ch. 4).
To keep the player´s interest and engagement on the game, between the scanning actions and the final exam, several minigames and items to find and collect were placed along the school space. A total of nine hazards must be found and solved, for instance coffee machine with damaged electric cord, mobile phone charging close to water point, paint over exit plan, or object obstructing fire exit door. Six newspapers must be found and collected. Each newspaper included a real story of a fire event occurring a school in Finland. The three minigames were designed to also teach skills and improve knowledge. In the skateboard minigame, including three levels, the player needs to follow exit signs to find the fastest way to escape from a building getting covered of smoke. The FireMan minigame represented a modified version of arcade game PacMan where player needs to rescue four friends before they are reached by moving flames. The Fire extinguisher minigame applied AR technology and displayed a virtual fire in the real room occupied by the player that need to be switched off. The aim was to teach the operation of a real extinguisher like pull out the pin, aim to hose to the base of the flames, squeeze the handle. Furthermore, in the Fire drill exercise the player must exit the building following the exit signs.
To study learning outcomes and play experiences of the game, we organized play tests in schools. The participants were Finnish comprehensive school pupils aged 9-13 (avg. 10,5 years old) from two schools in Southwest Finland area. The recruitment took place through personal relations and the participants were not rewarded. As the players were under 15 years of age, permission for game testing was asked from the headmaster, the class teacher and from the parents.
The test subjects created a nick name they had to use in both pre- and post-test questionnaires (T1 and T2). The first test questionnaire (T1) was carried out right before the game play and the second questionnaire (T2) about 14 days after the initial game play. The average play time during 14 days is not known. The total of 260 (n=260) test subjects participated in T1, and 228 subjects (n=228) in T2. Based on the nick name, we could match 193 participants’ pre- and post-test questionnaire answers, and further combine 169 participants’ IDs with their game play data. Besides lower number of participants in T2, unequal nick names led to matching problems and missing data.
Based on our earlier studies on game learning outcome and usability we included 12 multiple choice questions about safety signs but also players´ perceptions about how they learned in the game. Questions number 1-9 of T1 were repeated in the questionnaire T2 to allow the comparison of game play effects in knowledge and behavior of participants. These were complemented in T2 with questions number 10-14 that surveyed participants’ learning and play experiences. In this paper, descriptive statistics is used to show and discuss learning outcomes. 1. Have you noticed any safety signs in your school? (no/one/many) 2. How many kinds of safety signs have you noticed? (0/1-2/3 or more) 3. How often do you notice fire safety signs? (every day/ every week/ seldom) 4. Do you know where in the school area this sign is [assembly point]? (no/maybe/yes) 5. Do you know the meaning of this safety sign [assembly point]? (no/maybe/yes) 6. Have you talked about fire safety with your parents? (no/once/many times) 7. Have you talked about fire safety with your friends? (no/once/many times) 8. How often do you think about fire safety? (never/seldom/every now and then/often) 9. What would you pay attention to if you had to leave a burning school building? (open) 10. Have you talked about the Virpa game with your friends? (no/once/many times) 11. What fire safety issues did the game taught you best? (I find the signs easier/ I notice the signs more often/ I know what the signs mean/ I know what to do in case of fire/ I know what to do to avoid fire/ I think more about fire safety) 12. Which part of the game taught you the best fire safety issues? (Scanning signs, Room questions, Newspaper stories, Minigames, Hazards, Final exam) 13. Which one was more fun: playing in the real school or virtual school? 14. What was the best in the game play? (open)
For the open-ended Question 9, a distinguished qualitative inductive content analysis was carried out (see the chapter 6.1 for results). This kind of method for analysis is used when it is expected that the knowledge of the answers will be fragmented. We carried out this separate analysis to get a holistic picture about how the pupils construct their understanding about their measures in the fire scenario at school. The question was: What would you pay attention to if you had to leave a burning school building? In total, 240 (T1) and 203 (T2) participants answered to this question. A typical length of answer was 1 – 3 sentences. The written material was read through several times. The students' answers were compiled into a matrix from which the meaning units, words or word clusters were retrieved. Typical for the content analysis qualitative method, in the abstraction phase, upper categories and groups were created from responses by selecting the meaning units from the text. These were individual words (for example exit) or related entities of a few words (for example to find the assembly point). Two researchers carried out the categorizing independently and the groups were compared and discussed one by one. Eventually, the responses were formulated into 10 main categories. A second round of categorizing answers was carried out by both researchers individually with the agreed groups. After formatting the groups, the meaning units were quantified to calculate the change in answers between T1 and T2.
The questionnaire answers of the group of comprehensive school beta-testers are compared before and after the game play (questions 1 to 14). In addition, we present the results based on game metrics, such as the learning rate, the number of signs, scans and areas among participants. 5.1.
The first three multi-choice questions were about practical fire safety skills and behavior. Before the game play, 22 % of respondents answered they had not seen any fire safety signs in their school (Question 1). After the game play and period of 14 days, their amount had dropped to zero, whereas the percentage of respondents who have observed many signs had increased from 57 % (T1) to 96 % (T2) (Table 1). Thus, the proportion of people who observed multiple fire safety signs increased by 49 % (Question 1). The percentage of people who observed three or more signs increased by 180 % (from 27% to 88%) compared to the situation before playing the game (Question 2). The percentage of people observing the signs every day increased by 67 % (from 29% to 55%) compared with the situation before playing the Virpa - Fire Expert game (Question 3).
Questions 4 and 5 were about fire safety knowledge. The percentage of respondents who knew the location of the assembly point sign at school increased from 2 % to 9 % (from 4 to 20 respondents). Although there is a 400 % increase in the situation before playing, yet 83 % of the respondents did not know where at school the assembly point sign was located. One reason is that many participants played only inside the school and did not find the sign in its real location. Only about 1 % of the respondents (3 participants) answered that they know the meaning of the assembly point sign before the game play (Question 5). In the post-test questionnaire (T2), 22 % of respondents (49 participants) thought they know the meaning.
Although the fire safety skills were developing based on the information that the respondents had noticed more often and a higher number of safety signs, it seems that playing the game did not increase talking about fire safety with parents or with friends (Questions 6 and 7). In addition, we asked how often respondents were thinking of fire safety (Question 8). 2 % of the respondents answered they think often of fire safety. There was 17 % increase in the figure, but only one more student answered often to this question. 36 % of the respondents answered they think every now and then of fire safety. The respective number was 38 % after the game play. 2. Number of safety signs (3 or more signs)? 3. Noticing fire safety signs (Every day)? 4. Place of assembly point sign? (Yes) 5. Meaning of assembly point sign? (Yes)
Based on the current game data, it is hard to verify knowledge and skills related learning outcomes: Only 47 players (28 %) out of 169 test participants answered questions in the virtual school environment (Table 2). Naturally, even less, only 6 participants, took the final exam. Therefore, the measurement of learning rate is by no means valid (5,1% of improvement on average), yet possible to collect and follow in the future 2.
Based on the game data, a total of 139 players (82,2 %) of all confirmed players (N = 169) have scanned fire safety signs, and of them, 74 % in only one area (Table 3). Low number of different areas is due to playing the game mostly in school premises. However, each (school) building has been scanned very carefully as on average (median) each player has scanned 38 times and found 7 different signs out of 9 possible. Thus, game statistics are consistent with questionnaire answers about the increase in sign observations.
The distinguished content analysis in the openended question (Question 9) resulted in ten categories (Table 4): discourse of things, paying attention to others, following instructions, being calm, responding to the fire, evacuating rapidly, evacuation in general and empty or inappropriate answers. 2 The game has now more than 1800 players (December 2021) and the learning rate data accumulates fast. Change % 49 180,3 67,6 400 1533,3
Two positive changes in results were found. After the game play 38 % less respondents talked about taking or leaving things in the case of emergency (discourse of things and objects). Additionally, there was a result 141 % increase of the meaning units mentioning the safety or exit sign. Statistical significances have not been calculated, yet other changes seem minor. This suggests that two week period with varying amount of game play had not been effective enough to change how participants think they would act in an emergency situation.
However, we think this categorization is valuable knowledge for fire safety communication itself, even without quantification. These spontaneous open answers may portray the most truthful picture of fire safety knowledge, skills and attitudes among participants that the gaming interventions can be compared with.
In the Question 11 (What fire safety issues did the game teach you best? Select one or more options) the options “I find the signs more easily” and “I know what the signs mean” were the most selected (59 % and 53 % of all respondents). In line with answers in questions 6, 7, and 8, the least answered option here was “I think more often about fire safety”. Yet, it was still mentioned by 35 % of all respondents denoting rather even distribution of answers across the different options. This implies that the game manages to teach fire safety in a variety of ways without sacrificing any aspect. This is supported in that, on average, each respondent selected 2.8 out of 6 options. The game had sparked more debate with friends than fire safety issues themselves
Different elements of the game (Table 5) seem to be in balance in terms of perceived learning (Question 12). Only teachability of the final exam remains rather poor (14% of respondents), but the explanation is that this has been visited and performed by only minor proportion of players. The most important element in the game in terms of subjective learning was, as expected, the search and scanning of signs (57% of respondents).
Mini-games were mentioned as the second most important in terms of learning (Table 5), but mini-games were also clearly perceived as the best aspect of the game (Question 14), which may affect the perceived learning of the respondents.
In open-ended Question 14, we asked what the best part of the game was. The qualitative content analysis, like in question 9, produced four themes: 49 % of the respondents liked the minigames the most, the respective percentage of sign scanning was 21 %, whereas 7 % liked most the questions and answers, and other activities (such as avatar tuning) were mentioned best by 23 % of respondents.
This research investigated the learning outcome after playing AR game in fire safety. When enhancing competence in safety culture, memorizing facts and knowing the definitions or concepts is not enough. Thus, we see learning as a construct, and as a combination of knowledge, skills and attitudes. A well-designed learning game provides possibilities for individual tasks, and it is both experiential and memorable.
Playing a digital game provides us with a view of pupils´ behavior in the case of emergency. To create a learning environment that enables ‘transferring’ i.e., applying something learned in the game to real environment, is somewhat challenging. However, after the Virpa game play, school aged children seemed to know the meaning of safety signs better, and they also knew where the signs were located. Both, participants’ subjective answers and game data, point to increase in their knowledge and change in behavior. We conclude that with the help of digital game intervention the school-aged children were able to recognize the safety signs better than before. With the results of this study, we agree that digital technology and AR can help pupils to learn the fire safety signs and remember fire safety issues in their built environment. This is a valuable finding as these actions were made without any assistance from the school. These results show that playing a digital game may give a new start in the person´s behavior. Later, it is possible that this behavior of observing carefully the safety signs in the buildings may lead to develop a positive attitude towards fire safety. These results and technics may be applied in many areas, for instance the traffic or water safety. However, we recognized that knowledge and skills gained while playing AR game are not necessarily transplanted to another context, place or time. For example, our game data showed only rare usage in other than one location.
Yet, we are aware that besides improved
statistical analysis, more objective and precise
assessment of individual behavior change would
be necessary. For example, mobile eye trackers
and virtual reality environments could reveal the
baseline: where, how much and how often players
observe signs before playing. One possibility for
learning game assessment setting would be a
mixed technology gaming experience [
However, we see a lot of potential in future
technologies, especially solutions that are based
on augmented reality promoting fire safety for
children and youth. Gamified solutions exist
especially in the field of education [
Finally, based on our experiences and this study, we see that the conditions for gaming of this kind are co-development procedures, exact concepts and visuals, authenticity and curriculumbased content of the game. With these results, we want to encourage other researchers to design curriculum-based learning games.
The work is part of the Virpa 2 project funded by Fire Protection Fund in Finland. Thanks to both Virpa teams (Fig. 5) at Turku Game Lab in the Turku University of Applied Sciences.