=Paper=
{{Paper
|id=Vol-3305/short3
|storemode=property
|title=Gamifying a flight instructor simulator with an evidence-based training approach (poster)
|pdfUrl=https://ceur-ws.org/Vol-3305/short3.pdf
|volume=Vol-3305
|authors=Javier Cruz,Antonio J. Guerra,Rubén Dapica,Federico Peinado
|dblpUrl=https://dblp.org/rec/conf/cev/CruzGDP22
}}
==Gamifying a flight instructor simulator with an evidence-based training approach (poster)==
https://ceur-ws.org/Vol-3305/short3.pdf
Gamifying a flight instructor simulator with an
evidence-based training approach
Javier Cruz1, Antonio J. Guerra1, Rubén Dapica2, Federico Peinado1,*
1
Complutense University of Madrid, Department of Software Engineering and Artificial I ntelligence. c/ Profesor José
García Santesmases 9, 28040 Madrid, Spain
2
European University of Madrid, Department of Mechanical and Aerospace Engineering. c/ Tajo s/n, Urb. El Bosque,
28670 Villaviciosa de Odón, Madrid, Spain
Abstract
Even today, the human factor is one of the causes that contributes to the majority of air accidents. That
is why, in recent years, more emphasis has been placed on improving the formation received by pilots.
To this end, the aviation industry is driving a new paradigm called Evidence-Based Training, which
collects data and integrates information on pilot performance across all necessary competencies, both
technical and non-technical. Evaluation of non-technical skills requires well-trained instructors, so a
gamified simulator has been developed with the purpose of teaching to identify observable behaviors
of non-technical competencies in pilots, under this new paradigm. A first experiment with 8 experts
provided a lot of feedback with which to improve this game as a complement to aeronautical training.
Keywords
Aeronautics, Game Development, Assessment, Non-Technical Competencies, Observable Behavior
1. Introduction
Despite all the advances in the field of aviation safety, the human factor continues to be one of
the causes that contributes to the majority of air accidents [1].
For this reason, in recent years, both the International Civil Aviation Organization (ICAO)
[2] and the International Air Transport Association (IATA) [3] have been promoting a new
paradigm called Evidence-Based Training (EBT). This paradigm attempts to adapt training to
the real needs of pilots, integrating both technical and non-technical competencies.
This new training model is concerned not only with pilot technical skills (i.e. the correct
execution of manoeuvres, knowledge of the aircraft controls, etc.), but also with their non-
technical skills, more related to the so-called Human Factors. Flight instructors who are
required to train pilots using the EBT model must observe certain patterns of pilot conduct
called Observable Behaviours (OBs) that reveal whether or not the pilot has the corresponding
competencies, such as Communication or Leadership.
I Congreso Español de Videojuegos, December 1–2, 2022, Madrid, Spain
*
Corresponding author.
$ javiercldo2000@gmail.com (J. Cruz); aguerragarduno@gmail.com (A. J. Guerra); rubendap@ucm.es (R. Dapica);
email@federicopeinado.com (F. Peinado)
https://federicopeinado.com/ (F. Peinado)
0000-0002-1891-1625 (J. Cruz); 0000-0003-3189-306X (A. J. Guerra); 0000-0002-0750-7424 (R. Dapica);
0000-0002-8893-0020 (F. Peinado)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
It is not easy to follow the guidelines of the assessment process in this new paradigm and
often subjective assessments are made in which the instructor does not simply observe and
record the pilot’s OBs. Instructors are therefore faced with this new training paradigm that is
quite different from that traditionally used in aviation, so the question “Who trains the trainers”
arises, the central topic of a previous paper by Hernández, Dapica and Peinado [4].
2. Related Work
Aviation has a long tradition in the use of simulators, mainly due to the need to ensure a safe and
controlled environment for pilot training. That is why the industry has not stopped investing
and innovating in the development of these devices. Proof of this is the fact that the European
Aviation Safety Agency (EASA) is fully adapted to new virtual reality technologies1 .
Serious games, originally defined as games for educational purposes that are still fun, have
been used since the 19th century, even if they were originally paper-based [5]. Today, techno-
logically advanced serious games of different kinds serve a wide range of purposes [6].
The combination of simulation and gaming gives rise to the gamified simulator, a very useful
model for training professionals that is the main proposal of this work. Some examples of
gamified simulators in other areas of application are Afghanistan Pre-deployment: Cultural
Awareness [7], used by the US armed forces to train their soldiers before being deployed in
Afghanistan at the beginning of this conflict; STAR Classroom Simulator [8], where a teacher
can improve his skills as a tutor by interacting with virtual students; and the Versa Project 2 ,
about developing soft skills through the use of commercial video games, on PhD students from
different countries under the European University Alliance “Aurora”.
3. Flight Instructor Simulator
After a first phase of documentation and design, advised by aeronautics experts, the flight
instructor simulator FI-SIM, was developed. For its development we use the latest version of the
Unity development environment, and some help from the educative game company Didactoons3 .
Its architecture allows different flight scenarios to be loaded and different pairs of virtual pilots
to be tested, with the player taking the role of an instructor who, seated behind the cockpit,
evaluates their performance in the different situations that arise (see Figure 1). Currently the
simulator contains a basic tutorial and a detailed scenario based on the real crash of Air Niugini
73, in 2018[9]. We were assisted by expert flight instructors and the official accident report, to
achieve the most credible simulation possible.
The simulator is gamified by simplifying the player’s objective and establishing a clear
dynamic, with short-term rewards. The game is available for free at Itch.io4 , and the source
code is also available in a public repository on GitHub5 .
1
https://www.easa.europa.eu/newsroom-and-events/press-releases/easa-approves-first-virtual-reality-vr-based-
flight-simulation
2
https://gecon.es/versa-project/
3
https://didactoons.com
4
https://narratech.itch.io/fi-sim
5
https://github.com/Narratech/TFG-Cruz-Guerra
Figure 1: FI-SIM in-game footage, showing virtual pilots in action and possible OBs that can be
identified by the player.
4. Results and Discussion
As a first experiment, we conducted a test session at Global Training Aviation6 ’s facilities with
8 expert instructors, who were to play FI-SIM’s tutorial and test scenario (6-minute mission
in which the dialogues of the pilots are very close to the real ones). Experts can choose when,
whom, and what observable behaviors of the pilot to assess, so this interaction helped us a lot to
get relevant information for the evaluation of our game. Compatible apps with mobile devices
(Android and iOS) has been produced that allows validation of its functionality and usability
with real users in those devices.
In short, we can say that all experts considered the new EBT paradigm useful and that the
majority (75%) understood the purpose of the proposed gamified simulator. Only 50% of the
experts were convinced by the tutorial, in order to learn basic game mechanics, and only 37.5%
were convinced by the clarity and the possibility to perform their work well as instructors inside
the game, which means there is still plenty of room for improvement.
Most of them conveyed that they lacked information to fairly evaluate the pilots in the
scenario. Some examples of improvements they suggested in this regard were to have a briefing
before starting the scenario so that the player knows what the pilots in the scenario are going
to face, to show the state of the aircraft by representing it in real time, or to know the pilot’s
previous experience. In addition, we were recommended that each scenario should focus only on
one or two competencies, thus achieving a better dosage of knowledge, which would translate
into better performance when it comes to putting it into practice.
6
https://www.globaltrainingaviation.com/en
5. Conclusions
A computational model for flight simulation scenarios based on EBT has been created with
enough credibility and depth to be accepted as an educative proposal by aviation specialists.
Thanks to the help of official crash reports, as well as independent experts, we were able to
build a realistic simulation, including the evaluation criteria for the pilots training.
In addition, a flight instructor simulator (FI-SIM) based on that model has been designed
and developed to show scenarios allowing the player to reproduce the main part of the EBT
instructor’s job, recognizing whether the OBs of pilots reveal the acquisition of non-technical
skills necessary to carry out their missions safely and effectively. Although we have managed
to reproduce the essentials of an instructor’s work, there is still room for improvement, as
expert comments say that in some situations of the proposed scenario the correct assessment is
debatable, and several or all of the observable behavioral options could be selected as correct, and
a more holistic approach to the problem is desirable. It will also be necessary to apply a formal
methodology, such as [10], to complete the gamification and validate its playful effectiveness.
References
[1] H. Kharoufah, J. Murray, G. Baxter, G. Wild, A review of human factors causations in com-
mercial air transport accidents and incidents: From to 2000–2016, Progress in Aerospace Sci-
ences 99 (2018) 1–13. doi:https://doi.org/10.1016/j.paerosci.2018.03.002.
[2] M. I. C. A. Organization, Manual of evidence-based training, International Civil Aviation
Organization, 2013. URL: https://skybrary.aero/sites/default/files/bookshelf/3177.pdf.
[3] I. A. T. Association, Evidence-Based Training Implementation Guide, International Air
Transport Association, 2013.
[4] A. Hernández, R. Dapica, F. Peinado, Who trains the trainers? towards a flight instructors
simulator based on training scenarios, in: VI Congreso de la Sociedad Española para las
Ciencias del Videojuego, 2020.
[5] J. Torrente, A. del Blanco, E. J. Marchiori, P. Moreno-Ger, B. Fernández-Manjón, lt;e-
adventure gt;: Introducing educational games in the learning process, in: IEEE EDUCON
2010 Conference, 2010, pp. 1121–1126. doi:10.1109/EDUCON.2010.5493056.
[6] A. De Gloria, F. Bellotti, R. Berta, Serious games for education and training, International
Journal of Serious Games 1 (2014). doi:10.17083/ijsg.v1i1.11.
[7] A. Barbieri, A. Tesei, A. Bruzzone, R. Ion, D. Beligan, Learning impact evaluation of the
serious game “cultural awareness – afghanistan pre-deployment”, International Journal of
Serious Games 1 (2014). doi:10.17083/ijsg.v1i3.27.
[8] B. DaCosta, Virtual character conceptual framework: Assessing, preparing and retaining
star teachers for effective classroom management in urban schools (2007).
[9] A. I. Commission, Final Report AIC 18-1004, Technical Report, Accident Investigation
Commission, 2018.
[10] F. J. Gallego-Durán, C. J. V. Arnedo, R. Satorre-Cuerda, P. Compañ-Rosique, R. Molina-
Carmona, F. Llorens-Largo, A guide for game-design-based gamification, Informatics 6
(2019) 49.