Despite all the technological improvements that exist in the aeronautical sector, the majority of accidents are due to human factors. That is why in recent years, the International Civil Aviation Organization has been promoting a new training paradigm that integrates all kinds of skills in crew performance, both technical and non-technical. The training in each session is speci c to the training needs of each pilot, all based on the data obtained in both real operations and training. This alternative paradigm is called Evidence-Based Training. In this training paradigm, pilots are subjected to situations speci cally designed to test and improve their speci c skills. The design of these scenarios is not trivial and is highly dependant on the ability of the ight instructor, who does usually not have much time to generate new and adapted material for each pilot. In this paper we propose the design of a Flight Instructors Simulator that could assist in the training and assessment of these professionals, assuming the mentioned terms.
When we talk about simulators, the rst ones that often come to mind are ight simulators. They have become an essential tool for the aerospace sector.
One of the reasons why simulators are so important is because they allow a
safe environment in which pilots can develop and improve the skills necessary to
operate the aircraft in the safest way possible at all times, minimizing the risk
of putting the lives of others and their own in danger [
The objective of our simulator will not be the training of pilots, instead we will focus on the training of ight instructors in order to improve their teaching Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). skills in such critical task. A simulation environment gives the instructor the possibility to test his/her own abilities when it comes to detecting errors of pilots in certain situations, as well as to practice the possible ways of giving feedback to optimize learning.
The role of the instructor in our simulator concept is creating training sessions oriented to improve the speci c competence pro le of the students to be trained, following the principles of Evidence-Based Training (EBT) philosophy.
This training approach is promoted by the most important aeronautical
institutions like the International Civil Aviation Organization3 (ICAO) [
The development of simulators for the training of ight instructors in EBT is an unexplored eld today. The objective of this paper is to make a rst approach to the problem in order to identify possible strengths and future challenges of this technology.
The rest of the paper is structured as follows. Section 2 reviews the types of ight simulators, the methodologies that guide the work of the instructors in training sessions and the justi cation of a simulation software to be trained in these tasks. Section 3 de nes the competency model used in this work, analyzing how it is related to our methodology. Section 4 explains the role of each part of the work done. Section 5 explains how the proof of concept of our system has been tested on future users, presenting the results that we have obtained. And
nally in Section 6, our conclusions, and the possibilities and limitations of this line of research are discussed. 2
Simulators are virtual learning environments that allow the acquisition,
application and practice of knowledge and skills necessary for the performance of a
certain job. The use of this technology allows improving the learning curve while
o ering a safe environment while reducing costs at the same time [
It was not until the 1940's that ight simulators that used computers able
to solve ight mechanics equations appear. Following this line, in 1948
Curtiss3 https://www.icao.int
4 https://www.iata.org
Wright developed the rst ight simulator that was used by an airline [
Flight simulators would continue to evolve, reaching our days, where they
have high graphic quality and the ability to recreate aircraft and environment
issues with great delity. Simulators could be di erentiated today based on their
motivation:
{ For professional use: Its objective is the training of pilots, allowing them to
acquire and improve both knowledge and skills. This type of simulator must
be approved by the responsible organizations such as the Federal Aviation
Administration (FAA), the European Union Aviation Safety Agency (EASA)
or the Civil Aviation Authorities of the di erent countries.
{ For recreational use: Its objective is the entertainment of the user and
although it can get the user to acquire a series of knowledge, in no case is
their motivation to train the user to become a pilot. In some cases, as in
the case of the X-Plane videogame, organizations such as the FAA give the
possibility of being approved for professional training when used with speci c
hardware. Nowadays, the possibilities o ered by web mapping services and
cloud computing for rendering have raised the visual delity of this software
to the highest level. The most recent example of this is Microsoft Flight
Simulator 2020 [
However, the suitability of a ight simulator training session is not limited
to the ability to accurately represent the stimuli (visual, auditory, kinesthetic
and tactile) of the environment, but it is also necessary to pay attention to
cognitive delity. With current technology, practically any situation that may
arise in ight can be simulated, but the way in which events occur must conform
to consistency criteria. Otherwise, the proposed training scenario would not be
credible and could spoil the usefulness of the session, regardless of the technical
sophistication of the simulator [
The approach based on exposing the trainee to situations suitable to improve
their skills is called Scenario-Based Training (SBT). Balance between repetitive
and unexpected settings are of great importance, as some skills are not always
used during normal work, but are useful in emergency situations. Although the
case at hand is that of ight simulators, SBT has also successfully used in the
medical [
The complexity involved in designing suitable scenarios is very high. The
number of highly quali ed personnel to carry out these tasks has been a limiting
factor in the number of suitable scenarios available in the airlines, hence the
reuse of them has been a common practice [
However, the competency-based EBT approach currently promoted by the
aeronautical authorities requires not only having well-designed scenarios, but
that these be adapted to the competency pro le of the pilot to be trained,
according to the information available by the airline. Knowing what conditions
and events have to be combined during the session, their temporal distribution,
as well as what behaviors of the pilots are to be expected, how they should be
evaluated and, most importantly, what feedback should be given to optimize
learning, require from the instructor a very complete training [
Although the use of simulators for pilot training has a very extensive
background, there are no precedents for similar tools for the training and assessment
of instructors. While there have been tools for the design of scenarios, their
suitability as well as the skill in their application were always evaluated with real
pilots [
Therefore, instructors' performance on the simulator continues to be directly monitored by another evaluator. In this way, the instructor is assessed while performing the work in the simulator with pilots in a real simulator session. This work aims to explore the suitability of a simulator concept that trains the instructor himself, so that he/she can develop the skills necessary for his performance before performing real training sessions. 3
The tool presented is a ight instructor simulator and the idea is to recreate
the work that the instructor requires. We focus on the EBT approach and use
the competencies identi ed as relevant to a pilot by IATA in their guide [
Taking these competences into account, the choices the instructor will have to make will focus on the competencies in which the virtual pilot has the most weaknesses.
It is important to highlight that the elements chosen to make the scenarios (such as emergencies, meteorological phenomena, operational conditions in which the latter take place, etc.) as well as the way in which they are combined are associated with the development of certain competencies. One of the main objectives of this simulator is that the user ends up learning how to combine these elements to work on the skills to be developed based on the speci c pro le of the pilot. Therefore, the simulator must have a knowledge base that associates the architecture of the scenarios and the competences they develop.
As will be seen in the next section, the player (the instructor to be trained by means of this simulator) will have the possibility at low levels of di culty to have guidance about the events and conditions to choose from. However, at the high level of di culty this guide is missed, and the instructor must demonstrate how to correctly choose all those elements. 4
The di erent phases that the player has to go through to complete a session are listed below: { Selection of a pilot. In this part, a list will be shown with all the possible pilots where we can see the full name, an image and a reference of the di culty of teaching said pilot. The di culty will depend on the competences that the pilot has weak grades, with the exception of a 3-star di culty that will mean that the instructor will not receive any type of recommendation. { Selection or creation of a scenario. The instructor will be given the option to select from a set of scenarios or create a scenario from scratch. In case of choosing a preconstructed scenario the instructor will be able to modify the events belonging to the scenario making it more suitable for the pilot. If on the other hand the instructor what he does is build it from scratch, he will simply choose in each ight phase the event that best suits the pilot's needs. { Selection of trigger events. They are a series of events that the instructor can select and that can be launched at will during the pilot evaluation at any ight phase (Ground, Take-O , Climb, Cruise, Descent, Approach, Landing, and Ground again). { Evaluation of the pilot. A \simulation" will be shown, which in our case is a static image of both the exterior and the cockpit of the aircraft representative of the event that would be running. The atmospheric and aircraft information will also be displayed. On the other hand, the instructor will be able to launch the di erent trigger events that he has previously selected, as long as he activates the phase to which they belong, as it can be seen in Figure 1. Once the phases are completed, the instructor must evaluate the pilot in the di erent competitions and, possibly, making observations about his/her performance. { Final report. This phase is in which the instructor himself is evaluated based on the choices made regarding the events on the stage in relation to the needs of the pilot. If the choices were those recommended by the simulator, you will be evaluated correctly and you will be evaluated erroneously if they are not recommended. 5
As a previous step before testing the simulator in person, a video was produced that served as a \proof of concept" showing the functionality of each part of the simulator. In this way the di erent specialists, although they could not use it, could get an idea of how it was used and the image was shown.
Once the video was made, it was shown to 13 di erent specialists within the aerospace eld (pilots, ight instructors and members of di erent organizations within the Peruvian Air Force). After the video, a form was presented that they had to ll out. Among the questions we could distinguish three blocks:
First, to demonstrate that they are specialists related to the aerospace sector. Second, aimed at learning their opinions about the education received by both pilots and instructors. And the third one, destined to know the opinions about the simulator shown in the video and that they would improve or add.
Taking this into account, we will focus on the answers regarding the simulator that are the ones that interest us to evaluate its usefulness.
The specialists' responses (more than 90%) indicated that they understood the concept we were presenting with this ight instructor simulator.
Regarding the information from the pilot, a high percentage (more than 69%) of the responses indicated that the information displayed was su cient. But when indicating what information they believed would be appropriate to add, two results were repeated among the responses, one was the pilot's ight hours and the other referred to the psychological pro le of the pilots.
The responses regarding the information shown both in the scenarios and in the events were very similar, they agreed that the information was su cient (more than 90%), but indicated that more detailed meteorological information than what we already gave would be recommended, and they also believe that it would be necessary to include information on air tra c.
Regarding the evaluation screen, they indicated that it was a screen that was understood (more than 90%), but that it would be necessary for a simulation to be shown instead of a static image of the event.
They agreed with the statement about clarity and ease when navigating between the di erent menus (100%).
And regarding errors, they indicated that the information they mentioned in previous questions would have to be added and also that both the way of qualifying and the ways of presenting the information would have to be adapted to those used by companies in the sector. 6
Considering the answers of the experts after watching the proof of concept video, it seems clear that the proposed work is considered novel, relevant and interesting for the eld. However, this work is only a rst approximation, and the feedback given by the experts will be an unvaluable resource for future developments.
According to the answers given, future steps should be directed mainly in two lines of work. First, the enrichment of the mission information available to create the scenarios, which should be used as an enriched input for an automatic scenario generator. Second, the representation of the possible reactions of the pilots by means of intelligent agents than can express both correct and wrong behaviors in a pedagogical proportion.
In both cases, the use of a knowledge base based on real simulation and operation data would be essential, such as the database at FAA's Aviation Safety Reporting System, so that the simulation of the virtual pilots would be as coherent and realistic as possible.