In this paper, we introduce a new framework called Flight Planner for generating Aviation Risk Information mobile-compatible user interfaces from Knowledge Graphs. The framework is divided into three components, a templating engine, a Linked Data Knowledge Graph and a mobile app. Traditional Linked Data frameworks provide poor support for user interface generation and common web frameworks do not natively support Linked Databased Knowledge Graphs. Moreover, Aviation Risk Information systems is specialised area mainly studied by organisational psychologists or experts in human factors and it has received little attention by computer scientists. Thus, a flexible, efficient, and usable framework for Aviation Risk Information is needed. A comparative evaluation was done with Linked Data Reactor framework as a benchmark. It is observed that Flight Planner produces equivalently flexible and efficient, but less usable results despite being an early prototype.
Aviation Risk Information provides pilots and aviation operation staff with warnings
of potential hazards or risks on their schedule. Aviation Risk Information is generated
every day in huge amounts by many different data publishers [
Templating engines provide an approach to flexibly combine data for users. From the state of the art, researches are ongoing into templating engines, Linked Data and mobile apps. However, Aviation Risk Information is a specialised field with specific established ontologies and this could be exploited by a domain-specific templating engine. Web-based, flexible and efficient mobile user interface tools for aviation Knowledge Graphs are immature. Therefore, a new framework for Aviation Risk Information needs to be developed.
The research question explored in this paper is “to what extent a framework can be developed to generate Aviation Risk Information user interfaces from Knowledge Graphs in terms of flexibility, efficiency and usability”. The developed framework is called Flight Planner, which addresses issues: (i) dynamic and heterogenous Aviation Risk Information, (ii) generation of Aviation Risk Information and (iii) immature UI tools for Knowledge Graphs. Three general technical approaches are used: (i) the NASA Air Traffic Management ontologies and graph-based database, (ii) a flexible templating engine and (iii) mobile-compatible user interfaces powered by web technologies. The paper offers a couple of contributions listed below: • An efficient and flexible templating engine for flexibly generating Aviation
Risk Information user interfaces from Knowledge Graphs • A usable and professional aviation-specific user interface element for an
Aviation Risk Information circulation system. • A usable mobile-compatible user interface for browser-based applications.
The remains of this paper is divided into five sections. Firstly, the related work
section defines the gap addressed in current research about Linked Data, templating
engines, mobile apps and Aviation Risk Information system. Secondly the use cases
and requirements section indicates the intended user group for the system and the
requirements of the framework needs to be met in terms of flexibility, efficiency and
usability. Thirdly the system design section describes how the framework is
implemented and constructed. Fourthly, the evaluation section describes a set of
methods used to evaluate Flight Planner by comparing with the Linked Data Reactor1
framework developed by Khalili, Ali, et al [
This section describes related work for four different areas: Linked Data, templating engines, mobile apps and Aviation Risk Information circulation systems. The missing gap among them is defined at the end of the section.
Linked Data (also known as RDF data) is a technique for publishing structured data on
the web. It enables data linked with others in the form of subject -> predicate -> object
called a triple described by Bizer, Christian, et al [
Knowledge Graphs. One of the implementation of Knowledge Graphs is Apache Jena2, which is a graph-based database integrated with SPARQL.
The challenges described by Keller, et al [
For representing Linked data on a UI, one of the ways is to use RDForms.js4, which is an open source project designed for representing Linked Data in RDF/JSON5 on a form without a huge of amount of semantic web knowledges.
A templating engine by definition is a tool to support automatic data generation and
population into a template (e.g. a form or survey). A templating engine researched by
Tatsubori, Michiaki, et al [
UI generations are related to templating engine. However, templating engines
mostly are for developers, which is an obvious difficulty for non-developers to use.
Palmieri, Manuel, et al [
Mobile developments have been seen a huge increase since Android and iOS were
released. According to M.S Ferreira, Cristiane, et al [
The Alitalia app [
There is a significant gap for quickly building Aviation Risk Information mobile apps. The dynamic and heterogeneity of Aviation Risk Information is suitable to be dealt with by Knowledge Graphs. However, there is a missing framework for dynamically and flexibly generating aviation risk information mobile-user interfaces from these Knowledge Graphs. 3
In this section, use cases and associated requirements are described. The users for the framework are identified as developers and aviation risk domain experts. Use cases for the target users are listed below. In order to provide use cases below, the framework needs to meet requirements in terms of flexibility, efficiency and usability.
Use Case 1: Use Case 2:
Use Case 3:
A UI for Aviation Risk Information can be configurable.
The features of a sample aviation-risk information app (Blocker
Reports [
A UI produced by the framework is mobile-compatible.
The flexibility implies whether the framework provides specific requirements defined
as follows. A UI is configurable: a template defines UI structure for data. UI generations
of Khalili, Ali, et al [
A SPARQL query is configurable [
The efficiency implies how easy the functionalities are performed by users which define
as follows. UI generation is efficient: Bizer et al [
According to the Alitalia app [
For the use case 3, the Flight Planner framework is to provide mobile-compatible
UIs, which is a factor to affect the usability level. The precise usability level of a system
is affected by the intended users, and the measurement of usability can vary widely.
But the general usability of a system can be measured by the effectiveness and
satisfaction in System Usability Scale (SUS) described by Brooke, John [
This section describes in detail the implementation and technical design of the Flight Planner framework which is shown in the design Fig. 2. Screenshots of two implementation apps, flight reports on the left hand side and Blocker Reports on the right hand side show in Fig. 1. The Flight Planner framework is constructed by a server and client side described in two subsections below respectively.
Flight Report
List of Blocker Reports
Fig. 1. Flight Planner Interface
The server side is constructed by the Apache Jena and a server written in Node.js. The Node.js sever shown in the Fig. 2 consists of a SPARQL module, UI generator, and request and response handlers. The general process of UI generation is that when the request handler receives a request to generate a UI, the Template Analyser is told to generate a template (a file having empty entries to be filled with data), and the SPARQL module is told to query requested Aviation Risk Information from Jena via SPARQL. After that, a template and queried data are passed to the UI generator, which is for composing the template and Aviation Risk Information into a UI template. It is a template populated with data. The UI template is defined as a configuration file by RDForms.js. Once the UI template is composed, the client side could use it to render a UI on a mobile device by RDForms.js APIs.
Client Render RDForms.js
UI Template
Request Handler Response Handler
Template Analyser
Queried data UI Generator SPARQL Module
Communicate via SPARQL query
retrieve
Jena
TDB2
To make the UI configurable, the generation of a template by the Template Analyser
and queried Aviation Risk Information need to be based on the request, and be matched
in order to generate a correct UI template. For example, the Fig. 3 shows the snippet of
a UI template/configuration file where flightInfo is the type of Aviation Risk
Information containing elements actualArrivalTime and arrivalAirport as data labels.
An initial template generated by the Template Analyser contains only data labels
(actualArrivalTime, etc). The values and types under data labels are to be queried from
Jena. Both data labels and queried data (types and values) need to be matched. More
specifically, the SPARQL module provides a SPARQL query template adapted from
Uger, et al [
For the client side, it is a browser-based app launched on mobile devices’ browser tested in the Google Chrome web browser. A UI template generated by the server is compiled into human-readable forms displaying on devices. According to stated requirements, the framework needs to provide a mobile-compatible UI. The Google Material Design for Bootstrap8 theming framework is used to provide UI components that are styled and shaped into mobile native UI components. Slider and menu components can provide mobile app behaviours. To make the UI more usable in terms of effectiveness and satisfaction, a traditional web UI component is avoided such as selection with large 8 Bootstrap for Google Material Design: https://fezvrasta.github.io/bootstrap-material-design/ amount of data, datetime picker, breadcrumbs navigation, etc, as they belong to desktop UI components illustrated by the Google Material Design guide. 5
To answer the research question and evaluate the Flight Planner framework, the framework is compared to Linked Data Reactor. This section firstly introduces the dataset and the participants, then followed by hypotheses. Secondly, experimental design and interpretation are described in terms of flexibility, efficiency and usability. Lastly, based on the results, suggested changes are discussed at the end.
The dataset contains 100 sample flight instance data that have been converted into Linked Data using the NASA ATM ontologies. 10 participants from the IT background and 2 Aviation Risk Information experts from Trinity College Dublin who have more than 20-years’ experience in Aviation Risk Information area are invited to evaluate the Flight Planner framework.
Flexibility: Efficiency: Usability:
Flight Planner can provide at least as many flexibilities as LD Reactor provides, including a configurable UI and a configurable query. There is no significant difference between two frameworks, in terms of the amount of time spent and trials for participants to perform tasks.
Flight Planner can score above 68.2, an average score in SUS.
Flight Planner was compared with LD Reactor in terms of the following flexibility requirements: • A configurable template: two frameworks are tested whether providing a configuration file or clickable UI components by examining the code level. • A configurable SPARQL query: two frameworks are tested whether they can dynamically generate SPARQL queries based on requests by examining their SPARQL modules and generated SPARQL queries.
Two frameworks are equivalently flexible. By examining two frameworks’ code levels,
two frameworks can provide configurable UI generation. Flight Planner provides a
configuration file (called a UI template described in section 4) and clickable UI buttons
to achieve the configurable UI generation, while LD Reactor provides multiple
configuration files. Also, both frameworks can provide configurable SPARQL query
generation. A query from LD Reactor’s SPARQL module is created by several blocks
of statements and each block can be dynamically generated. The query template from
Flight Planner is created by a number of pre-defined SPARQL sub-queries. Two
SPARQL generations are based on a configurable triple similar to Unger, et al [
10 participants (5 participants to test Flight Planner and another 5 participants to test LD Reactor) were invited to perform two sets of tests listed in Fig. 4. Two sets of tasks are not totally identical due to different use cases of two frameworks, but they are managed to be as related as possible. The equal division of 10 participants can produce records used by a t-test at the last step of procedure. The testing procedure is listed below: •
A single evaluation test of a participant is limited up to 30 minutes, including
introduction and presentation, 2-3 minutes getting familiar with and the tasks.
The “think aloud” methodology [
The number of iterations a participant used to get correct answers, the total time spent, and their thoughts and comments were recorded. One iteration is a failed execution including a navigation to an incorrect page, a click on an incorrect functional button, incorrect operation even on a correct functional UI (e.g. typing in an incorrect format for searching on a searching field), etc. An independent t-test was conducted on two sets of results to determine whether the null hypothesis can be accepted or not by calculating a p-value. Task Id 1 2 3 4 5 6
Common Tasks Query a flight with flight number UAL1479 and find the arrival airport for that flight.
Query a flight with flight number WJA1212 and find the airline for that flight.
Query a flight with a flight number DAL1776 and find its arrival airport IATA code.
Flight Planner LD Reactor Configure the flight report showing the following Query the CEO (a key people) of an airline called information: Aer Lingus. - The flight and flight callsign - The arrival airport name and airport FAA code - The departure airport name and airport ICAO code - The airline name and the headquarter Sort the list of Blocker Reports by the created date Sort the list of airlines by year founded and find and find the latest submitted report. the latest founded one.
Search the list of Blocker Reports for a created Search the list of airlines for a founded year 2002. date 2019-7-20U17:50:37.
Two sets of results are recorded shown in Fig. 5 below. The columns named “ldr” and “fp” represent LD Reactor and Flight Planner, respectively. Each cell represents data of time spent or the number of trials for an individual participant. The t-test results for time spent and the number of trials with significant level 0.05 is shown in Fig. 6. According to two sets of tasks stated in the analysis methods, the t-test can be conducted for first 3 identical tasks and for all tasks. Comparing to the significant value 0.05, the p value for time spent is greatly high which there is no significant difference between two frameworks in terms of time spent to perform tasks. The p value for first 3 tasks is relatively high and the one for all tasks shows less significance due to last 3 nonidentical tasks, but it still can deliver high enough significance. Thus, the null hypothesis for efficiency can be accepted.
Time spent
All tasks First 3 tasks
For usability, after an efficiency test finishes, participants were given a SUS survey9 to test Flight Planner's and LD Reactor’s usability. Feedback and suggestions were also be obtained from participants in terms of whether the Flight Planner app is able to provide a mobile-compatible user interface. Two Aviation Risk Information experts were invited to an interview to evaluate whether the Flight Planner framework is able to provide sample aviation-specific app. Introduction slides, app demo and questions were conducted for 30 minutes. Comments and suggestions were recorded during the interview.
After participants finish SUS survey, two sets of scores can be calculated shown in Fig
7 by the formula provided by Brooke, John [
Sus1 72.5 75.0
Sus2 90.0 67.5
Sus3 Sus4 77.5 75.0 82.5 87.5 Fig. 7. SUS score results
Sus5 87.5 72.5
Mean 80.5 77.0 9 SUS template: https://www.usability.gov/how-to-and-tools/methods/system-usabilityscale.html
According to the comments from two experts with over 40 years combined experience, the implementation of Blocker Reports is acceptable for safety event reporting and mobile-compatible UIs are also accepted by two experts. However, there are two missing functionalities in terms of aviation-specific elements. The first one is an integration with predictive data analytic engine. The second one is a second-layer interface for Blocker Reports, which is a receiver of Blocker Reports provides action recommendation by (1) tracking a flight safety issue, (2) reviewing the issue, (3) analysing the issue and (4) suggesting a solution. This functionality requires different types of reports to handle phases (1) – (4) and they would be then linked together. Safety reports’ status are monitored in phase (1) and (2). The analytics engine would be used in phase (3) and (4). Thereby, the UI generation needs to circulate the process from reporting a Blocker to suggesting a solution, which is missed in Flight Planner considered by experts. 6
The Flight Planner is a framework for generating Aviation Risk Information user interfaces from Knowledge Graphs in terms of flexibility, efficiency and usability. It fills in the missing gap among Linked Data, templating engines and mobile apps. With a comparative evaluation to the LD Reactor, the Flight Planner is able to be equivalently flexible and efficient but less usable. Also, UIs produced by the framework are mobilecompatible accepted by invited participants.
However, the framework has not fully meet the circulation described in the section 5.6, which is significant for an Aviation Risk Information circulation system. The first layer has been implemented, which is reporting safety information. Thus, our plan for the future work is to implement the second layer, an action recommendation consisted of phases (1) – (4) described in the section 5.6. Besides, we plan to make two layers working in a circulation way, which a suggested solution in phase (4) may have impact for a safety issue in the safety reporting layer of other circulation processes. More importantly, the whole process needs to be integrated into the mobile-compatible UI and Knowledge Graphs.
This research has received funding from the Enterprise Ireland Commercialisation Fund (CF2018-2012) and co-funded by the European Regional Development Fund under Ireland’s European Structural and Investment Funds Programme 2014-2020.