=Paper=
{{Paper
|id=Vol-2121/paper3
|storemode=property
|title=Technology for Revolutionary Authoring of Adapative Intelligent Tutors (TRAIT)
|pdfUrl=https://ceur-ws.org/Vol-2121/paper3.pdf
|volume=Vol-2121
|authors=Perakath Benjamin,Kumar Akella,Andrew Stephenson
}}
==Technology for Revolutionary Authoring of Adapative Intelligent Tutors (TRAIT)==
https://ceur-ws.org/Vol-2121/paper3.pdf
19
Technology for Revolutionary authoring of Adaptive
Intelligent Tutors (TRAIT)
Perakath Benjamin, Kumar Akella, and Andrew Stephenson
KBSI, Inc.1408 University Drive East, College Station, Texas 77845, USA
pbenjamin@kbsi.com
Abstract. This paper describes the motivations, method, and architecture of a
Technology for Revolutionary authoring of Adaptive Intelligent Tutors (TRAIT).
Adaptive training solutions such as Intelligent Tutoring Systems (ITS) are show-
ing much promise in inducing learning and improving performance in different
application areas, including military combat, maintenance, Intelligence, Surveil-
lance, and Reconnaissance (ISR), and cyber. A central barrier that impedes in-
creased use of adaptive tutor-based training is the high time and cost required to
build these training applications. Specifically, there are currently no methods
that allow subject matter experts and instructional designers to create, reuse, and
maintain training content. Proprietary, “one-off,” and cost prohibitive tools are
not the answer for complex environments such as military combat, homeland de-
fense, ISR, and cyber. The new technology described in this paper addresses the
following unaddressed requirements for modern intelligent training applications:
(1) Allow for rapid design of instructionally sound training content by subject
matter experts and instructional designers; (2) enable efficient discovery and re-
use of training content between application domains and training system types;
and (3) provide mechanisms for both real-time and design-time training content
adaptation, thereby enabling graceful evolution of training content design in a
manner that addresses continuously changing learner needs and requirements.
Keywords: Adaptive Intelligent Tutor, Instructional Design, Reusable Content
1 Towards a Method for Revolutionary Authoring of
Adaptive Intelligent Tutors
Traditional Intelligent Tutoring Systems (ITS) are expensive to create and maintain.
These systems are restricted in their ability to adapt and deliver targeted training that
address observed learner deficiencies. Finally, current ITS solutions are incapable of
providing self-regulated training experiences for complex and poorly-defined military
tasks. Consequently, adaptive tutoring methods and tools are needed that provide new
capabilities, such as tracking learner data, leveraging leaner data to determine a
learner’s state, and recommending optimal instructional strategies. These adaptive
training solutions are effective in inducing learning and improving performance for a
wide range of application areas [6, 10]. Thus, to adequately address limitations of ITS,
Back to Table of Contents
20
innovative methods and tools are necessary for (1) rapidly creating instructionally
sound content; (2) allowing efficient content sharing and reuse between domains and
training system types; (3) promoting best instructional practices rooted in science; and
(4) enabling training content adaptation. The Technology for Revolutionary authoring
of Adaptive Intelligent Tutors (TRAIT) was designed to overcome these ITS limita-
tions.
The benefits of the TRAIT methods described in this paper include (1) order of mag-
nitude reductions in time to create adaptive training content; (2) significant savings in
training life cycle costs for adaptive training applications via rapid information sharing
and reuse; and (3) accelerated migration of research to a broad range of fielded training
applications through the use of a standards-based approach.
Today’s complex cross-domain operational environment poses serious challenges to
decision makers tasked with providing effective training opportunities for warfighters.
To overcome these hurdles, the US military is currently maximizing access to training
leveraging diverse modes such as, simulations, tutors, and games. However, rapid ad-
vances in training methodology, performance assessment, learning management sys-
tem, and adaptive training makes it imperative to reuse software components across
various training platforms to optimize cost savings and realize ultimate processing ef-
ficiencies. The prime benefit of redeploying software components for performance
measures in a new training environment allows maintaining consistency in learning
management opportunities [5]. Furthermore, adaptive training methodologies leverage
data analytics to forecast trainees’ skill state, rate of skill decay, and recommend a cus-
tomized training regimen [8]. Finally, TRAIT methods complement the past contribu-
tions described above by providing a framework that is compatible with diverse training
platforms and leverages domain ontologies to share training content among various
systems. The main activities of the TRAIT method and their inter-relationships are
shown (see Fig. 1). The TRAIT method is described in more detail in the following
paragraphs.
Back to Table of Contents
21
Fig. 1. Method for Revolutionary Authoring of Adaptive Intelligent Tutors
1.1 Define Training Goals
The first step of the TRAIT method is to formulate the context and goals for the
training event. These goals lay the foundation for determining the appropriate train-
ing content and designing the learner performance evaluation metrics. The training
goals, in turn, are driven by the (trainable) competencies, which are essential to con-
ducting the mission of the enterprise performing the training (e.g., Mission Essential
Competencies (MECs) for military training applications). MECs are defined as “the
higher-order individual, team, and inter-team competencies that a fully prepared pi-
lot, crew or flight requires for successful mission completion under adverse condi-
tions in a non-permissive environment” [4]. The MEC process identifies the skills
necessary for combat and experience required to become proficient in those skills.
MECs are decomposed into sets of Knowledge, Skills, and Experience for the pur-
poses of designing performance evaluation criteria and training missions to address
specific deficiencies and ultimately improve the effectiveness of MEC-based train-
ing [11].
1.2 Select Ontology
The next step in the TRAIT methodology is to select a training application domain,
which is an important decision. The ontology, which is selected from a library of
Back to Table of Contents
22
different training application domain ontologies, provides critical information that
will help guide and focus the search for reusable training content. The use of auto-
mated ontology management technology will facilitate the identification of the ap-
propriate domain ontology models. To gain an appreciation of ontologies and their
selection, they will be discussed briefly next.
An ontology is a characterization of a domain that includes a description of the
concepts and relationships of that domain. Subsequently, an ontology model in-
cludes a representation of the rules that govern how the terms that are used to de-
scribe the domain may be combined to make valid statements about situations in a
domain and the inferences that can be made when these statements are used. More-
over, the Terms in an ontology model include classes (kinds) and individuals (in-
stances of the classes). Additionally, commonly re-occurring relations that charac-
terize ontology models include (1) the ‘part-of’ relationship between classes or in-
dividuals (e.g., the Spark Plug is part-of the Engine); (2) the ‘instance-of’ relation-
ship between a class and an individual (e.g., the Camry is an instance-of a Car); and
(3) the ‘sub-kind’ relationship between classes (e.g., a Car is a sub-kind of a Vehi-
cle).
1.3 Discover and Mediate Training Content
The third action of the TRAIT method is to discover and mediate the training content.
This activity begins by using an ontology to guide the search over a shared training
content repository to identify potentially relevant material that supports the training
goals. Next, reusable training content discovered from multiple sources in the reposi-
tory are compiled and refined to assemble and package the content for the primary
training event. The search for reusable training content is performed over a shared
content repository, which contains collections of training materials such as courses,
case studies, multi-media videos, simulation scenarios, and games. These materials are
indexed to facilitate semantic search for optimal retrieval of desired training content.
Ultimately, the search for relevant and reusable training content is initiated by the user
via combinations of keywords and is enhanced by using domain ontologies in a manner
that focuses the search through an elimination of unintended meanings [9].
1.4 Configure and Execute Training
The fourth step of the TRAIT method is to configure and execute training. This activity
involves the assembly and sequencing of the training content followed by execution of
the activity. The details of the training execution will vary by the type of training and
might include a combination of simulation-based training, game-based training, and
computer-based training.
Back to Table of Contents
23
1.5 Evaluate Learner Performance
The fifth step of the TRAIT method is to evaluate learner performance. This activity
uses a combination of subjective and objective methods to evaluate learner perfor-
mance. First, the subjective methods leverage the expertise of the instructors to grade
the relative merit of a learner’s performance during training. Next, the objective meth-
ods use automated methods to generate performance evaluation metrics from the data
collected during training event’s execution. For example, rule-based scripts may be
used in a warfighter combat training application to grade skills such as (1) communi-
cation discipline; (2) targeting; (3) mutual support; (4) weapons employment; (5) rules
of engagement adherence; etc. Lastly, skills may be graded either offline or in real-
time. One advantage of real-time evaluation is that instructors will be able to provide
timely intervention to provide context-based feedback to trainees.
1.6 Adapt and Refine Training Content
The final step of the TRAIT method is to adapt and refine training content. This
activity uses semi-automated ways to adapt and refine training content based on the
measured learner performance. For instance, the adaptation methods include data
driven methods (e.g., machine learning, deep learning, etc.) and those that use a
priori domain knowledge (e.g., rule-based methods). The primary goal of training
content adaptation is to optimize the selection and progressive refinement of training
content to adequately address deficiencies of individual and team trainees [1]. Now
that we have covered the steps of the TRAIT methodology, we will turn our atten-
tion to its architecture.
2 TRAIT Architecture
The TRAIT conceptual architecture (see Fig. 2) is described in this section. To start
off, TRAIT end users include Instructional System Designers, Instructors, and Trainees.
Subsequently, TRAIT uses a standards-based, service-based big data cloud infrastruc-
ture to manage the high volume and velocity of training data over extended periods of
time. Furthermore, TRAIT information exchanges occur across multiple types and mo-
dalities of training application supporting different standards and protocols, including
High Level Architecture (HLA), Distributed Interactive Simulation (DIS), and Experi-
ence Application Programming Interface (xAPI).
Back to Table of Contents
24
Fig. 2. TRAIT Conceptual Architecture
A key component of the TRAIT architecture is the Generalized Intelligent Frame-
work for Tutoring (GIFT), which is an empirically-based, service-oriented framework
of tools, methods and standards that make it easier to author adaptive training environ-
ments, manage instruction and assess effectiveness [6]. Currently, GIFT supports train-
ing in various domains with performance evaluations and adaptations specifically de-
signed for the training applications in those domains. However, recent and ongoing
enhancements to GIFT have been designed to enable ‘multi-domain adaptive training’
without the need for application-specific extensions to GIFT. Specifically, several ap-
plication-aware GIFT Application Program Interfaces (APIs) are provided to support
information exchanges between GIFT and dissimilar simulation-based and game-based
training applications in varying domains [2, 3].
The GIFT API’s are then used by TRAIT to facilitate semantic information ex-
changes between GIFT, the TRAIT Shared Content Repository, and other training ap-
plications. In addition, the TRAIT method was validated by designing a set of domain
specific APIs in GIFT that may be used by different training applications for exchang-
ing messages with GIFT, evaluating performance, and adapting scenarios. These ap-
plications utilize the APIs to allow interoperability, while reducing the time require-
ment for integrating different training applications with GIFT. Moreover, the APIs are
used to pass information to GIFT, which then uses for training performance evaluation
by leveraging definitions provided in the GIFT course DFK file. Next, the results of
the training performance evaluations are then sent to either the GIFT tutor or the adap-
tation constructs in GIFT. If training adaptation is needed, the adaptation strategy is
conveyed to the training application using the GIFT APIs. It is important to note that
this strategy allows for the communication between a training application and GIFT to
be application-specific, facilitating the use of pre-existing training application interface
mechanisms.
The TRAIT architecture enables ontology-directed semantic search and content dis-
covery [7] for multi-domain adaptive training content creation and management.
Specifically, TRAIT enables semantic matching and discovery of reusable training
content across multiple training applications (Fig. 3).
Back to Table of Contents
25
Fig. 3. TRAIT Ontology-directed Semantic Search to Rapidly Find, Share, and Reuse Training
Content
As noted in Section 1, the use of ontologies helps focus the search by automatically
excluding unintended meanings. Additionally, the TRAIT-accessible ‘Shared Content
Repository’ stores and maintains the training content that is available for reuse across
multiple training modalities and applications [1]. Furthermore, the scope of the training
types includes intelligent tutors, simulation-based training, and game-based training.
Moreover, the semantic search and discovery capability uses ontologydirected methods
to find relevant training content from related domains that matches the current training
event goals. For example, when creating tutor-based content for training space opera-
tions personnel, it will be possible to reuse materials previously created in the air-to-air
combat training domain. To aid the reader, the concept of using ontology-directed se-
mantic search for discovering reusable training content is illustrated in Fig. 3). Finally,
the role of ontologies for improving the effectiveness of semantic search is noted in
Section 1 and is described in greater detail in [9].
Ultimately, the ‘discovered’ content information is semantically matched and medi-
ated to address the current training goals. The ‘Reference Ontologies’ are then used
for the semantic search and managed using an ontology management capability. Fi-
nally, adaptive learning methods are used to fit the training content used on learner and
instructor behavior patterns observed over extended periods of time [9].
3 Illustrative TRAIT Application Example
An example application of TRAIT is illustrated in Fig. 4. The TRAIT system contains
a “shared content repository” and a “domain ontology.” The repository content con-
tains training information from diverse mediums such as tutors, simulation engines, and
Back to Table of Contents
26
game engines. Each training medium provides unique learning opportunities and in
aggregate, the trainees gain comprehensive understanding of the learning modules. For
example, game engines are a good platform to train individuals on desired skills in the
context of relatively less complex exercises. Simulation engines enable higher fidelity
training of individuals and teams in more complex and larger exercises. Tutoring sys-
tems are used to help individuals learn training concepts with the aid of use case studies,
instructional videos, bench-top exercises, and questionnaires. Additionally, metadata
tags are used to organize training content in a “shared content repository.” Example
metadata tags are course description, course concepts, learner actions type, tasks, strat-
egy references, instructional strategies, and instructional interventions. Notional exam-
ple entries for each metadata category is shown in the figure. Metadata information
when combined with domain ontologies provide an efficient mechanism for organizing
and indexing the training content repository. An example domain ontology, “Clear
Building”, is shown in Fig. 4.
Fig. 4. Example TRAIT Application
Once a TRAIT system is designed, built, and fielded, instructors will be able to query
the system to extract desired training content. The example Query Input box to the top-
right of Fig. 4 shows four parameters, namely, Keywords, Search Scope, Ontology, and
Ontology Depth In this example situation, the following parameter values are used:(1)
Keywords: "Perimeter Security" and –Surveillance. The negative sign before Surveil-
lance will instruct the search system to avoid that term in the query string; (2) Search
Scope: “Shared Content Repository,” the target data source from which content is ex-
tracted; (3) Ontology: “Clear Building”, the domain ontology model that is used to ex-
pand the query string; and (4) Ontology Depth: “2”, the specification of the traversing
length along the ontology graph that is used to build the query string for performing the
search.
In this example, a match is found between “Perimeter Security” in the Keywords and
“Perimeter Security” in the domain ontology. In addition to “Perimeter Security,”
Back to Table of Contents
27
“Search and Clear Building,” “Enter Building,” Surveillance, and “Detain Persons” are
also collected for building query strings because the traversing length in the ontology
model is set at 2. Note that ‘Surveillance’ is dropped from the query string. It is also
observed that three of the terms from the query string, ‘Clear
Building’, ‘Detaining Persons’, and ‘Perimeter Security’ match with “Shared Con-
tent Repository.” The above example illustrates the process of finding and retrieving
relevant and reusable training content. The “Query Output” that is shown at the bottom-
right side of the figure illustrates the scope of training types used by the TRAIT appli-
cation: Tutor-based, Simulation-based, and Game-based.
As illustrated by the above application example, the TRAIT system enables the ef-
fective discovery and organization of training content to either individuals or teams
based on their specific requirements [10]. One of the biggest benefits of TRAIT is that
instructors will be able to significantly reduce the time and effort required to find reus-
able training content over large and disparate content repositories.
4 Summary and Benefits
The main contribution of the ideas described in this paper is a method that enables
dynamic training content authoring and adaptation for agile learning in intelligent tu-
toring environments. The TRAIT solution includes (1) a robust knowledge based ap-
proach to facilitate the rapid creation and refinement of tutor-based training content (2)
innovative knowledge discovery and automated reasoning methods for the graceful
evolution of tutoring training content over extended time; and (3) semantic search and
discovery methods to enable content sharing and reuse across multiple training appli-
cation domains and training system types.
Benefits of the solution approach described in this paper include (1) order of magni-
tude reductions in time to create tutor-based adaptive training content; and (2) signifi-
cant savings in training life cycle costs for adaptive tutor based training applications
through rapid information sharing and reuse.
References
1. Benjamin, P., Graul, M., Akella, K., Gohlke, J., Schreiber, B., and Holt, L.: Towards Adap-
tive Scenario Management. In: Proceedings of Inter-service / Industry Training, Simulation,
and Education Conference, Orlando, Florida (2012).
2. Benjamin, P., Wu, C., Fernandes, R., Li, B., and Long, R.: Architecture for Multi-Domain
Adaptive Training. In: Proceedings of Simulation Innovation Workshop, Orlando, Florida
(2016).
3. Benjamin, P., Stephenson, A., Wu, C., Fernandes, R., Li, B., and Long, R.: An Ontolo-
gydriven Framework for Enabling Adaptive Training Application. In: Proceedings of Sim-
ulation Innovation Workshop, Orlando, Florida (2017).
4. Denning, T., France, M., Bell, J., Bennett, W., and Symons, S.: Performance Assessment in
Distributed Mission Operations – Mission Essential Competency Decomposition. In: Pro-
ceedings of Interservice / Industry Training, Simulation, and Education Conference, Paper
No. 1616, Orlando, Florida (2004).
Back to Table of Contents
28
5. Emond, B.: Interoperable assessments using HPML – A novice conning skills acquisition
use case. In: Proceedings of Interservice / Industry Training, Simulation, and Education Con-
ference, Paper No. 17072, Orlando, Florida (2017).
6. Fletcher, J.D.: Intelligent Training Systems in the Military. In: S.J. Andriole and G.W. Hop-
ple, Defense Application of Artificial Intelligence – Progress and Prospects. Lexington, KY
(1988).
7. Sottilare, R., Brawner, K., Sinatra, A., and Johnston, J.: An Updated Concept for a General-
ized Framework for Tutoring (GIFT). In: DOI – 10.13140/RG.2.2.12941.54244. US Army
Research Laboratory, Orlando, Florida (2017).
8. Halverson, K.C., Carlin, A., Reynolds, K., Perlaza, D., and Oster, E.: Operational Learning
– Leveraging Mission Data to Optimize Skill Development. In: Proceedings of Interservice
/ Industry Training, Simulation, and Education Conference, Paper No. 17226, Orlando, Flor-
ida (2017).
9. Hamilton, J., Gopal, B., and Benjamin, P.: Enabling Technology for Collaborative Semantic
Search. In: Proceedings of Conference on Collaboration Technologies and Systems, IEEE
(2013).
10. McCarthy, J.E.: Military Applications of Adaptive Training Technology. In: M.D. Lytras,
D. Gasevice, P. Ordonez de Pablos, and W. Huang, Technology Enhanced Learning – Best
Practices. Hershey, PA (2008).
11. Rodriguez, D., Tossell, C., Garrity, M., and Morley, R.: Promoting Air and Space Operations
Center (AOC) Training Transformation by Quantifying and Refining AOC Training Scenar-
ios. In: Proceedings of Interservice / Industry Training, Simulation, and Education Confer-
ence, Paper No. 1822, Orlando, Florida (2004).
12. Sottilare, R.: Special Report – Adaptive Intelligent Tutoring System Research in Support of
the Army Learning Model, Research Outline. Army Research Laboratory, ARL-SR0284
(2013).
13. Sottilare, R.: Adaptive Training Research – Guided Instruction for Individual and Unit Level
Tasks in Support of FHTE-LS (2014).
Back to Table of Contents