This paper describes the motivations, method, and architecture of a Technology for Revolutionary authoring of Adaptive Intelligent Tutors (TRAIT). Adaptive training solutions such as Intelligent Tutoring Systems (ITS) are showing much promise in inducing learning and improving performance in different application areas, including military combat, maintenance, Intelligence, Surveillance, and Reconnaissance (ISR), and cyber. A central barrier that impedes increased use of adaptive tutor-based training is the high time and cost required to build these training applications. Specifically, there are currently no methods that allow subject matter experts and instructional designers to create, reuse, and maintain training content. Proprietary, “one-off,” and cost prohibitive tools are not the answer for complex environments such as military combat, homeland defense, ISR, and cyber. The new technology described in this paper addresses the following unaddressed requirements for modern intelligent training applications: (1) Allow for rapid design of instructionally sound training content by subject matter experts and instructional designers; (2) enable efficient discovery and reuse of training content between application domains and training system types; and (3) provide mechanisms for both real-time and design-time training content adaptation, thereby enabling graceful evolution of training content design in a manner that addresses continuously changing learner needs and requirements.
innovative methods and tools are necessary for (
The benefits of the TRAIT methods described in this paper include (
Today’s complex cross-domain operational environment poses serious challenges to decision makers tasked with providing effective training opportunities for warfighters. To overcome these hurdles, the US military is currently maximizing access to training leveraging diverse modes such as, simulations, tutors, and games. However, rapid advances in training methodology, performance assessment, learning management system, and adaptive training makes it imperative to reuse software components across various training platforms to optimize cost savings and realize ultimate processing efficiencies. The prime benefit of redeploying software components for performance measures in a new training environment allows maintaining consistency in learning management opportunities [5]. Furthermore, adaptive training methodologies leverage data analytics to forecast trainees’ skill state, rate of skill decay, and recommend a customized training regimen [8]. Finally, TRAIT methods complement the past contributions described above by providing a framework that is compatible with diverse training platforms and leverages domain ontologies to share training content among various systems. The main activities of the TRAIT method and their inter-relationships are shown (see Fig. 1). The TRAIT method is described in more detail in the following paragraphs. The first step of the TRAIT method is to formulate the context and goals for the training event. These goals lay the foundation for determining the appropriate training content and designing the learner performance evaluation metrics. The training goals, in turn, are driven by the (trainable) competencies, which are essential to conducting the mission of the enterprise performing the training (e.g., Mission Essential Competencies (MECs) for military training applications). MECs are defined as “the higher-order individual, team, and inter-team competencies that a fully prepared pilot, crew or flight requires for successful mission completion under adverse conditions in a non-permissive environment” [4]. The MEC process identifies the skills necessary for combat and experience required to become proficient in those skills. MECs are decomposed into sets of Knowledge, Skills, and Experience for the purposes of designing performance evaluation criteria and training missions to address specific deficiencies and ultimately improve the effectiveness of MEC-based training [11]. 1.2
The next step in the TRAIT methodology is to select a training application domain, which is an important decision. The ontology, which is selected from a library of different training application domain ontologies, provides critical information that will help guide and focus the search for reusable training content. The use of automated ontology management technology will facilitate the identification of the appropriate domain ontology models. To gain an appreciation of ontologies and their selection, they will be discussed briefly next.
An ontology is a characterization of a domain that includes a description of the
concepts and relationships of that domain. Subsequently, an ontology model
includes a representation of the rules that govern how the terms that are used to
describe the domain may be combined to make valid statements about situations in a
domain and the inferences that can be made when these statements are used.
Moreover, the Terms in an ontology model include classes (kinds) and individuals
(instances of the classes). Additionally, commonly re-occurring relations that
characterize ontology models include (
The third action of the TRAIT method is to discover and mediate the training content. This activity begins by using an ontology to guide the search over a shared training content repository to identify potentially relevant material that supports the training goals. Next, reusable training content discovered from multiple sources in the repository are compiled and refined to assemble and package the content for the primary training event. The search for reusable training content is performed over a shared content repository, which contains collections of training materials such as courses, case studies, multi-media videos, simulation scenarios, and games. These materials are indexed to facilitate semantic search for optimal retrieval of desired training content. Ultimately, the search for relevant and reusable training content is initiated by the user via combinations of keywords and is enhanced by using domain ontologies in a manner that focuses the search through an elimination of unintended meanings [9]. 1.4
The fourth step of the TRAIT method is to configure and execute training. This activity involves the assembly and sequencing of the training content followed by execution of the activity. The details of the training execution will vary by the type of training and might include a combination of simulation-based training, game-based training, and computer-based training.
The fifth step of the TRAIT method is to evaluate learner performance. This activity
uses a combination of subjective and objective methods to evaluate learner
performance. First, the subjective methods leverage the expertise of the instructors to grade
the relative merit of a learner’s performance during training. Next, the objective
methods use automated methods to generate performance evaluation metrics from the data
collected during training event’s execution. For example, rule-based scripts may be
used in a warfighter combat training application to grade skills such as (
The final step of the TRAIT method is to adapt and refine training content. This activity uses semi-automated ways to adapt and refine training content based on the measured learner performance. For instance, the adaptation methods include data driven methods (e.g., machine learning, deep learning, etc.) and those that use a priori domain knowledge (e.g., rule-based methods). The primary goal of training content adaptation is to optimize the selection and progressive refinement of training content to adequately address deficiencies of individual and team trainees [1]. Now that we have covered the steps of the TRAIT methodology, we will turn our attention to its architecture. 2
The TRAIT conceptual architecture (see Fig. 2) is described in this section. To start off, TRAIT end users include Instructional System Designers, Instructors, and Trainees. Subsequently, TRAIT uses a standards-based, service-based big data cloud infrastructure to manage the high volume and velocity of training data over extended periods of time. Furthermore, TRAIT information exchanges occur across multiple types and modalities of training application supporting different standards and protocols, including High Level Architecture (HLA), Distributed Interactive Simulation (DIS), and Experience Application Programming Interface (xAPI).
A key component of the TRAIT architecture is the Generalized Intelligent Framework for Tutoring (GIFT), which is an empirically-based, service-oriented framework of tools, methods and standards that make it easier to author adaptive training environments, manage instruction and assess effectiveness [6]. Currently, GIFT supports training in various domains with performance evaluations and adaptations specifically designed for the training applications in those domains. However, recent and ongoing enhancements to GIFT have been designed to enable ‘multi-domain adaptive training’ without the need for application-specific extensions to GIFT. Specifically, several application-aware GIFT Application Program Interfaces (APIs) are provided to support information exchanges between GIFT and dissimilar simulation-based and game-based training applications in varying domains [2, 3].
The GIFT API’s are then used by TRAIT to facilitate semantic information exchanges between GIFT, the TRAIT Shared Content Repository, and other training applications. In addition, the TRAIT method was validated by designing a set of domain specific APIs in GIFT that may be used by different training applications for exchanging messages with GIFT, evaluating performance, and adapting scenarios. These applications utilize the APIs to allow interoperability, while reducing the time requirement for integrating different training applications with GIFT. Moreover, the APIs are used to pass information to GIFT, which then uses for training performance evaluation by leveraging definitions provided in the GIFT course DFK file. Next, the results of the training performance evaluations are then sent to either the GIFT tutor or the adaptation constructs in GIFT. If training adaptation is needed, the adaptation strategy is conveyed to the training application using the GIFT APIs. It is important to note that this strategy allows for the communication between a training application and GIFT to be application-specific, facilitating the use of pre-existing training application interface mechanisms.
The TRAIT architecture enables ontology-directed semantic search and content discovery [7] for multi-domain adaptive training content creation and management.
Specifically, TRAIT enables semantic matching and discovery of reusable training content across multiple training applications (Fig. 3). Fig. 3. TRAIT Ontology-directed Semantic Search to Rapidly Find, Share, and Reuse Training Content
As noted in Section 1, the use of ontologies helps focus the search by automatically excluding unintended meanings. Additionally, the TRAIT-accessible ‘Shared Content Repository’ stores and maintains the training content that is available for reuse across multiple training modalities and applications [1]. Furthermore, the scope of the training types includes intelligent tutors, simulation-based training, and game-based training. Moreover, the semantic search and discovery capability uses ontologydirected methods to find relevant training content from related domains that matches the current training event goals. For example, when creating tutor-based content for training space operations personnel, it will be possible to reuse materials previously created in the air-to-air combat training domain. To aid the reader, the concept of using ontology-directed semantic search for discovering reusable training content is illustrated in Fig. 3). Finally, the role of ontologies for improving the effectiveness of semantic search is noted in Section 1 and is described in greater detail in [9].
Ultimately, the ‘discovered’ content information is semantically matched and mediated to address the current training goals. The ‘Reference Ontologies’ are then used for the semantic search and managed using an ontology management capability. Finally, adaptive learning methods are used to fit the training content used on learner and instructor behavior patterns observed over extended periods of time [9]. 3
An example application of TRAIT is illustrated in Fig. 4. The TRAIT system contains
a “shared content repository” and a “domain ontology.” The repository content
contains training information from diverse mediums such as tutors, simulation engines, and
game engines. Each training medium provides unique learning opportunities and in
aggregate, the trainees gain comprehensive understanding of the learning modules. For
example, game engines are a good platform to train individuals on desired skills in the
context of relatively less complex exercises. Simulation engines enable higher fidelity
training of individuals and teams in more complex and larger exercises. Tutoring
systems are used to help individuals learn training concepts with the aid of use case studies,
instructional videos, bench-top exercises, and questionnaires. Additionally, metadata
tags are used to organize training content in a “shared content repository.” Example
metadata tags are course description, course concepts, learner actions type, tasks,
strategy references, instructional strategies, and instructional interventions. Notional
example entries for each metadata category is shown in the figure. Metadata information
when combined with domain ontologies provide an efficient mechanism for organizing
and indexing the training content repository. An example domain ontology, “Clear
Building”, is shown in Fig. 4.
Once a TRAIT system is designed, built, and fielded, instructors will be able to query
the system to extract desired training content. The example Query Input box to the
topright of Fig. 4 shows four parameters, namely, Keywords, Search Scope, Ontology, and
Ontology Depth In this example situation, the following parameter values are used:(
In this example, a match is found between “Perimeter Security” in the Keywords and “Perimeter Security” in the domain ontology. In addition to “Perimeter Security,” “Search and Clear Building,” “Enter Building,” Surveillance, and “Detain Persons” are also collected for building query strings because the traversing length in the ontology model is set at 2. Note that ‘Surveillance’ is dropped from the query string. It is also observed that three of the terms from the query string, ‘Clear
Building’, ‘Detaining Persons’, and ‘Perimeter Security’ match with “Shared Content Repository.” The above example illustrates the process of finding and retrieving relevant and reusable training content. The “Query Output” that is shown at the bottomright side of the figure illustrates the scope of training types used by the TRAIT application: Tutor-based, Simulation-based, and Game-based.
As illustrated by the above application example, the TRAIT system enables the effective discovery and organization of training content to either individuals or teams based on their specific requirements [10]. One of the biggest benefits of TRAIT is that instructors will be able to significantly reduce the time and effort required to find reusable training content over large and disparate content repositories. 4
The main contribution of the ideas described in this paper is a method that enables
dynamic training content authoring and adaptation for agile learning in intelligent
tutoring environments. The TRAIT solution includes (
Benefits of the solution approach described in this paper include (