Traditional Intelligent Tutoring Systems (ITS) are expensive to create and maintain. These systems are restricted in their ability to adapt and deliver targeted training that address observed learner deficiencies. Finally, current ITS solutions are incapable of providing self-regulated training experiences for complex and poorly-defined military tasks. Consequently, adaptive tutoring methods and tools are needed that provide new capabilities, such as tracking learner data, leveraging leaner data to determine a learner's state, and recommending optimal instructional strategies. These adaptive training solutions are effective in inducing learning and improving performance for a wide range of application areas [6,10]. Thus, to adequately address limitations of ITS, innovative methods and tools are necessary for (1) rapidly creating instructionally sound content; (2) allowing efficient content sharing and reuse between domains and training system types; (3) promoting best instructional practices rooted in science; and (4) enabling training content adaptation. The Technology for Revolutionary authoring of Adaptive Intelligent Tutors (TRAIT) was designed to overcome these ITS limitations.
The benefits of the TRAIT methods described in this paper include (1) order of magnitude reductions in time to create adaptive training content; (2) significant savings in training life cycle costs for adaptive training applications via rapid information sharing and reuse; and (3) accelerated migration of research to a broad range of fielded training applications through the use of a standards-based approach.
Today's complex cross-domain operational environment poses serious challenges to decision makers tasked with providing effective training opportunities for warfighters. To overcome these hurdles, the US military is currently maximizing access to training leveraging diverse modes such as, simulations, tutors, and games. However, rapid advances in training methodology, performance assessment, learning management system, and adaptive training makes it imperative to reuse software components across various training platforms to optimize cost savings and realize ultimate processing efficiencies. The prime benefit of redeploying software components for performance measures in a new training environment allows maintaining consistency in learning management opportunities [5]. Furthermore, adaptive training methodologies leverage data analytics to forecast trainees' skill state, rate of skill decay, and recommend a customized training regimen [8]. Finally, TRAIT methods complement the past contributions described above by providing a framework that is compatible with diverse training platforms and leverages domain ontologies to share training content among various systems. The main activities of the TRAIT method and their inter-relationships are shown (see Fig. 1). The TRAIT method is described in more detail in the following paragraphs.
The first step of the TRAIT method is to formulate the context and goals for the training event. These goals lay the foundation for determining the appropriate training content and designing the learner performance evaluation metrics. The training goals, in turn, are driven by the (trainable) competencies, which are essential to conducting the mission of the enterprise performing the training (e.g., Mission Essential Competencies (MECs) for military training applications). MECs are defined as "the higher-order individual, team, and inter-team competencies that a fully prepared pilot, crew or flight requires for successful mission completion under adverse conditions in a non-permissive environment" [4]. The MEC process identifies the skills necessary for combat and experience required to become proficient in those skills.
MECs are decomposed into sets of Knowledge, Skills, and Experience for the purposes of designing performance evaluation criteria and training missions to address specific deficiencies and ultimately improve the effectiveness of MEC-based training [11].
The next step in the TRAIT methodology is to select a training application domain, which is an important decision. The ontology, which is selected from a library of different training application domain ontologies, provides critical information that will help guide and focus the search for reusable training content. The use of automated ontology management technology will facilitate the identification of the appropriate domain ontology models. To gain an appreciation of ontologies and their selection, they will be discussed briefly next. An ontology is a characterization of a domain that includes a description of the concepts and relationships of that domain. Subsequently, an ontology model includes a representation of the rules that govern how the terms that are used to describe the domain may be combined to make valid statements about situations in a domain and the inferences that can be made when these statements are used. Moreover, the Terms in an ontology model include classes (kinds) and individuals (instances of the classes). Additionally, commonly re-occurring relations that characterize ontology models include (1) the 'part-of' relationship between classes or individuals (e.g., the Spark Plug is part-of the Engine); (2) the 'instance-of' relationship between a class and an individual (e.g., the Camry is an instance-of a Car); and
(3) the 'sub-kind' relationship between classes (e.g., a Car is a sub-kind of a Vehicle).
The third action of the TRAIT method is to discover and mediate the training content. This activity begins by using an ontology to guide the search over a shared training content repository to identify potentially relevant material that supports the training goals. Next, reusable training content discovered from multiple sources in the repository are compiled and refined to assemble and package the content for the primary training event. The search for reusable training content is performed over a shared content repository, which contains collections of training materials such as courses, case studies, multi-media videos, simulation scenarios, and games. These materials are indexed to facilitate semantic search for optimal retrieval of desired training content. Ultimately, the search for relevant and reusable training content is initiated by the user via combinations of keywords and is enhanced by using domain ontologies in a manner that focuses the search through an elimination of unintended meanings [9].
The fourth step of the TRAIT method is to configure and execute training. This activity involves the assembly and sequencing of
The fifth step of the TRAIT method is to evaluate learner performance. This activity uses a combination of subjective and objective methods to evaluate learner performance. First, the subjective methods leverage the expertise of the instructors to grade the relative merit of a learner's performance during training. Next, the objective methods use automated methods to generate performance evaluation metrics from the data collected during training event's execution. For example, rule-based scripts may be used in a warfighter combat training application to grade skills such as (1) communication discipline; (2) targeting; (3) mutual support; (4) weapons employment; (5) rules of engagement adherence; etc. Lastly, skills may be graded either offline or in realtime. One advantage of real-time evaluation is that instructors will be able to provide timely intervention to provide context-based feedback to trainees.
The final step of the TRAIT method is to adapt and refine training content. This activity uses semi-automated ways to adapt and refine training content based on the measured learner performance. For instance, the adaptation methods include data driven methods (e.g., machine learning, deep learning, etc.) and those that use a priori domain knowledge (e.g., rule-based methods). The primary goal of training content adaptation is to optimize the selection and progressive refinement of training content to adequately address deficiencies of individual and team trainees [1]. Now that we have covered the steps of the TRAIT methodology, we will turn our attention to its architecture.
The TRAIT conceptual architecture (see Fig. 2) is described in this section. A key component of the TRAIT architecture is the Generalized Intelligent Framework for Tutoring (GIFT), which is an empirically-based, service-oriented framework of tools, methods and standards that make it easier to author adaptive training environments, manage instruction and assess effectiveness [6]. Currently, GIFT supports training in various domains with performance evaluations and adaptations specifically designed for the training applications in those domains. However, recent and ongoing enhancements to GIFT have been designed to enable 'multi-domain adaptive training' without the need for application-specific extensions to GIFT. Specifically, several application-aware GIFT Application Program Interfaces (APIs) are provided to support information exchanges between GIFT and dissimilar simulation-based and game-based training applications in varying domains [2,3].
The GIFT API's are then used by TRAIT to facilitate semantic information exchanges between GIFT, the TRAIT Shared Content Repository, and other training applications. In addition, the TRAIT method was validated by designing a set of domain specific APIs in GIFT that may be used by different training applications for exchanging messages with GIFT, evaluating performance, and adapting scenarios. These applications utilize the APIs to allow interoperability, while reducing the time requirement for integrating different training applications with GIFT. Moreover, the APIs are used to pass information to GIFT, which then uses for training performance evaluation by leveraging definitions provided in the GIFT course DFK file. Next, the results of the training performance evaluations are then sent to either the GIFT tutor or the adaptation constructs in GIFT. If training adaptation is needed, the adaptation strategy is conveyed to the training application using the GIFT APIs. It is important to note that this strategy allows for the communication between a training application and GIFT to be application-specific, facilitating the use of pre-existing training application interface mechanisms.
The TRAIT architecture enables ontology-directed semantic search and content discovery [7] for multi-domain adaptive training content creation and management.
Specifically, TRAIT enables semantic matching and discovery of reusable training content across multiple training applications (Fig. 3). As noted in Section 1, the use of ontologies helps focus the search by automatically excluding unintended meanings. Additionally, the TRAIT-accessible 'Shared Content Repository' stores and maintains the training content that is available for reuse across multiple training modalities and applications [1]. Furthermore, the scope of the training types includes intelligent tutors, simulation-based training, and game-based training. Moreover, the semantic search and discovery capability uses ontologydirected methods to find relevant training content from related domains that matches the current training event goals. For example, when creating tutor-based content for training space operations personnel, it will be possible to reuse materials previously created in the air-to-air combat training domain. To aid the reader, the concept of using ontology-directed semantic search for discovering reusable training content is illustrated in Fig. 3). Finally, the role of ontologies for improving the effectiveness of semantic search is noted in Section 1 and is described in greater detail in [9].
Ultimately, the 'discovered' content information is semantically matched and mediated to address the current training goals. The 'Reference Ontologies' are then used for the semantic search and managed using an ontology management capability. Finally, adaptive learning methods are used to fit the training content used on learner and instructor behavior patterns observed over extended periods of time [9].
An example application of TRAIT is illustrated in Fig. Once a TRAIT system is designed, built, and fielded, instructors will be able to query the system to extract desired training content. The example Query Input box to the topright of Fig. 4 shows four parameters, namely, Keywords, Search Scope, Ontology, and Ontology Depth In this example situation, the following parameter values are used:(1) Keywords: "Perimeter Security" and -Surveillance. The negative sign before Surveillance will instruct the search system to avoid that term in the query string; (2) Search Scope: "Shared Content Repository," the target data source from which content is extracted; (3) Ontology: "Clear Building", the domain ontology model that is used to expand the query string; and (4) Ontology Depth: "2", the specification of the traversing length along the ontology graph that is used to build the query string for performing the search.
In this example, a match is found between "Perimeter Security" in the Keywords and "Perimeter Security" in the domain ontology. In addition to "Perimeter Security," "Search and Clear Building," "Enter Building," Surveillance, and "Detain Persons" are also collected for building query strings because the traversing length in the ontology model is set at 2. Note that 'Surveillance' is dropped from the query string. It is also observed that three of the terms from the query string, 'Clear Building', 'Detaining Persons', and 'Perimeter Security' match with "Shared Content Repository." The above example illustrates the process of finding and retrieving relevant and reusable training content. The "Query Output" that is shown at the bottomright side of the figure illustrates the scope of training types used by the TRAIT application: Tutor-based, Simulation-based, and Game-based.
As illustrated by the above application example, the TRAIT system enables the effective discovery and organization of training content to either individuals or teams based on their specific requirements [10]. One of the biggest benefits of TRAIT is that instructors will be able to significantly reduce the time and effort required to find reusable training content over large and disparate content repositories.
The main contribution of the ideas described in this paper is a method that enables dynamic training content authoring and adaptation for agile learning in intelligent tutoring environments.