This paper describes the purpose, goals, and guiding questions for the Adaptive Instructional System (AIS) standards workshop within the 2018 Intelligent Tutoring Systems (ITS) Conference Industry Track. Adaptive instructional systems (AISs) use human variability, learner goals and preferences, and other learner/team attributes along with instructional conditions to develop/select appropriate strategies (domain-independent policies) and tactics (actions). The goal of adaptive instruction is to optimize learning, performance, retention, and the transfer of skills between training environments and the work or operational environment where the skills learned during training are to be applied. The Institute for Electrical and Electronics Engineers (IEEE) Learning Technologies Standards Committee (LTSC) established a study group in December 2017 to evaluate the efficacy of AIS standards and the authors of this paper proposed this workshop (and several others) to inform stakeholders and solicit their participation. The interaction with stakeholders at the ITS conference will be through the ideas presented in paper presentations and via an expert panel composed of the authors of this paper and other authors in this workshop.
Adaptive instructional systems (AIS) are defined as: “computer-based systems that
guide learning experiences by tailoring instruction and recommendations based on the
goals, needs, and preferences of each learner in the context of domain learning
objectives” [
In December 2017, the IEEE Learning Technologies Steering Committee (LTSC) formed a 6-month Standards Study Group to investigate the possible market need for standards across AISs. A recent AIS standards workshop in Orlando, Florida highlighted several problems related to the authoring and maintenance of AISs that could be resolved by improving the interoperability of AIS components. This paper discusses the purpose, goals and guiding questions for AIS standards and is a companion document to a live panel of experts who will discuss progress and opportunities for AIS standards. 2
The purpose of this workshop is to educate AIS stakeholders about the IEEE study group and expose them to some of the proposed approaches to standardization through the paper presentations accepted for this workshop. A primary goal is to solicit stakeholder participation in the study group and any resulting IEEE working groups formed in the future in order to bring a diverse set of views and participation to bear in the standardization process. Several interactions with stakeholder communities point to broad interest in AIS standards. Discussions to date indicate opportunities to influence their affordability, their interoperability and reuse, making them more appealing learning technology solutions for the masses. By the time of this presentation at the ITS conference, we will also have additional information to share about any proposals for working groups.
Through an interactive process, the expert panel will provide insight on activities to standardize terms, elements and functions of AISs. The panel members will interact with ITS Conference audience to answer questions and receive feedback from AIS stakeholders. 3
Several areas of AIS technologies are potential candidates for standardization. Robson,
Sottilare & Barr [
The primary answer to this question is “solve marketplace problems”. There must be a perceived value for any standard which is greater than the cost of implementing it. Another driver is breakeven. Typically, business decisions (e.g., implement or don’t implement a standard) are based on the premise of breaking even within two years. If the cost of the standard is so high that it takes longer than two years to get to a breakeven point, it is likely that the standard will not be implemented. This is especially true if there are other ways to solve the problem (e.g., innovation). If the standard (e.g., interoperability, quality, convenience, or usability) solves a problem and does it with minimal cost or requires only a short amount of time to recover the investment, the standard will be perceived to have value. 3.2
Given that we (as developers of standards) believe our standard has value, what is a reasonable measure of its success? Successful standards are adopted in relevant markets and are sustained in those markets. Sometimes standards are adopted based on policy, but the policymakers may be short-sighted with respect to the long term value of the standard. The persistence of a standard in the marketplace is a sign of value and a measure of a successful standard.
Other measures of a successful standard are the diversity of the community of interest who participate in the development of the standard and who ultimately adopt the standard. It is critical that the working group that develops the standard are fully representative of the global community who are intended to adopt the standard. Representation from governments, industry members, and academics are one dimension of diversity. Another dimension of diversity are the countries and regions of the global community. 3.3
The journey can often be as important as the destination when it comes to the standardization process. We advocate for standards development through an open, consensusbuilding process where all stakeholder groups are represented and active. This will give AIS standards the best possible chance of being successfully adopted, will aid in the growth and application of AIS technologies, enhance pathways for products to enter the marketplace and reduce the probability that AIS standards will restrict innovation and creativity in the AIS marketplace. 4
There are several AIS standards candidates that have been presented and discussed during the six months that the IEEE AIS standards study group has existed. Workshops like this one at ITS 2018 have engaged stakeholders and identified approaches ranging from conceptual models to component interoperability to common data structures for learner, domain, and instructional models. Below are four short descriptions of ideas that seem to be taking hold and have potential to meet the criteria discussed in Section 3 of this paper: • Common AIS Conceptual Model - a hierarchical common understanding of the composition of AISs to aid engineers and scientists in communicating their designs and ideas in lectures, presentations, and technical papers as well as in system specifications • AIS Component Interoperability and Reuse – a model of interoperability based on the common AIS conceptual model to facilitate integration and reuse of components through a set of common data messages • Common AIS Learner Model Features – a hierarchical common understanding of the most common features and their representation in both AIS short and long-term learner models • AIS Validation Standards – a testbed methodology to validate AIS compliance, interoperability, and compatibility with adopted AIS standards, and assess AIS system and component effectiveness
Discussion within this workshop covers all of the potential AIS standards. Mr. Avron Barr, IEEE LTSC, led off the AIS standards workshop with a presentation centering on the IEEE standards process and specifically addressed why we need standards. Mr. Barr also addressed the difficulties in the process and the characteristics of a good standard. Following this discussion, we presented and discussed papers covering standardization ideas with respect to: • • •
Component Interoperability (Sottilare & Brawner, p. 55) Learner & Domain Models (McCoy, p. 63)
Pedagogy (McCoy, p. 63, DeFalco, p. 73)
The focus of Sottilare & Brawner’s paper examined the use of the Generalized
Intelligent Framework for Tutoring (GIFT) [
A project authorization request (PAR) was submitted to IEEE LTSC in May 2018 and based on the plethora of activity in AIS stakeholder communities, we anticipated and received approval for an IEEE AIS Working Group in June 2018. Next steps will involve additional stakeholder recruiting to round out the diversity of the IEEE Project 2247 working group. Discussions about the organization, development of formal goals, and assignment of roles and responsibilities should quickly follow. We anticipate the development of a formal definition for AISs and a conceptual model for AISs that will guide future AIS standards development.
Also high on the agenda is understanding of what is and is not included in the family
of technologies known as AISs. This will help us examine appropriate exemplar
systems and discuss their attributes as part of standards discussions on conceptual models
and component interoperability. Since data-driven models are a hot topic, we anticipate
some discussion about the process of developing these types of models as a community
to speed their development [
A portion of the research described herein has been sponsored by the U.S. Army Research Laboratory. Statements and opinions expressed in this paper do not necessarily reflect the position or the policy of the United States Government, and no official endorsement should be inferred.