This paper will discuss a technology and methodology being researched for an US Army project called the Synthetic Training Environment Experiential Learning for Readiness (STEEL-R) that is producing a new training-data collection and employment strategy, and an experiential learning model [1]. The model is designed to address a need to more rapidly develop increased occupational hard and soft skill competencies that increases advantages over near-peer competitors, and with a higher probability of transfer when applied in a real occupational setting. To do this not only are explicit-technology knowledge and skills need to be trained but also implicit-tacit knowledge and skill required by teams and personnel; however, current training methods are incapable of producing both. By saving, reducing, and labeling real occupational contextual mission, task and performance data, as well as afteraction event reports, assessments, communications, and testimonies (i.e., experiences), we intend to then use a generative artificial intelligence (AI) model to not only enforce technical ability but essentially transfer experiences, and associated tacit knowledge, from existing occupational experts, to new generations of occupational workers. The generative AI model, along with new occupational experience design tools, and a syntactic, machine-readable form of experiential content design, will be discussed, as they apply to a competency-based experiential learning model, that together with the use of adaptive learning technology, we hope will simplify and enable any instructor/trainer to employ experiential learning in their training curriculum.
For generations, occupational learning has been done in linear courses taught in secondary- schools, academic institutions, community-colleges, vocational institutions, and local training sites. This form of training uses mainly didactic, instructor or content centered courses, that require learners to extract knowledge from lectures or text, memorize it until some form of test (usually in a written format) is given, after which an assessment is determined if “learning” has occurred based entirely on a single test sample, using classical test theory based probability scores.
This paper will discuss a technology and methodology being researched for a US Army project
called the Synthetic Training Environment Experiential Learning for Readiness (STEEL-R), that is
producing a new training-data collection and employment strategy, and a competency-based
experiential learning model [
Several key concepts and capabilities are required to first be discussed in order to best convey the overall concept and objective of the research presented to readers not well-versed in the concept and capabilities, as well as refresh and/or align our ideas to those who are. These subjects are tacit knowledge, data-informed competence, generative artificial intelligence, experience design tool, experiential training support packages, and experience events. 2.1.
Tacit knowledge was conceived by Michael Polanyi (1891-1976) who argued that our work- related
ideas, novel practices, innovations, solutions, decisions, hunches that make good workers, leaders and
teams great, are actually a form of intrinsic knowledge that cannot be stated in formal terms. Polanyi
proposed the fact that “we can know and do more than we can tell”, and he termed this phenomenon of
ability as “tacit knowledge” [
Examples of using tacit knowledge include working in teams (that have a lot of experience working together), and knowing what each team member is doing, is best at or knowing when a team member is having difficulty in a specific sub-task they need to perform to effect the team- task outcome. Another example would be when performing a task, noticing and identifying abnormal conditions that require just-in-time alterations to “standard procedures”, and coming up with quick alternative actions. Tacit knowledge includes forms of deductive, inductive and abductive reasoning that occurs without thinking and doesn’t seem to be represented by singular forms of knowledge or skill but a composite of past experiences and outcomes. 2.2.
This paper asserts competence as a long-term indicator of both explicit-technical and implicit- tacit
proficiency for a given task or skill; whereas, traditional degrees and certificates are single- point
indicators that mostly only indicate technical knowledge and skills that may or may-not be current or
“ready” for on-demand use. As such, in our approach, a team or person’s competence is an objective,
unbiased, and data-informed (inspectable) state that can be indicated in two-ways. (1) Competence can
be indicated quantitatively, as a computed probability that a specific team or person is capable of
performing a specific task or skill to a standard, and on-demand, and in a given condition / environment.
The probability is computed from longitudinal data provided from an ALS in an experience application
program interface (xAPI) format (Experience API (xAPI) [
Generative artificial intelligence (AI) has become a huge sensation in recent months with the
inflection point of the Open AI ChatGPT product (Introducing ChatGPT, [
Once a generative AI model is trained, it can then be used to automatically respond to user requests with keyword attributes, to create new reasonable / rational information products like videos, audio, text-documents, software scripts and potentially synthetic environments and entity model behaviors. This paper proposes to use generative AI to add another use of its capability, which is to significantly simplify and improve a new form of synthetic experiential learning content using what’s called an Experience Design Tool (XDT). However, it must be stressed that the creation and/or collection of the noted data, in sufficient quantity, and its analysis and processing, will be the first-step before this capability can be tested and demonstrated; this capability is still at learning engineering design phase. 2.4.
The purpose of the Experience Design Tool or XDT isn’t to create the experience or missionscenario itself but to produce the plot of the mission-scenario, using manual design or generative AI; this occurs in the form of a JavaScript Object Notional encoded, syntactically normalized product we call an experiential training support package or XTSP (more below). When the XDT is enabled, the first step is to either create a new XTSP or re-use and modify an existing one from the XTSP database. In the case of a new XTSP, the process begins with the definition of several key occupational context inputs using the Experience Context form, with which the XDT user must select or accept automated inputs from, for example, an external training recommender engine or scheduling service (see figure 2).
The occupational context inputs are not only critical to define the learner’s occupational context for
tacit learning to occur [
Mission-scenarios can be selected manually from an existing meta-tagged repository or an option to use Generative AI. As noted in figure 2, with the mission-context inputs as query parameters, a randomrealistic mission-scenario, and performance prompts can be produced based on saved data from previous real occupational missions and performances.
Once the context inputs are submitted, a new XTSP is created in a database, with all the missionelements automatically filled in that will form the backdrop or setting of the scenario as shown in figure 4. The mission-scenario will always be based on verified realistic occupational environments, conditions, and tasking that is verified by a highly experienced occupational mission-expert. In addition, the XDT will import various other sources of authoritative data and information to produce or augment existing required metadata needed to make an XTSP searchable by other services or persons needing similar experiential learning. Once the XTSP is created, the XDT user can save it, view it, export it to a synthetic environment to test, or begin to design another function of the XDT, which is the creation of the mission-scenario experience events. 2.5.
An experience event is probably the most important component of the proposed overall experientiallearning model. As illustrated in figure 4, the experience event is a form of experiential lesson and test of a given technical competency, and associated tacit knowledge at the same time. The actual learning process will be discussed later. Learners are both taught by these single or multi-competency prompts (depending on the difficulty they are designed for), and the conditions they suddenly produce in a synthetic environment. Experience events are manually or automatically triggered prompts for performance, which are then assessed and measured by an ALS, using “normed-criterion referenced” measures and standards (that are pre-defined for the given task, skill, knowledge or affect being measured). Unlike the mission- scenario itself, these episodic events can be created manually in the EDT and supported with recommendation from a generative AI; however, as noted earlier, the latter capability requires that sufficient data has already been created / collected and processed from previous real competency-based situations. Creation and/or collection of this data will be the first-step before this capability can be tested and demonstrated.
Experience events can be static within a synthetic environment (meaning they remain in the same physical location and/or attached to a stationary object) or they can be dynamic, and be attached to synthetic entities that can move around, and or require certain conditions to trigger. Experience events can also be “scheduled”, meaning they can be set to only enable at certain times of the mission-scenario. In addition, more than one experience-event can be triggered or activated at the same time, meaning different teams and/or personnel can be prompted to perform different competencies at the same time within the mission-scenario.
Aside from being the element that teaches and tests a specified team or individual competency- item
in experiential learning, another function of an experience event is to form the plot of the larger
experiential mission-scenario story. Each experience event forms the structure and flow [
Each experience event can be “triggered” manually by a trainer or automatically as a result of a targeted entity activating what is called an “event-Handler”. These are sort of like internal software “alarms” that tell other software programs to activate functions when some form of learner-entity activity occurs, like entering within a specified distance of a point on the map, being in line-of-sight of an enemy, entering a building door, turning on a simulated system. Triggers mainly enable the synthetic environment to begin some form of stimulus or cue to start the task, and to activate internal data functions which both begins to send task-related measurement data to the ALS, as well as tells the ALS to begin measuring the player’s task- performance with the sent measurement data.
Experience events inherit most of the mission-scenario’s environment and other mission- properties (e.g., terrain, weather, assignments, limitations, enemy capability, etc…), and these properties are used to restrict the manual design or the automatic AI generation of experience events, in terms of event prompt(s), scenario-time (time of day), difficulty or condition (e.g., adding more enemy to fight when only so many enemy are allocated to the overall mission- scenario). We expect that changes to experience events will only happen in order to manage the degree of challenge that matches the competence level of the targeted team or individual and to ensure they complete the experience they need to be trained in. Finally, another function of the experience event is that they act as a form of “unit of experience” that can be tracked and counted for other logic or even other research purposes.
As noted, the outcome of each experience event is reported from the ALS in a xAPI data formatted message that provides the data-evidence to support later competence computation for the technical and tacit knowledge competency being trained and inspected. 2.6.
The last capability to discuss is the XTSP, which as stated is a standardized syntactic,
machinereadable artifact that stores and transfers real occupational mission-scenario “experiences”, as well as
configures the synthetic training environment to auto-setup scenario characters (called “actors”) who
can be learners or “non player characters” (NPCs), as well as objects and terrain or environment
properties that makes up the setting in a synthetic environment (e.g., a serious game, VR or XR server
and its clients). The XTSP is designed specifically for automating this setup and configuration which
would otherwise take hours to do with current synthetic simulation systems as shown in figure 5. The
XTSP will also auto-configure an ALS, a capability discussion outside the scope of this paper but which
basically helps the trainer/instructor monitor individual learners, and control the synthetic
experientialmission session, as well as assists in data collection, activity assessment, and can even adapt the scenario
or experience events as needed based on an instructor’s instruction or automatically in response to an
actor’s performance. In our particular case, the ALS we are using is called the Generalizable Intelligent
Framework for Tutoring or GIFT [
Overall, aside from storing and managing occupational mission-scenarios and associated experience events, the XTSP is designed to significantly reduce the current steps necessary to employ synthetic training environments, as well as provide an automated process so that any instructor, trainer or professor can employ it in their existing curriculum.
While techniques like storytelling, discussing or studying written case-studies of others’ past experiences and strategies could be considered a means of transferring tacit knowledge, as it was done for centuries, its effectiveness is questionable in that it simply cannot produce the same degree of visceral learning that happens if one was able to actually “re-live” another’s real experience. Today we no longer need to rely on these obsolete methods in today’s era of high- performance computer technology, video and audio processing, three-dimensional computer graphic technology, and artificial intelligence models like generative AI.
Because tacit knowledge is a very personal form of knowledge, and can only really be built from visceral practical experience, we propose using one of many forms of synthetic experiences available today like serious games, virtual reality or mixed reality, in which we can provide individuals and teams with those same types of experiences that are highly concentrated to maximize the technical and tacit execution of a specific tasks or skills.
We propose using a learning model based on experiential learning theory [
For the discussion of developing tacit knowledge, this key experiential (tacit) learning method is proposed. For a given course, subject (associated with specific occupational tasks), and performance objects as delineated within an overall mission-scenario (in the case in figure 6, we’ll use one of the US Army’s synthetic environments), the following events occur.
As noted earlier, the first step is to define the course, subject or target occupational competencies (or technical tasks and associated tacit knowledge), and mission-scenario, which as noted can be actual occupational missions or AI generated missions that will define the specific performance objectives, difficulty level (matched to the learner’s competence level) and conditions to perform in.
This step is optional but provides a point in which any expert provided or media provided necessary preliminary information can be provided to learner(s) that may assist and guide them in the what, where, why, when, and who they will be learning in the experience. This could be considered a form of advanced organizer that consists of a briefing of the overall mission-scenario, as well as any needed preliminary instruction, examples and/or general guidelines or advice to perform with. This preliminary instruction can also be supported by generative AI, dependent upon the database from which it is trained in the specific occupational context.
This step has already been discussed earlier but can be done well in advance of an experiential
learning event or done just before, depending on the overall curriculum being followed in the course.
As noted in figure 6, a key consideration at this step is to make any adjustments needed to ensure the
experience will match the cognitive and learning readiness of the learner [
this step is when the learner is engaged in the mission-scenario by themselves or as part of a team. They interact with the synthetic environment in the same way they would interact in a real occupational work environment, and their activities, communications, and other performance indicators are actively monitored and recorded, as well as assessed in the ALS supporting this experience. The instructor / trainer can monitor any learner at any time (in real-time or by data play-back), and can insert changes to the scenario, provide interventions both electronically, and verbally or can “inject” unplanned experience events as needed. The result of the experience, as indicated by the ALS data is then sent to a storage capability which then awaits the outcomes of the feedback and observation step to complete.
Once an experience is completed, the raw data collected, evaluated and classified in real-time within the ALS can be first analyzed by the trainer / instructor or directly presented back to the learner(s) as states of performance, and evidence of why that state was determined. This step is when learners not only get direct data- informed feedback on what they are good at or need improvement in, but they can vicariously observe what other learners (current or past) did in the same situation. This step is considered an extension of the instructional stimulus provided during the execution step. Both these sources of stimuli are what will produce the actual learning activity and tacit knowledge generation during the next step.
Reflection from experience is considered the major way in which human’s learn [
This paper discussed a learning-science informed education and training method with the support of generative AI technology, to produce new training-data collection and employment strategies and capabilities, and is based around a competency-based experiential learning model. We discussed the concepts we are focused on such as tacit knowledge, competence, and the new technology we are developing to make experiential learning more easier to incorporate into today’s academic, vocational or local training programs and curriculum. We also noted that this method assumes an paradigm of lifelong learning, and supporting synthetic training ecosystem exists, which support the development of tacit knowledge through experience, as well as deepens technical knowledge before a new worker enters the workforce – as opposed to current one-and-done or “check-box” episodic training courses that result in single-point degrees or certifications. Most importantly, we discussed how this learning model continuously collects data presented through synthetic experiential learning opportunities, as stimulated in real occupational mission scenarios. Data that will not only help refine and reveal new competencies in an occupation as it changes in practices and technology, but data needed to train the supporting generative AI, adaptive learning technology, and learner- and trainer - centered design needed to make it easier to employ experiential learning within any institutional curriculum.