Applying Artificial Intelligence (AI) to support Guided Experiential Learning (GEL) requires careful consideration from a pedagogical perspective. In this paper, we explore the role of a recommender engine in training decision support with a goal of optimizing the skill acquisition and sustainment process. This involves establishing learning science informed design assumptions grounded in experiential learning and defining associated data requirements and dependencies to drive a mathematical approach to structuring training guidance. We distinguish learning requirements across the different phases of competency acquisition and highlight the role of varying activities that address different foundational functions of the overall learning process.
Experiential learning emphasizes the central role of active engagement with real-world experiences in facilitating learning and expertise development [1]. By actively participating in authentic tasks and environments, learners gain firsthand exposure to the complexities and nuances of the domain, leading to the construction of meaningful mental representations and the development of domain-specific expertise [2]. The immersive and situated context of experiential learning provides learners with opportunities to encounter and resolve real-world challenges, promoting the integration of knowledge and skills into practical application [3].
Deliberate practice complements experiential learning by highlighting the im-portance of focused and intentional effort to improve performance at the knowledge and skill level [4]. Deliberate practice involves engaging in structured activities that target identified gaps, weaknesses or areas for improvement, with the goal of achieving incremental advancements and mastery. Through this approach, learners engage in repetitive and deliberate exercises that challenge their existing faculties, allowing for targeted feedback, reflection, and refinement of performance [5]. By systematically breaking down complex skills into manageable components and engaging in deliberate practice, learners gradually enhance their domain-specific expertise and achieve higher levels of proficiency.
When considering Guided Experiential Learning (GEL), we explore the extension and ultimate role of intelligent tutoring and Artificial Intelligence (AI) to assist and optimize a learner or team’s progression through competency development. With advancements in simulation, eXtended Reality (XR) interfacing, and multi-modal analytics, focused training programs can leverage these immersive technologies to support early exposure and active experiential learning in safe and controlled settings. Furthermore, AI technologies offer unique opportunities to enhance the effectiveness of experiential learning by providing personalized guidance, adaptive feedback, and tailored learning pathways [6].
For this workshop paper, we focus on the pedagogical considerations of experiential learning, with a goal of conceptualizing and identifying initial design requirements for a data-driven recommender engine. To focus the conversation, we examine the use of GEL to support individualized competency development aligned to a team context (i.e., improving the individual to benefit the team). This involves supporting development of interdependent cognitive, psychomotor and affective KSB associations that align to a set of team tasks that have individualized competency requirements. To guide the discussion, we consider the role of pedagogy and GEL recommendations across two learning paths and their underlying objective: (1) a learner progressing from novice to expert and (2) a learner sustaining expertise and proficiency to support future application.
The "crawl, walk, run" framework provides a simple way to conceptualize the progression of skill development, highlighting the incremental nature of competency acquisition. It emphasizes the importance of building a solid foundation, gradually advancing skills, and continuously refining proficiency over time. Here are some generic definitions to help frame the discussion to follow. • Crawl. The crawl stage represents the initial phase of learning and skill development, where learners are introduced to foundational concepts and basic skills. At this stage, learners are acquiring fundamental knowledge and building a solid understanding of the subject matter. They may require significant guidance, repetition, and practice to grasp the basics and establish a strong foundation [7]. • Walk. The walk stage signifies the intermediate phase of skill development, where learners have acquired a reasonable level of proficiency and can perform tasks with increased independence and accuracy. In this stage, learners begin to apply their knowledge and skills in more complex contexts, exploring and expanding their capabilities. While still benefiting from guidance and support, learners become more self-directed and capable of carrying out tasks with greater fluency and efficiency [4]. • Run. The run stage represents the advanced level of skill development, where learners have attained a high level of expertise and can perform tasks with ease, efficiency, and mastery. At this stage, learners exhibit a deep understanding of the subject matter and can apply their skills in complex and challenging situations. They demonstrate advanced problem-solving abilities, adaptability, and the capacity to handle novel or demanding tasks with minimal guidance [4].
While we differentiate the phases of skill acquisition, it is important to account for high level associations on the types of learning interactions associated with experiential learning, and their intended impact on the competency acquisition process. In this paradigm, we establish three distinct performance activity types, drill vs. practice/scrimmage vs. perform.
• Drill. In the context of practice and skill development, a drill refers to a structured and repetitive exercise or activity that focuses on developing specific components or sub-skills of a larger skill or task. Drills often involve isolating particular aspects of a skill and providing repeated practice opportunities to reinforce and automate the associated actions or cognitive processes. They typically follow a predetermined set of steps or patterns and may involve the use of instructional cues, prompts, or demonstrations to guide learners in executing the desired actions accurately and efficiently. • Scrimmage/Practice. Scrimmage, also known as scenario-based practice, involves engaging in practice sessions or activities that simulate real-world or game-like situations. Unlike drills, scrimmages aim to replicate the complexity, unpredictability, and dynamics of actual performance contexts. Scrimmages provide learners with opportunities to apply their skills in more authentic and dynamic settings, often involving interactions with teammates, opponents, or changing environmental conditions. These practice sessions emphasize decision-making, problem-solving, and the integration of various skills and strategies within a realistic context. • Perform. This performance context associates execution of tasks in the real-world operational environment when it matters most. This is the ultimate activity we aim to influence through experiential learning, serving as a culminating event to gauge training effectiveness. 2.1.
The integration of experiential learning and deliberate practice provides a powerful framework for supporting learners in their progression from novice to expert across the Crawl/Walk/Run paradigm. Experiential learning offers the necessary context and authenticity, enabling learners to develop deep understandings of the domain, while deliberate practice provides the structured and focused approach to refine and optimize performance. By combining these two approaches, educators and learning practitioners can design pedagogical interventions that effectively guide learners through the various stages of expertise development, ultimately leading to improved learning outcomes and increased expertise.
Understanding that learning is a social process, the early competency development phases are inherently dependent on a more competent other (i.e., zone of proximal development [8]). From this perspective, a Subject Matter Expert deconstructs the KSB requirements to achieve expertise and builds a focused longitudinal training plan that structures lessons, resources and coaching strategies to address the foundational knowledge and skill components required in the crawl phase.
An interesting case study in this area is referred to as the DanPlan [9]. Dan McLaughlin was a professional photographer and in his late 20’s was introduced to the theory of deliberate practice and the 10,000 rule to attain world class expertise [5]. After deep contemplation, Dan decided to quit his job, start a gofundme account, and to ultimately test this theory in the domain of golf. It’s important to note, at this point in his life, Dan had never held a golf club with real intent, with the exception of a few games of putt-putt across his life. This adventure caught the attention of world class human performance experts, and partnered with him to help apply learning science best practices to see what level of performance and proficiency could be attained through a focused deliberate practice strategy. The following is an excerpt from an article reviewing his journey [9].
“As he progressed, McLaughlin found that many of our instincts turn out to be self-defeating. “People’s intuitions about practice are nowhere near optimal,” says Robert Bjork, a professor in cognitive psychology at the University of California, Los Angeles, whose research has demonstrated the effectiveness of introducing “deliberate difficulty” into practice—for instance, constant variety, “interleaving” between different skills and “spacing” study to force students to retrieve, and embed, new knowledge between sessions. “You want to increase arousal so [the brain encodes] in-formation at a deeper level,’” says Mark Guadagnoli, a professor of neuroscience and neurology at the University of Nevada, Las Vegas, School of Medicine. “It’s [like] using a laser to engrave something versus a ballpoint pen.” With advice from Bjork, Erics-son, Guadagnoli, and others, McLaughlin incorporated these principles.”
While Dan never attained his goal of becoming a professional golfer, his self-administered experiment provides interesting insights into the real-world application of focused deliberate practice. He produced impressive results, but this could not be accomplished without help and support in defining exactly how to drill and practice in support of his overarching progression through the skill acquisition curve. This highlights an interesting opportunity for AI to provide world class coaching support when more competent peers are not available to guide your performance pursuits. 2.2.
Experiential learning continues to play a crucial role in maintaining superior performance for learners who are already experts in a specific domain. While experts have attained a high level of proficiency, their ongoing engagement in experiential learning allows them to adapt, refine, and extend their expertise to remain at the forefront of their field. Part of this is maintaining an emphasis on the basic fundamentals and their role in successfully executing novel tasks with desired performance outcomes. These KSB elements are the focal point in the crawl and early walk phase of skill acquisition but require application at appropriate intervals and under context free conditions to maintain proficiency and automaticity when they are required under novel and critical performance situations.
Anecdotally, here’s a quote from an interview with Kobe Bryant on his practice regiments. Kobe is considered one of the best basketball players of all time and was meticulous with his approach to training.
world? I never get bored with the basics!”
For this purpose, a recommender engine must account for drill level training requirements at appropriate intervals to maintain required levels of proficiency. An associated competency model aligned to experiential learning will require evidence of ex-pert application of fundamentals prior to initiating more context-oriented practice scenarios.
The role of a recommender engine in the context of intelligent tutoring and adaptive instruction is to provide personalized recommendations and guidance to learners based on their individual needs, preferences, and performance data. A recommender engine employs algorithms and machine learning techniques to analyze vast amounts of learner data, such as their past interactions, learning outcomes, and demographic information, to generate tailored suggestions for instructional content, learning activities, or learning pathways [10]. By leveraging these data-driven insights, recommender engines can offer adaptive and individualized support, ensuring that learners receive targeted recommendations that align with their specific learning goals and capabilities. 3.1.
In our case, GEL extends a recommender engine type service to support interaction across an ecosystem of learning resources that can combine to drive the competency acquisition process. In this instance, we theorize that there are a number of technologies that can be used to support a learning requirement, and a learner’s current acquisition phase will dictate the type of environment and psychological fidelity to support their associated goals. This can involve use of simulations, game environments, and XR modes of interaction to target specific KSB elements that are required to across the cognitive, psychomotor and affective learning dimensions. This also accounts for personalization of interaction characteristics (e.g., task difficulty and complexity) that assist in facilitating ideal deliberate practice [11]. When considering a recommender engine, the with an emphasis on driving experiential learning benefit.
When considering the goals of GEL and the role AI can play in supporting learner objectives, a recommender engine capability requires a mathematical approach to represent the variables and theories that drive skill acquisition theory [12]. In this section, we examine the role of a recommender engine to help learners plan and prioritize their scheduled training sessions, with a goal of selecting competencies that need most attention and balancing activity types based on current competency and proficiency levels. 4.1.
To create a mathematical equation for planning a scheduled session for experiential learning, we consider a weighted sum approach that takes into account the competency state, recency, decay rate of each competency, and an associated ratio looking at the balance between drill and scrimmage task recommendations. The following variables are considered: • N: The number of competency frameworks aligned to tasks • S_i: Current competency state for competency framework i (untrained, practiced, proficient) • R_i: Recency of competency application for competency framework i (measured in time units, e.g., days) • D_i: Decay rate for competency framework i (measured in competency loss per time unit, e.g., proficiency points per day) • T: Training session time allotment (measured in time units, e.g., hours)
Total_score = Σ (w_i * S_i * exp(-D_i * R_i)) * ratio (1)
In this equation, the total score is multiplied by the ratio parameter. This allows you to adjust the balance between drill and practice time based on the learner's current acquisition phase. To clarify the interpretation of the ratio parameter: • If ratio > 1: It indicates a greater emphasis on drill time compared to practice time. The learner will spend more time on structured exercises, repetitive tasks, or knowledge acquisition. • If ratio < 1: It indicates a greater emphasis on practice time compared to drill time. The learner will spend more time on hands-on application, real-world tasks, or problem-solving activities. • If ratio = 1: It represents an equal balance between drill and practice time.
Here are some additional assumptions to take into consideration of an early design. In the initial stages of learning, during the crawl phase, it is beneficial to focus more on drill activities to build a solid foundation of knowledge and basic skills. A recommended ratio for the crawl phase could be in the range of 70% drill to 30% practice. As the learner progresses to the walk phase, they have developed a basic understanding and proficiency in the skills. At this stage, it is important to start increasing the emphasis on practice activities to enhance the application and problem-solving abilities. A recommended ratio for the walk phase could be around 50% drill to 50% practice, striking a balance between reinforcing foundational knowledge and promoting practical application. In the advanced stage of skill development, the run phase, the learner should focus more on practice activities to further refine their skills and apply them in real-world scenarios. Practice activities in this phase could involve complex, challenging tasks that require higher-order thinking and decision-making. A recommended ratio for the run phase could be in the range of 30% drill to 70% practice.
These recommended ratios provide a general guideline, but it's important to adapt them based on the specific learning objectives, the complexity of the competencies, and the individual learner's progress and needs. Regular assessment and feedback can help gauge the learner's readiness to progress from one phase to another and adjust the ratio accordingly. Remember that the purpose of these ratios is to strike a balance between building foundational knowledge (through drill) and promoting practical application and problem-solving (through practice) to support effective skill development throughout the crawl-walk-run continuum. 4.2.
The mathematical equation for planning a scheduled session can be modified to incorporate reinforcement learning concepts such as reward and value functions. Instead of using state values or action values directly, we can adapt the competency math model to incorporate reinforcement learning components. Let's assume we have a reward function R(s, a) that provides a numerical reward for taking action a in state s. The modified equation can be expressed as: (2)
Here, s_i represents a specific state or competency associated with a task, and a_i represents the corresponding action or practice activity. R(s_i, a_i) represents the reward obtained from taking action a_i in state s_i. R_i represents the recency of the state-action pair, and D_i represents the decay rate associated with the competency.
In the context of reinforcement learning, the ratio of drill vs practice activities can be associated with the exploration vs exploitation trade-off. Exploration involves taking actions to gather more information about the environment and learn better strategies, while exploitation involves selecting actions that are known to yield high rewards.
To incorporate the ratio, we can adjust the balance between exploration and exploitation during the learning process. A higher ratio would encourage more exploration, allowing the learner to try different actions and gain a better understanding of the task. A lower ratio would prioritize exploitation, focusing on actions that have previously resulted in high rewards. By dynamically adjusting the ratio parameter during the reinforcement learning process, we can influence the learner's exploration-exploitation tradeoff and guide their decision-making.
In this paper, we introduce considerations for a recommender engine designed around the tenets of experiential learning and deliberate practice principles. We emphasize the need for a balance of focused drill type activities that target specific knowledge, skill and behavior components with realistic handson practice opportunities that replicate the real-world environment these competencies are applied within. This involves identifying and prioritizing training requirements aligned to tasks and the underlying competencies required for optimal performance. We identify specific variables that must be tracked at the learner and learning resource level, and emphasize sustainment of basic fundamental skills required for expert proficiency.
As a limitation, the work introduced above has been presented in a relatively general manner. The forward goal is to take these modeling assumptions and directly align them to an implementation of a training ecosystem and data modeling approach that supports the GEL requirements. The first application will be within the Synthetic Training Environment Experiential Learning for Readiness (STEEL-R [6]) data strategy, which leverages adaptive instructional systems components and standards aligned to the ADL Total Learning Architecture (TLA [13]). A carefully developed eXperience Training Support Package (XTSP) data model was established to support the measurement of discrete experience events within a GEL type setting, and is used to support the configuration and calibration of assessment and data management techniques that will guide recommender engine design and implementation [11].
[1] Dewey, J. "Experience and education." In The educational forum, vol. 50, no. 3, pp. 241-252.
Taylor & Francis Group (1986). [2] Kolb, David A. Experiential learning: Experience as the source of learning and development. FT press (2014) [3] Lave, J. & Wenger, E. Situated learning: Legitimate peripheral participation. Cambridge university press (1991) [4] Ericsson, K. Anders, Ralf T. Krampe, and Clemens Tesch-Römer. "The role of deliberate practice in the acquisition of expert performance." Psychological review 100, no. 3 (1993) [5] Ericsson, K. Anders, Robert R. Hoffman, and Aaron Kozbelt, eds. The Cambridge handbook of expertise and expert performance. Cambridge University Press (2018). [6] Goldberg, Benjamin, Kevin Owens, Kevin Gupton, Kevin Hellman, R. Robson, S. Blake-Plock, and M. Hoffman. "Forging competency and proficiency through the synthetic training environment with an experiential learning for readiness strategy." Proceedings of the 2021 I/ITSEC (2021). [7] Bjork, Elizabeth L., and Robert A. Bjork. "Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning." Psychology and the real world: Essays illustrating fundamental contributions to society 2, no. 59-68 (2011). [8] Shabani, Karim, Mohamad Khatib, and Saman Ebadi. "Vygotsky's zone of proximal development: Instructional implications and teachers' professional development." English language teaching 3, no. 4 (2010): 237-248. [9] S. Phillips. The Average Guy Who Spent 6,003 Hours Trying to Be a Professional Golfer. The Atlantic. Retrieved on 25 June 2023 from: https://www.theatlantic.com/health/archive/2017/08/the-dan-plan/536592/, 2017. [10] Mousavinasab, Elham, Nahid Zarifsanaiey, Sharareh R. Niakan Kalhori, Mahnaz Rakhshan, Leila Keikha, and Marjan Ghazi Saeedi. "Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods." Interactive Learning Environments 29, no. 1 (2021): 142-163. [11] Hernandez, Mike, B. Goldberg, R. Robson, K. Owens, S. Blake-Plock, T. Welch, and F. Ray. "Enhancing the Total Learning Architecture for Experiential Learning." In Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). 2022. [12] Robson, Robby, Fritz Ray, Mike Hernandez, Shelly Blake-Plock, Cliff Casey, Will Hoyt, Kevin Owens, Michael Hoffman, and Benjamin Goldberg. "Mining Artificially Generated Data to Estimate Competency." International Educational Data Mining Society (2022). [13] Walcutt, J. J., and Sae Schatz. "Modernizing Learning: Building the Future Learning Ecosystem." Advanced Distributed Learning Initiative (2019).