Adaptive scafolding of students' planning activities shows significant potential for helping students regulate their learning in challenging tasks. Large language modeling provides new capabilities for augmenting such scafolds in real-time game-based learning environments. We introduce a novel plan generation framework that leverages a text-representation of students' interactions in a game-based learning environmentC,rystal Island. We formalize the plan generation task as follows: given a sequence of gameplay events, completed goals, and target goals, a language model trained on T5-small will generate a set of low-level actions to accomplish the given target goals. Gameplay data collected from 144 middle school students is used as input for the model, with 11,610 total event sequences. The generated plans are then evaluated against plans students created during gameplay through a planning support tool. The construction of this framework is designed so it will mimic a real-time system, in that it generates plans at the same point in gameplay that students do. We compare generated plans to students' plans based on how many low-level actions in the plans match. We also analyze how many actions match when mapped to high-level actions or action categories. Results show that the generated plans did largely match high-level actions with fewer low-level actions than the students' ones. This implies that the generated plan can guide the students to achieve their goals in more eficient ways. These results demonstrate potential for using language models for enhancing adaptive environments through hinting or prompting to plan eficiently in real-time.
Self-regulated learning (SRL) in Winne and Hadwin’s theoretical model is goal-driven learning
that involves formulating goals, developing plans, and monitoring and adapting their goals and
plans [
With the recent developments in large language models (LLMs), there are new possibilities for data representations using trace data obtained from game-based learning environments as natural language to help support such SRL processes. Generally, LLMs are pretrained with a large corpus of text data to enable general language understanding and generation, while ifne-tuning LLMs with domain-specific data allows to solve the problems important to the target domain application. This functionality creates many opportunities for educational applications, particularly in generating assistive educational content for students like practice problems, step-by-step solutions, and explanatio6n].s O[ur work investigates the application of language models for generating plans meant to assist in students SRL processes and science problemsolving.
This work explores the use of a language model (T5) to generate potential plans for students as they navigate a science game-based learning environmCernyts,tal Island. A plan is defined as a series of low-level actions that can be enacted in gameplay. Textual representations of students’ problem-solving activities in the game serve as input into the language model, as well as student selected goals. The language model outputs a plan, which is a series of low-level actions, that can be completed in the game. We evaluate the generated plans against plans created by students while playing the game. Our work aims to understand the efectiveness of using language models to generate plans in a game-based learning environment and discuss the implications for enhancing students’ learning in real-time.
Classical planning in AI involves the generation of action sequences in a given environment,
which we extend to the task of plan generation in this w7o]r.kTe[chniques for plan generation
have been explored for several years in the field of automated plann8i,n9g,1[0]. More recent
approaches to plan generation involve hierarchical plan1n1i]n, ga n[d narrative plan generation
with self-supervised learnin1g2[]. These techniques can be applied to educational settings
where the environment is finite and small. For example, graph networks are used to evaluate
and enhance student planning in Betty’s Br1a3i]na[nd have also been used for study plan
generation1[
To generate useful plans for students in real-time, an understanding of their underlying
strategies is needed. However, students’ gameplay in game-based learning environments is
incredibly idiosyncratic and exploratory. Recently the natural language processing (NLP) field
has had much success through the development of large language models, such as tasks for
summarization (BART1[
Prior work inCrystal Island has explored the task of plan and goal recognition, which is a classification problem for predicting students’ goals and plans based on their gam1e9p,l2a0y]. [ Long-short term memory networks were shown to have performed best for the constructed multi-task classification problem, with students’ problem-solving activities being represented through one-hot encoding vector2s1[, 22]. To our knowledge, language models have not been used to represent students’ problem-solving activities for plan generation tasks. Through textual representations of the trace log data and textual output of plans that can be enacted in the game, language models like T523[] have the potential to generate more interpretable and generalizable plans.
Crystal Island is a narrative game-based learning environment that teaches eighth grade microbiology concepts. Students are tasked with exploring a remote island to discover the source and treatment plan of a mysterious illness that has plagued the island. In this version of the game, students are presented with a planning support tool, meant to scafold students’ planning processes and help them navigate the narrative. Throughout the game, students are prompted to create plans through a drag-and-drop interface (1F)i.gIunrtehis work, we define plans to be goal clamps (green) with at least one nested low-level action (blue) inside the goal clamp. Students are able to access the planning support tool at any point in the game voluntarily as well. There were 20 goals and 55 low-level actions to choose from in the planning support tool. These goals and actions are categorized into high-level goals and actions.
This work utilizes a dataset collected during the COVID-19 pandemic in a remote asynchronous science classroom. A total of 144 middle school students (60% female, average age 13.2 years) played theCrystal Island version containing the planning support tool over a two-day time span. Students playeCdrystal Island for an average of 94.7 minutes (SD = 47.7). They also completed a demographic survey, as well as pre- and post-tests. Students’ problem-solving activities in the game and their planning support tool use were logged automatically as they played. The trace logs generated from these interactions were used in the analysis presented in this work and students’ plans were used as a source for evaluating the generated plans.
In this section, we present a plan generation framework that takes in students’ gameplay interactions and a goal as input. Using a fine-tuned LLM, plans are generated from the framework which consists of a series of low-level actions that can be enacted in the game. These plans are then evaluated against plans students created using the planning support tool.
We refer to a sequence of students’ interactions wCrityhstal Island as an event sequence. These are logged automatically while each student is playing the game. To construct an input representation, we derived three key features from the event sequences: event type, event argument, and event location. The event type refers to 9 possible in-game activities students can complete within the learning environment. For example, reading a book and talking to a character are two types of events in the game. Event arguments are phrases that provide more detail to the event type. For example, if the event type is talking to a non-playable character, then the event argument could be the name of the character. Lastly, the event location is the location in the game where the event took place. There are 24 fine-grained locations in the game. The given example of event can be denoted a=s (, , ) .
Once these three components were constructed for each event a student took, we segmented event sequences for each student according to their use of the planning support tool. Formally, we represent the inp ut∶ [ = ( , +1 , ..., + ), ∶ , ∶ ] where is an event sequence, consisting of a series of event s, indicates a set of completed goals, which can be an empty set, an d is the target goals the student set during planning. The aim of this framework is to generate potential plans for students, so we constructed event sequences based on the potential for real-time use. In a real-time system, the input to any model would be the action students took up to the point of prediction. A typical interactCiroynstinal Island would be a sequence of gameplay events (1), then a planning tool interaction where they create one or more plans with a target go al=( {Explore camp}), then another series of even ts2() and then another planning tool interaction for the nex t go=al{L(earn about outbreak}) and so on until they finish playing. A students’ plan consists of a goal and a series of low-level actions. Since students interactions with the planning support tool were dispersed throughout gameplay, they serve as good points for real-time prediction of the next plan a student would make. Thus, we used the sequence of student events completed up to the point of a planning tool usage and predict possible plans at every event in the sequence.
For example, given the typical interaction wCirtyhstal Island, then the first row of input would be = [ 1, = {Explore camp}, = {}], since there were no completed goals. Then, in the case that a student enacted all low-level actions from plan 12dtuhreingthe next row of input would b e = [ 2, = {Learn about outbreak}, = {Explore camp}]. We constructed event sequences cumulatively with the maximum event sequence length being 30 (the median length of event sequences).
In this work, we leverage a language model to understand the student’s game trace logs as text and generate the corresponding plans to achieve target goals. In the first part, the input is not a general natural language, but consists of a list of actions and location information used in theCrystal Island game domain, and thus an encoder is needed to understand the contextual representation of these characteristic inputs. In the second part, the generated plan can include actions of various lengths, and this problem is one of language generation rather than classification, so a decoder capable of language generation is required. That is, for our task, we need a Seq2Seq model with both an encoder and a decod2e4r].[Various Seq2Seq language models currently exist, such as Transform2e5r] a[nd BART [15]. We chose to use T5 for this task, as T5 has the advantage of being easy to use by formatting both input and output into textual strings for various NLP task2s3[]. We trained T5-small for our plan generation models with the following hyperparameters: learning rate = 0.0003, batch size = 4, input max_length=1024, weight_decay=0.01, and warmup=1000.
We evaluated the language-model generated plans from the framework with the plans students created. The key evaluation metric is the percentage of actions that match between the student’s plans (SP) and the generated plans (GP), with a higher percentage being better. We considered three cases for our evaluation: (1=) : the case where the amount of low-level actions in the generated plans was the same as the students pl an<, (2) : the case where the generated plan had more low-level actions than the student plan, a>n d (3): the case where the students’ plan had more actions than the generated plan. For each of these cases, we examine how many true matches occur for each GP. A true match is when a predicted low-level action from a GP is present in the SP. For each generated plan, we counted the number of low-level actions that were a true match and divided that by the number of low-level actions in the GP, denoted as . Using the high-level action mappings derived from the planning support tool, we derived a similar score based on how many high-level actions from the GP are present in the high-level SP, denoted a s . For example, take the “explore camp” plan in Fig1u.re The low-level actions in this case are the four “go to” actions nested inside the goal. For our analysis, we would examine true matches between these and the GP low-level actions. Then we would map these low-level actions to the high-level actions and cal cu,lwatheich in this case would all be “explore”. This analysis helps to better understand the quality of plan generation and highlights potential for future work in enhancing students’ planning processes, further discussed in Section6.
Figure 2 shows the distribution of diference in plan size between students’ plans and the generated plans (left), as well as the distribution of match percentages for both low (center) and high-level actions (right). In the left boxplot, the median of the diference in plan size is 0.0 (mean=0.0, SD=3.4). A negative plan size represents the scenario where students plans were larger. The largest diference in plan size was 26, meaning students included 26 more low-level actions in their plan than the generated plan. This occurrence serves as an example of when the language models could help students build plans more eficiently. In the center boxplot, the median of match percentage in low-level action)s i(s 0.0% (mean=25.2%, SD=32.3%).
Out of 11,610 generated plan instances, 8.1% of these plans contained all low-level actions that were in the students’ plans (exact matches) and 51.2% had no low-level actions that matched the low-level actions of the corresponding student plan. In the right boxplot, the median of match percentage in high-level action s () is 66.7% (mean=59.8% , SD=41.9%). For the high-level actions, we saw an increase in exact matches to 44.4% and a decrease in no matches to 24.8%. This indicates that while the GP did not match students’ low-level actions as much, the generated plans still produced low-level actions in the same category as the student.
This is also demonstrated in Tab1lfeor both low-level and high-level action matching, as we see an increase in match percentage for all cases. The first case where the generated plan and the student plan were the same size=(SGPP) had the highest percentage of low-level matching. This set of generated plans could be seen as most accurate as they were the closest to students’ plans. The second case with the biggest increase in match percentage was the case where the size of the students plan was larger than the generated p>laGnP()S.PThis indicates that though the students plan was more detailed, the generated plan had fewer low-level actions but in the same high-level category. This could potentially be a more eficient generated plan, which in real-time could help the student maneuver the game faster or learn more educational content. The third case where the student plan size was less than the generated plan size (SP < GP) had a much lower match percentage for both the low and high-level actions. This case needs to be explored further to better understand why the model was producing more actions and if they are viable actions for the goal. For example, a closer analysis will be needed, such as whether there is any correlation between the type of goal and the size of the plan, or whether a larger plan focuses on the efectiveness to achieve a goal rather than focusing on the eficiency.
Overall, results demonstrate the promise of using language models to generate plans to prompt students’ gameplay. We find that over half of the low-level actions generated by the framework align with the same high-level action category that students’ used for a given plan. This indicates that even in cases where the exact matches were low, plans still contained low-level actions within the same category. Actions within the same category generally serve a similar purpose in the narrative of the game. For example,Etxhpelore goal category contains primarily “go to” actions, which all serve as an exploration of the environment. An example plan from a student included: go to infirmary, go to lead scientist’s quarters, go to laboratory, go to living quarters, speak with lab technician, read about smallpox, speak with virus expert. The plan predicted in this case was: go to infirmary, go to lead scientist’s quarters, go to laboratory, go to living quarters, read about how diseases spread, speak with camp nurse, and speak with patients. In this case, all of the ”go to” actions align and the ”read” and ”speak with” actions difer. All action categories match between the two. Further analysis can be done to examine the specific diferences in predicted plans. For example, one of the diferences in the predicted plan from the student plan is the action speak with camp nurse. Generally, it is good to speak with the camp nurse early in the game, so prompting a student to do so could be beneficial to their gameplay. Anecdotally, we observed that some unmatched low-level action plans were found in the other students’ plans with the same target goals. This could be because the model generates most probable plans rather than the one personalized to a specific student. If these predictions were implemented in a real-time scenario, they could serve as more instructive hints to students or potentially more eficient plans, allowing the game to dynamically adapt to students’ strategies.
A limitation of this work is that the ground truths of optimal plans are unknown or dificult to obtain. In this work, we take the students’ plans as ground truth; however, we noticed a large range of planning activities. Students’ use of the planning support tool varied between students for how often they opened the planning and how many plans they created throughout gameplay. While our language models seemed to generate plans similar to students’, since we cannot evaluate the quality of plans it is hard to tell if the generated plans are “better” plans than what the student created. Additionally, there are several strategies that students could take in the game, which adds complexity to plan evaluation. In educational games, there are typically two overarching strategies: to win the game and to learn the curricular content. Depending on a students’ primary goal, this could afect how an adaptive system would prompt students. To partially address this issue, we used an existing hierarchy to map low-level actions to high-level actions as a point for evaluation. One potential avenue for future work could be to create a larger hierarchy of actions that align with science problem solving to assess if plans follow a trajectory of scientific reasoning. This hierarchy could help evaluate both student and system generated plans.
Another interesting finding was that the language model sometimes produced low-level actions that can be played in the game, but are not present for students to select in the planning support tool. For example, students are able to pick up items in the game to test for various diseases. An output low level action was “pick up yogurt”, which is an item that can be picked up in the game but it is not presented as an option for students. In the design of the planning tool, we limited the number of low-level actions presented to students so as not to overwhelm them. Thus, “pick up yogurt” was intentionally excluded, since it is not necessary to solve the mystery. However, it is interesting that the model was able to generate coherent low-level actions. This case identifies another potential area of using LLMs to generate diverse content for SRL scafolding. The planning support tool was human designed, but considerable human efort could be reduced to have goals and actions created by LLMs trained on gameplay data. This model-based content generation could be extended to other online-based learning environments, where trace logs can have textual representations.
This work introduces a framework that utilizes textual-representations of students’ problemsolving actions withiCnrystal Island as input for a T5-small-based language model to generate plans that are evaluated against real-world plans created by students. Results show the potential for using such language models for providing adaptive support to students in real-time through generated plans. Currently, the framework is able to produce plans similar to plans that students created, with half of actions in generated plans aligning with the same action category as students’ planned actions.
These findings highlight future work in many areas. Plan validation for science problemsolving in game-based learning environments is an open problem that requires investigation. More work needs to be done to interpret the quality of both student and system generated plans. Exploring the performance of other LLMs in this framework is another potential avenue for future work. Additionally, applying a scientific reasoning hierarchical-based approach to the language models has the potential to both help evaluate plans and enhance the generation of plans. Lastly, identifying how LLMs could enhance existing scafolding in a real-time version of Crystal Island has the potential to enhance students’ learning experiences.
This research was supported by funding from the National Science Foundation under grant DUE-1761178. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. regulated learning behaviours in open-ended learning environments, Journal of Learning Analytics 2 (2015) 13–48. [14] E. W. C. Leung, Q. Li, A dynamic conceptual network mechanism for personalized study plan generation, in: Advances in Web-Based Learning-ICWL 2003: Second International Conference, Melbourne, Australia, August 18-20, 2003. Proceedings 2, Springer, 2003, pp. 69–80. [15] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer, BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 7871–7880. URLh:ttps://aclanthology.org/2020.acl-main..703 doi:10.18653/v1/2020.acl-main.703. [16] M. Chen, J. Tworek, H. Jun, et al., Evaluating large language models trained on code, CoRR abs/2107.03374 (2021). URL: https://arxiv.org/abs/2107.033.7a4rXiv:2107.03374. [17] T. B. Brown, B. Mann, N. Ryder, et al., Language models are few-shot learners, CoRR abs/2005.14165 (2020). URL: https://arxiv.org/abs/2005.141.6a5rXiv:2005.14165. [18] R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H. Cheng, A. Jin, et al., Lamda: Language models for dialog applications, CoRR abs/2201.08239 (2022). URL: https://arxiv.org/abs/2201.082.3a9rXiv:2201.08239. [19] W. Min, B. Mott, J. Rowe, B. Liu, J. Lester, Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence(IJCAI-16), , New York, 2016, pp. 2590–2596. [20] W. Min, B. Mott, J. Rowe, R. Taylor, E. Wiebe, K. Boyer, J. Lester, Multimodal goal recognition in open-world digital games, in: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-17), 2017, pp. 80–86. doi:10.1609/aiide.v13i1.12939. [21] A. Goslen, D. Carpenter, J. Rowe, N. Henderson, R. Azevedo, J. Lester, Leveraging Student Goal Setting for Real-Time Plan Recognition in Game-Based Learning, in: Proceedings of the Twenty-Third International Conference on Artificial Intelligence in Education (AIED-22), , 2022, pp. 78–89. [22] A. Goslen, D. Carpenter, J. Rowe, R. Azevedo, J. Lester, Robust Player Plan Recognition in Digital Games with Multi-Task Multi-Label Learning, in: Proceedings of the 18th AAAI Conference on AIIDE , , 2022, pp. 105–112. [23] C. Rafel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, JMLR 4 (2020). [24] I. Sutskever, O. Vinyals, Q. V. Le, Sequence to sequence learning with neural networks,
CoRR abs/1409.3215 (2014). URL: http://arxiv.org/abs/1409.321.5arXiv:1409.3215. [25] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31th Conference on Neural Information Processing SystemsNeurIPS, 2017.