Simulation in Support of Lifelong Learning Design: A Prospectus David Edgar Kiprop Lelei Gordon McCalla Department of Computer Science Department of Computer Science University of Saskatchewan University of Saskatchewan Saskatoon, Canada Saskatoon, Canada davidedgar.lelei@usask.ca mccalla@cs.usask.ca ABSTRACT intelligence in education (AIED), with its focus on personalization, deep modelling, and innovative pedagogical We argue in this position paper that simulation is an important approaches. There have been many successes in AIED, but, again, tool to support the design of technology to support lifelong mostly in restricted domains and shorter term learning contexts. learning. We discuss various roles that simulation can play in As AIED has begun to venture into lifelong learning, it has faced helping the design of technology for lifelong learning, and then new challenges. Two of these challenges concern the design and present some issues that must be dealt with in building evaluation stages of AIED systems meant to support lifelong simulations in this context along with some preliminary insights learning. Design is costly [9], and it is often impossible to run into how to deal with these issues. We illustrate our discussion closely controlled experiments [10]. using SimDoc, a simulation we have developed of a doctoral Given these design and evaluation challenges, it is important to program, a real world longer-term mentoring environment. find a cheaper, faster, and more flexible approach for evaluating design decisions underlying AIED systems for supporting lifelong KEYWORDS learning. Simulation is a promising approach, analogous to the use Lifelong learning, Simulation, Lifelong learners of wind tunnels to evaluate the aerodynamics of various airplane components [11]. Simulation presents an opportunity to ACM Reference format: experiment with design decisions that explore various hypotheses David Edgar Kiprop Lelei and Gordon McCalla. 2019. Simulation in about learning and pedagogical support in a more cost-effective Support of Lifelong Learning: A Prospectus. In Proceedings of Supporting and faster way than using human learners. Lifelong Learning Workshop (SLLL’19) at AIED 2019, Chicago 2019. Use of simulation within AIED research is not a new idea. In the mid-1990s VanLehn, Ohlsson, and Nason [12] asserted that 1 Supporting Lifelong Learning with Technology technological advances had made it possible to create simulated The ubiquitous nature of technology means the time is ripe to pedagogical agents that could exhibit human-like behavior. They use technology to support lifelong learning [1]. Lifelong learning identified three main uses of simulation in learning environments: has been considered as part of continuing education [2] and adult (i) simulation can provide an environment for human instructors learning [3]. According to Cropley [4] and Bagnall [5], lifelong to practice their teaching methods; (ii) simulation can present an learning happens throughout a person’s life involving all three environment for evaluating different pedagogical instructional kinds of education: formal, non-formal and informal. designs; (iii) simulated learners can act as learning companions Technology can already support lifelong learning. One way is for human learners. Our use of simulation is of type (ii), which via collaborative learning environments aimed at design tasks and has not had nearly as much research as type (iii). However, there information sharing [6]. Another is through mobile technology has been some type (ii) research, including the development of that has the capacity to enable learners to access learning material complex simulation models such as SimStudent [13], simple from any location while at the same time facilitating model simulations such as [14], and medium complexity model communication between learners and their peers or their simulations such as those provided in [15], [16]. instructors (mentors) [7]. Technology has been designed that can help learners of all ages to participate in lifelong learning through 2 Potential Roles of Simulation in Exploring seeking help and social support, following up recommendations System Design Issues about content, scaffolding of learning, and finding mentors. As in the development of AIED systems themselves, most However, most advanced learning technology research and most simulations supporting the design of learning technology are educational institutes’ use of technology to support learning have focused on short learning episodes (measured at most in months) focused on shorter term learning episodes [8]. Such research has and covering well-defined subject matter [14]. There is a lot of led to the development of various systems that are helping knowledge and experience on how these subjects are taught and thousands of learners in numerous learning contexts and domains. learned that can be drawn upon to inform a simulation [13]. It is Perhaps the most ambitious approach to developing therefore often possible to anticipate potential learning outcomes. technology to support learning is in the area of artificial In a lifelong learning context simulation would enable advanced Copyright held by the author(s). Use permitted under the CC-BY license CreativeCommons.org/licenses/by/4.0/ SLLL’19, June, 2019, Chicago, Illinois USA D.E.K. Lelei and G. McCalla learning technology researchers to conduct experiments and shed Finally, simulation enables the capturing of fine grained light on AIED systems that are otherwise impractical to simulation data that allows exploration of many aspects of investigate because of the nature of the environment, in particular learning and teaching by data mining the simulation data for the lengthy time scales involved [17]. Simulation can be used to interesting patterns. Because by definition the simulation is a replicate lifelong learning by modeling a domain’s key simplified model of the complex real world, these patterns can characteristics and behavior over a span of time [15]. Further, often emerge more clearly than in the real world where noise and simulation makes it possible to evaluate AIED systems for the interactions of thousands of variables can obscure important supporting lifelong learning without having to wait a lifetime for relationships. Of course, it must be kept in mind that any results [16]. Such evaluation is crucial in determining the simulation is merely a prediction for the real world and any implications of using a lifelong learning system. patterns found in simulated data eventually need confirmation with actual data gathered in a real life learning scenario. 2.1 Exploring Design Issues When using simulation to explore design issues in building 2.3 Exploring “What If” Scenarios systems to support lifelong learning, system designers need to There are many other factors that affect learning outcomes beyond make decisions as to what system components to include in the system design, pedagogical instructional design, and learning final system design, and they need to explore various options in content. An example is the social interaction aspect of learning. how these components behave [15], [18]. With simulation, Simulation enables experimentation with different pedagogical designers can use simulated learners embedded in a simulated approaches and interaction patterns among learners and teachers version of the desired system, exploring the impact of various to see the impact on learning outcomes. In particular, with components and discovering implications of various design simulation it is possible to explore hypothetical “what if” decisions [17]. Further, designers can examine the impact of scenarios. These hypothetical scenarios can include situations including or excluding certain learner attributes by creating where there is, as yet, no real world data; situations where it simulated learners with various characteristics, and by would take too long to get real world data (a standard feature of manipulating the distribution of the learner population in various lifelong learning contexts); artificial configurations of (simulated) ways [16]. In addition, simulation enables system designers to learners and (simulated) teachers that could never occur in the real inform various system parameter values [14]. world but that bring out various patterns that could be illuminating; learning environments where certain kinds of 2.2 Exploring Evaluation Issues support tools are posited (even if not yet built) to see their effect; The initial exploration of design decisions is a kind of and so on. The ability to create “what if” scenarios is a major formative evaluation in that it involves examining the behaviour advantage that simulation provides. of various versions of an AIED system being developed to detect potential issues and opportunities before actually deploying the 3 Factors That Must be Considered in Using real system. But, it is also a kind of summative evaluation in that Simulation for System Design the outcomes of a particular simulation design are analyzed after Before using simulation to explore questions concerning the simulation runs. The patterns detected then inform decisions in supporting lifelong learning, a system designer must take into the next iteration of simulation design. Real world learning account several important factors, which we will discuss in this environments also use the summative evaluation of one version of section. In section 4 we will then illustrate our discussion with a learning system to inform the next, but the design cycles are lessons drawn from SimDoc, a simulation of a doctoral program, much longer than in a simulation, the factors at play are much designed by the first author in his Ph.D. thesis [19]. We first more numerous and interdependent, and there is often little discuss important issues to consider when designing and building flexibility in what changes can be made in a system design. a lifelong learning simulation model. We then end this section by Simulation thus provides an opportunity to explore formative and providing a description of SimDoc, a model of a longer-term summative evaluation and to find interesting connections between learning environment, the doctoral program. the two. We return to this when exploring simulation lifecycle issues below. 3.1 Simulation Model Fidelity There are other research issues associated with formative Simulation model fidelity refers to the degree of similarity evaluation such as exploring ways of hooking simulated learners between a simulation model and the real world phenomena under to other systems to explore those systems’ functionality. Such study. Different researchers have demonstrated that it is possible systems might be recommender systems, help systems, or to use different levels of model fidelity to gain insight into various mentoring systems. In addition, examining how simulation can pedagogical research issues. While Champaign and Cohen [14] draw from existing learning management systems (LMSs like used a very low fidelity model, Matsuda et al. [13] used a BlackBoard or Moodle), or online courses like MOOCs (which simulation model with high cognitive fidelity to explore the have large amounts student data) is an opportunity to explore how impact of personalized learning experiences. Medium fidelity real world data and simulation data can be mutually informative. simulation modelling has been used in [15] to uncover interesting results. Simulation in Support of Lifelong Learning: A Prospectus SLLL’19, June, 2019, Chicago, Illinois USA A system designer must consider the level of model fidelity choose one source, to compute an average across multiple they want to use before starting the simulation study. This is sources, or to run multiple versions of the same simulation, each particularly important because the level of simulation fidelity with a different source? affects the interpretation of the simulation results. The fidelity should be detailed enough to allow appropriate exploration of the 3.4 Validating a Simulation Model research issues the designer is investigating using the simulation Once calibration has been performed to tune the parameters, it model. is important for the system designer to validate the resulting “best tuned” simulation model. Validation involves checking that a 3.2 Informing a Simulation Model calibrated simulation model’s output and behavior are statistically To successfully use simulation to explore issues concerning like the output and behavior of the real world system under study supporting lifelong learning, a system designer must have clear [21]. This process necessitates prudent experimentation in order to research questions. This requires a system designer to think ascertain that the model works as expected. A single simulation critically about the focus of the research and the design of run is adequate when the simulation is based on a deterministic expected simulation experiments. This will allow the system model. However, when the simulation model contains stochastic designer to more fully understand the structure of the target elements, many runs of the simulation model are needed that yield learning environment. Knowing the questions of interest also outputs that are both relatively stable but also have appropriate allows the designer to decide on which elements of the variability. “Stability” means that over time the average of the environment and learners to model, as well as the level of aggregate outputs of the simulation runs are statistically like the simulation model fidelity. Ultimately, this will direct the system outputs of the real world system. “Appropriate variability” means designer’s search for data to use to inform the simulation model’s that the simulation outputs of the various runs vary enough that attributes, parameters, key assumptions, and algorithms. overfitting hasn’t occurred. A key research issue here is A key issue to focus on here is the availability of data, which determining how many such simulation runs are necessary in can be a big challenge in lifelong learning contexts. A system order to consider a simulation model validated. Another issue is designer needs to consider beforehand if there are data concerning how often should validation be done. We suggest that validation is the phenomena under investigation. Are the data easily necessary whenever a simulation model is revised. accessible? If yes, are the data from a single source or multiple sources? If multiple sources, how do we integrate data from 3.5 Use and Reuse of a Simulation Model different sources? What is the alternative if data is not accessible: Once a system designer has built, calibrated, and validated a can information be derived from known policies or procedures in simulation model, multiple experiments can be run in a relatively the target environment, from related empirical research, from short amount of time. In addition to this advantage of time saving, “commonsense” considerations? If data is available, a system a researcher can create hypothetical experimental setups to designer needs to consider what approaches to use in identifying, explore “what if” questions, as discussed above. collecting, and analyzing the data. After identifying research Once the simulation experiments are finished, the next questions and sources of data, a system designer can then question is, can the simulation be reused to explore new formulate a conceptual model and build the system incrementally questions? Research issues that a researcher needs to address until it behaves like the real world system being modelled. when reusing a simulation concern identifying the exact focus of the reuse. Is it based on improving the simulation model’s fidelity 3.3 Calibrating a Simulation Model level such as adding new data derived from the literature, or Often a simulation model will have parameters whose values further probing of the real-world setting, or adding new data cannot be directly derived from available raw data. There are at derived from insights learned from observing the results of the least two ways to determine the parameter values for the missing first simulation experiments? Is the emphasis on change in the data. One approach is to use commonsense assumptions. Another simulation model itself, which might involve adding or way is to use calibration to systematically derive the values for the subtracting parameters, learner model attributes, or the number of missing data. Calibration is the process of adjusting numerical agent types? Is it necessary to rebuild the simulation from scratch parameters in the computational model for the purpose of or is it possible to build on the existing model? At the very least, improving the match between the simulation output and data from the next iteration of the simulation can draw on lessons gleaned the real world system [20]. While performing calibration, a from the previous iteration(s), but any new simulation, even if system designer can only vary attributes and parameters that are built on an existing model, almost certainly requires performing not yet assigned values from other sources (which is why anew the steps of informing, calibrating, and validating the model. calibration is another way of informing a simulation). Calibration helps build a well-informed simulation model whose behaviour is 4 SimDoc Case Study statistically like the real world to a designer’s desired level of In this section we will discuss how the factors introduced in significance. section 3 played out in the design of our SimDoc simulation of a One open research issue concerning calibration is how a doctoral program [16], [18], [19]. We designed SimDoc to explore system designer handles multiple sources of data: is it better to SLLL’19, June, 2019, Chicago, Illinois USA D.E.K. Lelei and G. McCalla issues in Ph.D. students’ time in program and dropout rates. We by obtaining 10-years’ worth of raw data gathered about student decided on a medium fidelity simulation since we wanted to progress in the University of Saskatchewan doctoral program (the model a number of real world attributes (more than low fidelity “UofS dataset”). We augmented this “baseline” data with models that explore interactions of one or two parameters as in information derived from milestones and policies of the [14]), but we didn’t have access to vast amounts of fine-grained University of Saskatchewan doctoral program and with data (as in [13]) that would have allowed high fidelity modelling. information derived from research studies identifying supervisor Here is a very brief overview of SimDoc’s conceptual model. and student types. SimDoc has five key components: agents, normative rules, We then used system calibration processes (described in 3.3) dialogic rules, events, and scenes based on features for building an that allowed us to determine values of several otherwise electronic institution proposed by Esteva et al. [22] as illustrated unassigned parameters in the model. Specifically, we calibrated in Figure 1. We modeled SimDoc’s entities following the agent- the system to match the progress of students in the baseline UofS based modeling (ABM) [23] technique. Using ABM enables dataset. In our SimDoc simulation the calibration process resulted modelers to capture and represent characteristics of modeled in a tuning of parameters that matched the UofS dataset with 93% elements on an individual basis. confidence. We then moved on to validate this “best matching” model, to determine that it was, over time, statistically consistent with the UofS baseline dataset (it had stability), but also had statistical variability to ensure that it wasn’t overfitted (it had appropriate variability). To do this validation, we devised an algorithm that uses Levene, Chi-Square, and ANOVA tests cumulatively on a set of simulation runs and stops when the appropriate statistical properties of the simulation runs have been fulfilled (see [24] for more details). This algorithm stops after a certain number of runs n, which in our SimDoc model turned out to be 100 runs. We then used this number (100) as the appropriate number of runs that we should make for each simulation experiment, as we explored various supervisor-student interactions as they affected time in program and drop out rates in various “what if” scenarios. We could not do a new validation to determine the number of runs appropriate for each experiment since each “what if” scenario was purely hypothetical where all supervisors were of one type and all students of another (to help expose interesting patterns), not situations with any real world baseline data. In the end, we ran over two dozen experiments. This resulted in over 2400 runs, each a small mini-experiment, that shed interesting light on supervisor- Figure 1. SimDoc’s Conceptual Framework student interactions and their effect on time in program and In SimDoc, we modeled two types of agents representing dropout rates. For extensive details on these experiments (and all students and faculty supervisors. Each of these agent types plays other aspects of SimDoc) see [19]. different roles within SimDoc. We use the normative model to capture the complex characteristics of a doctoral program that 5 Future Research Directions result from various interactions that happen between different An open issue associated with using simulation to understand doctoral program elements. These normative rules inform the system designs for supporting lifelong learning is identifying the behaviour and evaluation functions at an appropriate level of contexts where the use of simulation is advantageous. What fidelity for the research issues we wish to investigate. The learning environment features are favorable for the use of dialogic model represents the interaction strategies used within simulation? One important aspect is the availability of some SimDoc. We used the notion of a scene to capture a single baseline data to inform the simulation. Another seems to be interaction that happens between two agents (such as a supervisor having relatively unambiguous questions to be answered. What and student). To capture the various events that take place within other factors are important? the doctoral program (e.g. taking comprehensive exams or Another issue is to determine when a simulation is accurate defending thesis proposals), we used an event model. These enough to be believed. We have identified two features: stability events trigger action and reactions by agents. and appropriate variability of the outputs from run to run of the In SimDoc we wanted to investigate ways of improving simulation. But, there must be other factors too. The ultimate student outcomes (shorter time in program, fewer dropouts) ‘reality check’ is that designs of lifelong learning support tools through exploring a variety of supervisor-student mentoring arising from simulation studies actually work in the real world. relationships. We therefore informed SimDoc appropriately first Simulation in Support of Lifelong Learning: A Prospectus SLLL’19, June, 2019, Chicago, Illinois USA Ultimately, a simulation is not a one-off. In most design [11] G. I. McCalla and J. Champaign, “Simulated Learners,” IEEE Intelligent Systems, vol. 28, no. 4, pp. 67–71, 2013. scenarios there will be an iterative design/experiment cycle with [12] K. VanLehn, S. Ohlsson, and R. Nason, “Applications of Simulated simulation in the loop. The initial iteration is instrumental in Students: An Exploration,” Journal of Artificial Intelligence in Education, vol. 5, no. 2, pp. 1–42, 1994. determining what aspects of the model to focus on and probably [13] K. R. Koedinger, N. Matsuda, C. J. MacLellan, and E. A. McLaughlin, what facets to discard. It is hoped that over successive generations “Methods for Evaluating Simulated Learners: Examples from of simulation design (with possible real world spin off SimStudent,” in The Proceedings of the 2nd Workshop on Simulated Learners at the 17th International Conference on Artificial Intelligence in applications along the way), the simulation can move gradually Education, 2015, vol. 5, pp. 45–54. from medium to high fidelity as data is gathered during each cycle [14] J. Champaign and R. Cohen, “A Multiagent, Ecological Approach to Content Sequencing,” in The Proceedings of the 9th International and new capabilities are integrated into the simulation model. Conference on Autonomous Agents and Multiagent Systems, 2010, pp. Moreover, going forward there should an increasing number of 10–14. datasets gathered over longer-term use (in MOOCs, in forums [15] G. Erickson, S. Frost, S. Bateman, and G. I. McCalla, “Using the Ecological Approach to Create Simulations of Learning Environments,” such as stack overflow and in other online learning contexts) that in The Proceedings of the 16th International Conference on Artificial would inform a simulation model and provide baseline data for Intelligence in Education, 2013, pp. 411–420. [16] D. E. K. Lelei and G. I. McCalla, “How to Use Simulation in the Design calibration and validation. Exploring how a simulation model and Evaluation of Learning Environments with Self-directed Longer- evolves through many cycles is an important direction for Term Learners,” in The Proceedings of the 19th International Conference on Artificial Intelligence in Education, 2018, pp. 253–266. simulation research. [17] F. Dorça, “Implementation and use of Simulated Students for Test and In conclusion our overall proposition is that simulation should Validation of new Adaptive Educational Systems: a Practical Insight,” be an essential tool in a lifelong learning system designer’s International Journal of Artificial Intelligence in Education, vol. 25, no. 3, pp. 319–345, 2015. toolkit. Simulation can be used to explore various aspects of [18] D. E. K. Lelei and G. I. McCalla, “The Role of Simulation in the learners’ knowledge acquisition and development, the effects of Development of Mentoring Technology to Support Longer-Term Learning,” in The Proceedings of 3rd International Workshop on possible changes in a learning system’s design, the implications of Intelligent Mentoring Systems, held in Conjunction with the 19th various types of learner interactions with different people and International Conference on Artificial Intelligence in Education, London, learning environments as they go through life, even hypothetical 2018. [19] D. E. K. Lelei, “The Role of Simulation in Supporting Longer-Term “what if” scenarios that cannot exist in the real world but Learning and Mentoring with Technology,” Ph.D, thesis, Department of nevertheless shed light on issues in lifelong learning. Computer Science, University of Saskatchewan, 2019. [20] D. S. A Vrijdag and T. van Terwisga, “Systematic modelling , verification , calibration and validation of a ship propulsion simulation ACKNOWLEDGMENTS model,” Journal of Marine Engineering & Technology, vol. 8, no. 3, pp. 3–20, 2014. We would like to thank the University of Saskatchewan Data [21] B. L. Heath and R. R. Hill, “Final Report Developing an Agent-Based Warehouse team for allowing us to access their dataset, and the Modeling Verication and Validation Approach for Improving Air Force Analytical Support,” Dayton, OH, 2010. Natural Sciences and Engineering Research Council of Canada [22] M. Esteva, J.-A. Rodriquez-Aguilar, C. Sierra, P. Garcia, and J. L. Arcos, (NSERC) for funding our research. “On the Formal Specification of Electronic Institutions,” in The Proceedings of the Agent Mediated Eectronic Commerce Conference, 2001, pp. 126–147. REFERENCES [23] N. Gilbert, Agent-Based Models. SAGE, 2008. [1] S. de Freitas et al., “The Development of a System for Supporting the [24] D. E. K. Lelei and G. I. McCalla, “How Many Times Should a Lifelong Learner,” British Journal of Educational Technology, vol. 37, Pedagogical Agent Simulation Model be Run?,” in to appear in The no. 6, pp. 867–880, Nov. 2006. Proceedings of 20th International Conference on Artificial Intelligence in [2] D. M. Gaymer, “Continuing Education and Lifelong Learning Trends,” Education, Chicago, 2019. Encyclopedia of Management. MM., Helms (Ed.), pp. 128–131, 2006. [3] Canadian Council on Learning, “State of Learning in Canada: Toward a Learning Future,” Canadian Council on Learning, Ottawa, Ontario, 2008. [4] A. J. Cropley, “Some Guidelines for the Reform of School Curricula in the Perspective of Lifelong Education,” International Review of Education, vol. 24, no. 1, pp. 21–33, 1978. [5] R. G. Bagnall, “Lifelong Learning as an Idea for Our Times: An Extended Review,” International Journal of Lifelong Education, vol. 28, no. 2, pp. 277–285, Mar. 2009. [6] G. Stahl, “Collaborative Information Environments to Support Knowledge Construction by Communities,” AI & Society, Journal of Knowledge, Culture and Communication, vol. 14, no. 1, pp. 71–97, 2000. [7] M. Sharples, “The Design of Personal Mobile Technologies for Lifelong Learning,” Computers & Education, vol. 34, no. 3–4, pp. 177–193, Apr. 2000. [8] M. Kalz, “Lifelong Learning and its Support with New Technologies,” International Encyclopedia of the Social & Behavioral Sciences. Oxford: Pergamon, 2014. [9] N. T. Heffernan and C. L. Heffernan, “The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching,” International Journal of Artificial Intelligence in Education, vol. 24, no. 4, pp. 470–497, 2014. [10] N. Matsuda et al., “Studying the Effect of Tutor Learning Using a Teachable Agent that Asks the Student Tutor for Explanations,” in The Proceedings of the 4th International Conference on Digital Game and Intelligent Toy Enhanced Learning, 2012, pp. 25–32.