What’s a Textbook? Envisioning the 21st Century K-12 Text Steven Ritter, Josh Fisher, Amy Lewis, Sandy Bartle Finocchi, Robert G.M. Haus- mann, and Stephen E. Fancsali Carnegie Learning, Inc., 501 Grant Street, Suite 1075, Pittsburgh, PA 15219, USA {sritter, jfisher, alewis, sandy, bhausmann, sfancsali}@carnegielearning.com Abstract. We present a vision for 21st century digital textbooks, reflecting on Carnegie Learning’s experience as a K-12 curriculum and service provider. Af- ter briefly considering the purpose of a textbook, we introduce components of Carnegie Learning’s blended mathematics curricula and consider the inherent tension between providing personalized, individual learning while also provid- ing opportunities for collaborative learning. We then describe a vision for an Adaptive Classroom Environment and a supporting Adaptive Learning Rec- ommendation System to deliver improved learning outcomes based on 21st cen- tury digital technologies. While Carnegie Learning has proto-typed facets of both the Adaptive Classroom Environment and Adaptive Learning Recommen- dation System, this area is ripe for exciting contributions and progress from the artificial intelligence in education (AIED) and related communities of research- ers, developers, and practitioners. Keywords: Mathematics Education, Intelligent Textbooks, Intelligent Tutoring Systems, Adaptive Learning, Adaptive Teaching, Collaborative Learning, Blended Instruction. 1 Introduction Our vision of an intelligent textbook starts with a consideration of the purpose of a textbook. Carnegie Learning’s primary experience has been with middle- and high- school math textbooks in the United States, so our observations reflect that scope. A variety of educational stakeholders often believe that the purpose of a textbook is to provide a coherent narrative about the topic, as if students read the text cover-to-cover in order to learn. This view is often reflected in naive hopes that free and/or open- source textbooks will revolutionize education by providing quality instruction to stu- dents who could not otherwise afford it (see, for example, [1]). However, in most cases, and in our experience, the educational environment(s) in which textbooks are used drive their form and content. While a complete and coherent narrative might 2 serve a student seeking to learn Algebra independently, it is the wrong form of prod- uct for students who are learning in a modern American classroom.1 Textbooks in current classrooms serve as guides to the teacher, school administra- tors, and students. A primary function of these guides is to present a scope and se- quence: the coverage and progression of all the academic standards required of the course. The text structures instructional time (both in-class and homework) so that these standards can be covered in the time allocated to the class. Current textbooks also recognize that not all students are alike. They provide recommendations and resources to differentiate instruction for remedial students, advanced students, English language learners, and learners with other needs. They provide additional resources and recommendations to teachers to allow flexible implementation, accounting for teachers who are able to assign more or less homework or whose students may have more or less access to technology, for example. While most textbooks provide do- main-specific examples and definitions, it is not the case–and has never been the case–that textbooks are used as exhaustive encyclopedias of grade-appropriate do- main-specific knowledge. We present here a vision of an intelligent, 21st century textbook that serves a re- source that helps structure the activities of students in a classroom. Key considera- tions for this textbook include balancing the inherently contradictory desire for per- sonalization with the desire for collaboration and considering the appropriate role of the teacher in classroom instruction. We start with a description of our current curric- ula. 2 Carnegie Learning’s Blended Curricula Since its start as a research project at Carnegie Mellon University, Carnegie Learn- ing’s core products have been blended curricula, combining text-based activities with intelligent tutoring software [3]. In the recommended implementation, classes spend 60% of classroom time working with the text activities and 40% of the time working with the software. These two instructional modes were intended to be complementary, with differences in the two modes driven by technical limitations of the software and affordances of the classroom environment. For example, the MATHia software (for- merly called Cognitive Tutor) was focused on individual problem solving and less with collaboration and communication, due to the difficulty of understanding natural language and of scaffolding collaboration in software. 2.1 MATHia MATHia is an intelligent tutoring system that monitors student problem solving step- by-step [4, 5]. Following from ACT-R model of cognition, complex task performance is modeled as the interaction of simple knowledge components, which MATHia dis- plays to teachers and students in its “skillometer.” In order to develop mastery, each 1 See [2] for a description of the history of the Algebra textbook and how it has changed in response to changes in educational systems and pedagogy. 3 knowledge component must be encountered multiple times, and the amount of prac- tice required to reach mastery of each component is determined through Bayesian knowledge tracing [6]. Within a section of the curriculum, tasks are picked for stu- dents based on a match with the particular knowledge components that the student still needs to master. Students must demonstrate mastery of all knowledge compo- nents identified for a particular topic before they can progress to the next topic in the curriculum. When multiple strategies are available for solving a particular problem, MATHia uses a process called “model tracing” to identify the particular strategy that a student is pursuing, and the system is able to provide hints relevant to the student specific to the individual student’s approach. 2.2 Text and Classroom Activities Our recommended classroom environment is designed to encourage active dialog, centered on structured activities. One particularly effective structure for collaboration is “think-pair-share” [7], in which students think about the goals and initial approach to solving the problem, pair up for interactive problem-solving, and then share their results with the rest of the class. To encourage active engagement, our text is consum- able: students can write directly in the book. The text includes whole-class, group and individual activities, in addition to worked examples and other instructional elements. Collaborative problem solving encourages an interactive instructional mode, but, as a practical matter, it can be difficult to make classroom collaboration work. Chi et al. [8] show that, in some cases, simply allowing students to work collaboratively can encourage sufficient depth of collaboration. However, structured collaboration, in- cluding scripted dialogs [9, 10] and routine prompts [11] are often recommended to ensure that collaborators encourage each other to engage deeply with the content. 4 Fig. 1. MATHia problem in which students construct a diagram and use it to answer questions about the relationship between two quantities (© 2019 Carnegie Learning, Inc.) 3 Personalization and Collaboration Our approach to blended instruction involves components where students learn indi- vidually and where such students receive personalized activities appropriate to their level of knowledge along with components where students are expected to collaborate with other students, including articulating their own reasoning and critiquing the rea- soning of other students. From the teacher’s perspective, managing a classroom in which students are progressing at their own pace is difficult [12], and supporting good student collaboration is difficult. Orchestrating a classroom where both instructional modes are expected is even more difficult. 5 Fig. 2. Activity from Carnegie Learning’s eighth grade text asking students to com- pare solution methods (© 2019 Carnegie Learning, Inc.) 4 The Adaptive Classroom Environment (ACE) We believe 21st century solutions will address the above difficulties by using data from both individual and collaborative activities to help teachers make instructional decisions that meet the needs of all learners. We call such a system an Adaptive Classroom Environment (ACE). ACE takes the form of a fully digital curriculum that presents, scaffolds, and observes both individual student activities and collaborative activities. It incorporates an Adaptive Learning Recommendation System (ALRS), which provides recommendations on how to select, adapt, and implement instruction- 6 al activities in a way that optimizes student learning, taking into account the af- fordances and constraints of the classroom and school. Such a tool focuses on personalizing instruction in the classroom context. Its role is to serve as a source of reference material, but also to structure the work of students and of classes. This kind of tool differs from an e-text is several important ways: • In the digital realm, there is no limit on “pages” or “lessons” that may be available. ACE can contain multiple approaches to a particular lesson, any of which might be presented to an individual student (or collaborating group of students), depending on that student’s profile and classroom con- text. However, as the amount of content and the complexity of selecting appropriate content for particular students or classes increases, the need for assistance in selecting appropriate content also increases. • ACE collects fine-grained data about the use and success of every activity presented to students, whether completed individually or as part of a group. Such data can be used to dynamically adapt ACE, both in terms of content presented and mode of presentation. • ACE can be both time- and location-aware in a way that non-digital tools cannot. ACE can know the affordances of classroom vs. homework activi- ties (i.e., that a teacher may be available for assistance in the classroom) and use that information to present the same content differently. Similarly, ACE knows about how much instructional time is available before various milestones (like important exams or the end of the course) and so can help to optimally adapt and schedule activities for students and classes. • ACE can learn about institutional, teacher, or contextual preferences and use this information to advise on modality of presentation. For example, ACE may understand the conditions (including aspects of classroom dy- namics and teacher preferences) under which a particular topic might be especially successful when done in a think-pair-share approach. • ACE supports multiple input modes (e.g., drawing, keyboarding, photos). It includes content from both MATHia and activities currently included in Carnegie Learning’s texts. In addition to this content, ACE includes sup- ports for classroom activities, including tools that help teachers form stu- dent groups, display and compare examples of student work and provide electronic feedback to students. Activities involve a mix of those that are auto graded and those that are open-response. Open-response items, in particular, are very flexible in their implementation. Some teachers will assign them for homework, others might use them as whole-class activi- ties, illustrating a solution at the board to the whole class, and still others might use a particular activity in a think-pair-share format. Teachers have the option to assign grades to open-ended work, but this is up to them. • Of particular interest to us is the way that ACE uses data to help teachers transition between personalized and collaborative classroom activities. MATHia provides fine-grained information about student knowledge and achievement. These data can be used to form collaborative pairs. For ex- ample, we might group students who have, individually, demonstrated dif- 7 ferent problem-solving approaches in a domain. Alternatively, we might group students who have demonstrated particular misconceptions togeth- er. Similarly, data from student work within collaborative activities could lead to individualized remediation or acceleration. 5 The Adaptive Learning Recommendation System (ALRS) One advantage of a digital resource over a physical one is that there is little cost to adding additional content. The challenge, then, is to provision the content to students (and groups of students) effectively. Our current MATHia software provisions prob- lems to students based on an estimate of the student’s mastery of a number of under- lying knowledge components. The provisioning is entirely determined by the AI sys- tem. In contrast, in a traditional teacher-led classroom, teachers provision content to students. In some personalized learning systems, students have varying levels of agency in choosing activities to work on, in effect provisioning resources to them- selves [13]. The ALRS within ACE can support any of these provisioning methods, in some cases, combining them (for example, picking the mathematics topic for the stu- dent, but allowing the student to choose specific word problems that interest them). Most importantly, ALRS has awareness of when different provisioning methods are most likely to be effective. For example, decisions that require large amounts of data, relying on approaches including multi-arm and context bandits to discover and im- plement effective provisioning strategies [14, 15], might be best made by the AI; de- cisions that rest on social interactions (like whether two students get along well) might best be made by teachers or students. 6 Conclusion We believe that the right way to think about the future textbook is to think of it less as a book than as a repository of activities, along with an intelligent recommendation system that provisions these activities to students. We emphasize two important con- siderations of such a system. First, it must balance the desire to personalize instruction for individual students with the need to support student collaboration. Second, it must be flexible in provisioning activities, including whole-class activities. Ultimately, such a system needs to serve the needs of real teachers in real classrooms. Pedagogi- cal decision making is extraordinarily complex [16]; we are at the point where our ability to instrument instruction, analyze data, and provide strong recommendations for classroom orchestration can lead to substantial improvements in student outcomes. 8 References 1. Vance, A.: $200 Textbook vs. free. You do the math. The New York Times (July 31), https://www.nytimes.com/2010/08/01/technology/01ping.html (2010). last accessed 2019/05/10. 2. Heeffer, A.: The genesis of the algebra textbook: From Pacioli to Euler. Almagest: Interna- tional Journal for the History of Scientific Ideas 3(1), 26–61 (2012). 3. Ritter, S., Anderson, J.R., Koedinger, K.R., Corbett, A.T.: Cognitive Tutor: Applied re- search in mathematics education. Psychonomic Bulletin & Review 14(2), 249–255 (2007). 4. Anderson, J.R.: Rules of the mind. Erlbaum, Hillsdale, NJ (1993). 5. vanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist 46(4), 197–221 (2011). 6. Corbett, A.T., Anderson, J.R.: Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4(4), 253–278 (1995). 7. Lyman, F.T.: The responsive classroom discussion: The inclusion of all students. In: An- derson, A. (ed.) Mainstreaming Digest, pp. 109–113. University of Maryland Press, Col- lege Park, MD (1981). 8. Chi, M.T.H., Roy, M., Hausmann, R.G.M.: Observing tutorial dialogues collaboratively: Insights about human tutoring effectiveness from vicarious learning. Cognitive Science 32, 301–341 (2008). 9. Rummel, N., Spada, H.: Learning to collaborate: An instructional approach to promoting collaborate problem solving in computer-medicated settings. Journal of the Learning Sci- ences 14(2), 201–241 (2005). 10. Weinberger, A., Ertl, B., Fischer, F., Mandl, H.: Epistemic and social scripts in computer- supported collaborative learning. Instructional Science 33, 1–30 (2005). 11. Soller, A., Linton, F., Goodman, B., Lesgold, A.: Toward intelligent analysis and support of collaborative learning interaction. In: Lajoie, S.P., Vivet, M. (eds.) Proceedings of the Ninth International Conference on Artificial Intelligence and Education, pp. 75–82. IOS Press, Amsterdam (1999). 12. Ritter, S., Yudelson, M.V., Fancsali, S.E., Berman, S.R.: How mastery learning works at scale. In: Haywood, J., Aleven, V., Kay, J., Roll, I. (eds.) Proceedings of the 3rd Annual ACM Conference on Learning at Scale, pp. 71–79. ACM, New York (2016). 13. Lindgren, R., McDaniel, R.: Transforming online learning through narrative and student agency. Journal of Educational Technology & Society 15(4), 344–355 (2012). 14. Clement, B., Roy, D., Oudeyer, P., Lopes, M.: Multi-armed bandits for intelligent tutoring systems. Journal of Educational Data Mining 7(2), 20–48, (2015). 15. Lan, A.S., Baraniuk, R.G.: A contextual bandits framework for personalized learning ac- tion selection. In: Barnes, T., Chi, M., Feng, M. (eds.) Proceedings of the 9th International Conference on Educational Data Mining, pp. 424-429. IEDMS (2016). 16. Koedinger, K.R., Booth, J.L., Klahr, D. Instruction complexity and the science to constrain it. Science 342, 935–937 (2013).