Towards a Pedagogical Framework for Designing and Developing iTextbooks * Chaohua Ou1, Ashok Goel1 and David Joyner1 1 Georgia Institute of Technology, Atlanta GA 30332, USA {cou, ashok.goel,david.joyner}@gatech.edu Abstract. Research studies on e-texts have focused on how various interactive features could be useful for engaging students with content during the learning process. As more and more interactive and intelligent components are available for embedding within e-texts, these smart e-texts, or iTextbooks, are no longer just digital textbooks with a few add-ons or tools to help students read. Instead, they should be viewed as a learning platform in their own right. This emerging trend demands a framework that provides guidance on how to design and develop the platform so that these components can be effectively integrated to create an interactive and intelligent environment for students to learn beyond reading text. In this paper, we propose a pedagogical framework for designing and developing iTextbooks as a learning platform, providing students with a visual, adaptive, personalized, and collaborative learning environment. We also present three case studies and one recommendation to demonstrate how to build iTextbooks as a learning platform with four learning strategies. Finally, we discuss how to use the framework to guide the directions of future work. Keywords: iTextbooks, Artificial intelligence, Pedagogy. 1 Introduction The adoption of e-textbooks, or e-texts, is becoming increasingly common in higher education. According to the results of the 2018 National Survey of Student Engagement (NSSE), out of 10,351 undergraduate students from institutions across the United States and Canada, about one-third of them (35%) used e-texts in two or more of their classes, and about a quarter (26%) used e-texts in one course [1]. The adoption is often driven by the advantages of e-texts over print textbooks in terms of affordability, accessibility, and portability – e-texts often cost less and can be conveniently accessed from students’ mobile devices [2]. Despite these benefits, there has been a screen inferiority issue – reading texts from screens was found to negatively affect student learning compared to reading from print textbooks, or paper texts [6, 8, 23]. Copyright © 2022 for this paper by its authors. Use permitted under Creative * Commons License Attribution 4.0 International (CC BY 4.0). 2 On the other hand, many studies have been conducted to investigate how interactive features afforded by screens, but not feasible with paper, could overcome the issue and thereby enhance learning. These studies tend to focus on comparing the effects on stu- dent learning between using e-texts with various interactive features and static e-texts and/or print textbooks [7]. Their findings shed some light on which individual features, or a combination of multiple features, could be useful for engaging students with con- tent during the learning process. As more and more interactive and intelligent compo- nents are available for embedding within e-texts, these smart e-texts, or iTextbooks, are no longer just digital textbooks with a few add-ons or tools to help students read. In- stead, they should be viewed as a learning platform in their own right [3]. This emerging trend demands a framework that provides guidance on how to design and develop the platform so that these components can be effectively integrated to create an interactive and intelligent environment for students to learn beyond reading text. In this paper, we propose a pedagogical framework for designing and developing iTextbooks as a learning platform that will provide students with a visual, adaptive, personalized, and collaborative learning environment. 2 A Pedagogical Framework for Creating Future iTextbooks To develop a framework for building iTextbooks, the first question that will be asked naturally would be what iTextbooks should be like. The geometry book of the future, for example, was envisioned to be a cloud platform that is “adaptive, collaborative, visual and intelligent” (p. 427) [22]. En route to the creation of this future learning platform, the main goal is to establish a general framework with fundamental building blocks in knowledge management, methodologies, and technologies. To attain this goal, it is necessary to examine the current state of research and development in these blocks before initiating large-scale collaborative multidisciplinary efforts. The author acknowledged that “the help of researchers with experience working with the connec- tions between the intersection of education and the technical issues will be very im- portant” (p.430). As researchers working at the intersection of learning and technologies, we would like to share three case studies we conducted on transforming e-texts or online learning by enriching content presentation and enhancing interactivity. Building on the four characteristics of iTextbooks envisioned by Quaresma [22], as well as related research findings, we propose a pedagogical framework for designing and developing iText- books as learning environments, integrating principles from learning science, instruc- tional design, as well as affordance of medium and AI technologies (Figure 1). 2.1 Five Key Components There are five core components, or fundamental blocks, in this framework: 1. Learners: The iTextbooks are designed for the learners, who will most likely use them to learn when the instructor is not present to offer any guidance and help. The learners are expected to interact with the other four components of the framework. 3 While this framework provides general guidelines for the design and development of iTextbooks, specific decisions and strategies need to be made based on the unique needs of the learners in different educational settings. Needs analyses are necessary to understand the learners’ knowledge, skills, capabilities, attitudes, motivations, preferences, etc., before starting the design and development. The analyses are also required for creating learning paths adapted to each learner [18, 24]. Fig. 1. A pedagogical framework for designing and developing iTextbooks. 2. Text Content: Content in static e-texts is often presented in a similar way as it is printed on paper. The only difference is that hyperlinks are available in e-texts for navigating within the book or to external resources. This static content presentation results from an easy conversion from a print version to a web version. In designing and developing iTextbooks, we need to take into consideration how to leverage the web as a medium to present content in an effective way that enables learners to in- teract with the content. A digital glossary, for example, is often used to help enhance comprehension by providing text with definition, translation, pronunciation, or back- ground knowledge. An intelligent way to address learners’ questions on text content would be providing Q&A agents that could answer their questions whenever and wherever they need help. 4 3. Visual Content: Using multimedia is a strategic application of the multimedia prin- ciple – people learn better when content is presented with both words and pictures rather than words alone [5, 16, 17]. In this context, words can be printed or spoken, and pictures can include photos, illustrations, videos, and animations. With the ad- vancement of technologies, it is possible to make visual content interactive. For ex- ample, questions and exercises can be embedded within the videos to allow learners to practice and get automated feedback or adaptive feedback from intelligent tutors. Discussion boards can also be embedded within the videos, allowing learners to ask questions at a specific moment of the video. Other learners or the instructor can re- spond to the questions. 4. Assessment: Questions or exercises for self-assessment are extensively used in e- texts. They provide students opportunities to practice, reflect on, and reinforce what they have learned. Research indicates that practice without feedback does not help students learn [10]. Closed-ended questions are typically used in e-texts as auto- mated feedback can be provided. However, it is challenging, if possible at all, to provide instant feedback on open-ended questions. Intelligent tutors may be able to address this challenge, making it possible to include more question types in the self- assessment and provide adaptive feedback based on the responses from the learners. 5. Artificial Intelligence (AI) Technologies: As discussed above, AI technologies such as intelligent tutors and Q&A agents could be integrated into iTextbooks to enable learners to interact with content, assessment, and their peers in ways that traditional paper texts or e-texts cannot afford. The AI technologies are essential in turning an e-text into a platform for adaptive and personalized learning. Intelligent tutors can provide learners with adaptive feedback on various types of questions embedded in the text and visual content for self-assessment. Q&A agents can address questions from students instantly so that they don’t have to wait to ask the instructor questions and then wait for a response. 2.2 Four Strategies There are four strategies in the framework that are used for designing and developing iTextbooks: (1) multimedia learning, (2) adaptive learning, (3) personalized learning, and (4) collaborative learning. In the next section, we will share three case studies we conducted to shed some light on how these strategies could be implemented with the support of technology affordance. It should be noted that we are still trying to explore these ideas and tools for building some components of an iTextbook. These case studies are intended to serve as catalysts rather than solutions for future research and develop- ment. Building the iTextbooks we envision would require interconnection and integra- tion of knowledge management, methodologies, and technologies in more areas. 5 3 Case Studies 3.1 Multimedia Learning Strategy: Transforming Content Presentation Research studies on interactive e-texts are often focused on examining the effects of adding interactive features, such as multimedia, hypermedia, digital glossary, self-as- sessment with automated feedback, and social annotation tools. Content design, or presentation by leveraging the web as the medium, has been rarely addressed in these studies. The results of a meta-analysis of 26 studies on this topic indicate an overall moderately-sized positive effect of interactive e-texts over static e-texts and/or paper texts [7]. Nonetheless, the authors noted that several factors could limit the generali- zability of the findings – (1) the small number of studies, (2) the small sample sizes in some studies, and (3) varied research designs and contexts across studies. The fourth factor in this regard is that neither the studies reviewed nor the meta-analysis came up with generalizable guidelines that could provide pedagogical guidance for designing and developing interactive e-texts. In this study, we redesigned an open textbook, Principles of Macroeconomics 2e, which is freely available for students to view online or download as a PDF document on the OpenStax website. They could also get a print copy for $5. Content-wise, the e- text is basically a reproduction of the PDF version. The e-text was adopted by the in- structor of an introductory economics class at Georgia Tech that enrolls between 200 and 300 undergraduate students every semester. The results of a student survey after the book was used for a semester in the class indicated that students appreciated that the book was free, but they wished the book could be improved with a better content presentation, more visuals, more interactivity, and more practices [21]. We turned this static e-Text into an interactive textbook through three major transformations: 1. Redesign of text content presentation: The text content was transformed by apply- ing the six guiding principles of multimedia learning (see Figure 2) [5, 16, 17]. 2. Embedding visual content: Visual content was added by embedding a total of 111 curated videos from reliable sources. The videos are generally no more than 10 minutes long, and they must be accessible with captions or transcripts. 3. Embedding self-assessment with feedback: The transformation of the book also in- cludes embedding self-assessment questions. We added 130 multiple-choice ques- tions right after the corresponding topics. There are also 74 open-ended questions at the end of each chapter. Automated feedback is provided for each of the answer choices for these questions. 6 Fig. 2. Multimedia learning principles for the content presentation of iTextbooks. The redesigned textbook was put into use in the course in Fall 2020. We conducted a survey among students, to which we received 128 responses, with a response rate of 46%. When asked about their satisfaction with the textbook, 87% of them said they were Very Satisfied or Satisfied. We also asked them to rate their agreement with sev- eral statements regarding the learning effectiveness of the textbook, as well as the use- fulness of the practice exercises. Figure 3 and Figure 4 show the results of their ratings on each statement. The results indicate overall positive feedback from students on the redesigned textbook. We also asked students an open-ended question on what elements of the textbooks were valuable to their learning. It is interesting to see that students’ responses cover many visual elements added, ranging from content layout and breakdown to flashcards, pictures, graphs, tables, and figures with explanations. They also found the practice questions with explanations after each section to be most useful in understanding the material. A student’s comment sums up the value of the book to him/her: “I love how the textbook is so interactive and really reinforces the lectures. I think it is one of the best textbooks for a class I have ever had due to its interactions, information, and ques- tions it provides”. 7 The textbook is easy to navigate when 90 viewed online. The textbook content is well-organized. 88 The textbook clearly explains concepts. 89 The visuals, e.g., graphics, videos, used in 87 the textbook help me learn better. The textbook is interactive and engaging. 78 Overall, the textbook helps improve my 88 learning experiences in this class. 0 20 40 60 80 100 Fig. 3. Percentage of students who strongly agreed or agreed with the learning effectiveness statements The Review Questions 76 The Quick Knowledge Check Questions 72 The Key Terms flash cards 69 0 20 40 60 80 100 Fig. 4. Percentage of students who rated the self-assessment items Very Useful or Useful. 3.2 Personalized Learning Strategy: AI-Powered Q&A Agents The above case study on transforming content presentation has suggested that a multi- media learning strategy could help enrich the content in e-texts and enhance interactiv- ity. However, for a book comprised of 20 chapters and more than 600 pages, how could we help students quickly get answers when they have questions about the content of the book? Some students may be able to sift through pages and lines of the text to find the information. Others may search the internet or wait until they have a chance to ask the instructor, the teaching assistant, or their peers for help. If only a virtual assistant is available to answer questions whenever and wherever the students need! In a preliminary experiment, we have developed an AI-based question-answering agent called AskJill that automatically answers users’ questions based on a text-based Users’ Guide [13]. VERA (Virtual Experimentation Research Assistant is an AI-based interactive learning environment for inquiry-based learning of scientific knowledge (An et al. 2020): VERA helps learners build conceptual models of complex phenomena, evaluate them through simulation, and revise the models as needed. VERA, along with its User Guide, is publicly available on its website http://vera.cc.gatech.edu. The 8 27-page User Guide covers an introduction to VERA, system requirements, steps to access the tool, and the general approach to building and evaluating a conceptual model of an ecological system. Users can find answers to questions such as (1) how to use the VERA tool for modeling and simulation (including steps to create a project describing a phenomenon and associated models to test various hypotheses), (2) how to use the model editor to manage constituent components and their relationships, (3) how to simulate a model, (4) how to edit model parameters to manipulate results, and (5) how to get help on the tool. Embedded in the VERA interactive learning environment, AskJill automatically an- swers users’ questions and thereby explains VERA’s domain, functionality, and oper- ation. When users first log in on the VERA website, AskJill welcomes them and prompts them to ask their questions about VERA. The users can type their questions into the AskJill question-answering interface (integrated into the VERA website). Ask- Jill provides accurate answers to the questions within the scope of the User Guide within a few seconds. When a user asks a question in VERA’s AskJill interface, it is sent to the AskJill system via a REST API. Inside AskJill, the question is parsed, and then sent to a 2D hybrid classification system. The system contains a 2-stage classification process. The first is a pre-trained NLP-based intent classification layer that classifies each new ques- tion into one of the existing question categories based on user intents. The second is a semantic processing stage that uses the intent to select a rule-based query template. From the 2D hybrid classification system, a query is sent to the VERA’s design knowledge database, and a response is generated. The response generation system re- trieves the associated query response and returns an answer if its confidence value ex- ceeds the minimum threshold (97%). Finally, the dialogue management system post- processes the resulting response, converts it into a “human-like” natural language an- swer, and sends it back to AskJill in the VERA user interface. After answering, AskJill prompts the user to provide feedback, asking “Was this answer helpful”, and stores the user feedback in her database. The feedback is subsequently used for retraining the agent. Suppose AskJill is unable to answer a question. In that case, it can (a) gently redirect the conversation into its domain of competence by suggesting alternate topics associated with the questions it is trained on and/or (b) share relevant links to the User Guide. Figure 5 shows examples of question-answering in AskJill. 9 Fig. 5. Human-generated questions and AskJill’s answers to them AskJill builds on earlier work on the Jill Watson virtual teaching assistant (TA) for automatically answering students’ questions on the discussion forums of online classes [14]. We may think of AskJill as a virtual TA for the VERA system. Indeed, we have extensively deployed VERA and AskJill in several classes in undergraduate biology. The success of Jill Watson and AskJill points to the potential for building AI agents that can similarly answer questions based on iTextbooks. 3.3 Adaptive Learning Strategy: In-Video Tutors The case study we discussed above regarding transforming a static e-text into an inter- active one shows promising results in adding visual content to the original text-heavy version. Although the carefully-curated videos add value to the content, there is a lack of mechanisms for learners to interact with the video content. In a survey conducted among about 1,792 undergraduate and graduate students at Georgia Tech, 95% of the online students and 80% of the residential students said instructional videos were val- uable in helping them learn. When asked what elements of the videos could be barriers to their learning, lack of interactivity is at the top of the list from their responses [19]. Embedding questions within videos is often used to address this issue and enable inter- activity. However, the challenge is that the types of questions that can be embedded are often very limited. They are mostly questions that have standardized answers, such as Yes/No, True/False, and multiple-choice questions. In addition, based on our experi- ence from Case Study 1, developing feedback for each answer choice is time-consum- ing and labor-intensive. Our study on embedding intelligent tutors within videos, or in- video tutors, indicates they could help address this challenge [20]. In this study, we examined online graduate students’ perceptions of the effectiveness of instructional videos designed and developed based on a 7-principle model. The model is comprised of four instructional methods, two principles for instructional presentation, and one for instructional sequence (see Figure 6). One of the methods is 10 adaptive feedback, which is provided by the intelligent tutors embedded within the course videos. Fig. 6. A seven-principle model for designing and developing video lessons [20] The online graduate course comprises 26 video lessons. Interactive exercises were embedded in the video lessons for students to practice and reinforce what they learned from the video. Figure 7 is a screenshot of an example of an open-ended exercise and two pieces of feedback a student may receive from the tutor based on his or her input. In this exercise, students are asked to fill 24 boxes to represent the possible next state of a problem in accordance with the rules provided. Students are provided with adaptive feedback from the intelligent tutor. The tutor operates first by examining whether the input to the problem even makes sense. If not, the tutor supplies feedback on the type of input it will understand, guiding students along to the closed input so that it can process. Then, once it understands the input, it examines whether that input is valid. If the input is valid according to the rules of the exercise, it moves on to checking. For some exercises, the tutor also checks to see if the answer is the best answer. The participants of the study were 1,913 students who took the online course during the eight semesters from Fall 2014 to Spring 2017. A total of 1,242 students completed the end-of-course survey with a response rate of 65%. The results indicate that more than 80% of the respondents agreed that the exercises kept them engaged and that the exercise feedback they received enhanced their understanding of the video lessons. The 11 exercises and adaptive feedback were rated the second on the list in the open-ended question regarding what video elements they liked. A student commented on them: “Very good exercises in videos with useful feedback for incorrect answers. OMG this was so helpful because it often addressed why I would choose an incorrect answer.” The comment validates that adaptive feedback is particularly useful when learners need more than a correct/incorrect response [15]. Fig. 7. An example of feedback provided by the intelligent tutor [12]. 12 3.4 Collaborative Learning Strategy: In-Video Discussions We have not performed any studies on how to integrate collaborative learning strategy into the design and development of iTextbooks. However, other researchers have looked into how to enable students to engage with their peers through collaborative annotation, or social annotation [4, 9, 25, 26, 27, 28]. Two popular software platforms, Hypothesis, and Perusall, allow students to share their annotations on their readings, ask questions, and discuss these questions while reading the texts. All these studies indicate that collaborative annotation on e-texts resulted in better learning than anno- tating on paper texts. It should be noted that annotation could be done with video con- tent as well. Annoto is a software that enables learners to ask questions in specific mo- ments in a video and respond to each other’s questions. The in-video discussion has the advantages of situating the discussion in the context of the video and providing com- prehensive analytics on how and when learners interact with the video content and with each other. For iTextbooks enriched with video content, the in-video discussion could be a good collaborative learning strategy, and future studies should examine how this strategy could help enhance learning with iTextbooks. On the other hand, some chal- lenges could occur in-video discussions – instructors teaching large classes may find it overwhelming to monitor a high volume of questions and discussions. In this case, we may want to explore how to use Q&A agents to help instructors address this challenge by answering questions from students. 4 Summary and Directions for Future Work We propose a pedagogical framework for designing and developing iTextbook, which includes five key components: learners, text content, visual content, assessment, and AI technologies. This framework integrates four learning strategies: multimedia learn- ing, adaptive learning, personalized learning, and collaborative learning. We also pre- sent three case studies demonstrating how iTextbooks could be built as a platform with multimedia learning, personalized learning, and adaptive learning strategies. In addi- tion, we suggest a new direction for implementing the collaborative learning strategy with in-video discussions. Our goal is to use this framework to guide the future work of building visual, adaptive, personalized, and collaborative iTextbooks that can be ac- cessed and used by learners to learn anywhere at any time. This resonates with our mission at the National Artificial Intelligence Research Institute for Adult Learning and Online Education (AI-ALOE). We, along with a team of cross-disciplinary researchers in AI and education, are dedicated to transforming adult online learning in effective- ness, efficiency, access, scale, and personalization. We would like to explore how to leverage AI technologies to help achieve this goal in the near future. Acknowledgements This material is based upon work supported by the National Science Foundation under Grant No. 2112532. 13 References 1. Abaci, S., & Quick, J.: University-wide e-text adoption and students’ use of, preferences for, and learning with e-textbooks. In Inclusive access and open educational resources e-text programs in higher education (pp. 113-123). Springer, Cham (2020). 2. An, S., Bates, R., Hammock, J., Rugaber, S., Weigel, E. & Goel, A.: Scientific Modeling Using Large Scale Knowledge. In Procs. Twentyfirst International Confer-ence on AI in Education (AIED’2020), pp. 20-24 (2020). 3. Boulanger, D., & Kumar, V.: An Overview of Recent Developments in Intelligent e-Text- books and Reading Analytics. iTextbooks@ AIED, 44-56 (2019). 4. Chen, C. M., & Chen, F. Y.: Enhancing digital reading performance with a collaborative reading annotation system. Computers & Education, 77, 67–81(2014). 5. Clark, R. C., & Mayer, R. E.: E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Hoboken, NJ: Wiley (2016). 6. Clinton, V.: Reading from paper compared to screens: A systematic review and meta‐anal- ysis. Journal of Research in Reading, 42(2), 288-325(2019). 7. Clinton-Lisell, V., Seipel, B., Gilpin, S., & Litzinger, C. (2021). Interactive features of E- texts’ effects on learning: a systematic review and meta-analysis. Interactive Learning En- vironments, 1-16. 8. Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L.: Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educa- tional Research Review, 25, 23-38 (2018). 9. Dennis, A. R., Abaci, S., Morrone, A. S., Plaskoff, J., & McNamara, K. O.: Effects of e- textbook instructor annotations on learner performance. Journal of Computing in Higher Education, 28(2), 221–235 (2016). 10. Fiorella, L., & Mayer, R. E.: What works and what doesn’t work with instructional video. Computers in Human Behavior, 89, 465–470 (2018). 11. Goel, A.: AI-powered learning: making education accessible, affordable, and achievable. arXiv preprint arXiv:2006.01908 (2020). 12. Goel, A., & Joyner, D. A.: Using AI to Teach AI: Lessons from anonline AI class. AI Mag- azine, 38(2), 48–58 (2017). 13. Goel, A., Nandan, V., Gergori, E, An, S., Rugaber, S.: Explanation as Question Answering based on User Guides. In Proceedings AAAI Workshop on Explanable Agency in AI (2022). 14. Goel, A., & Polepeddi, L.: Jill Watson. In Dede, C., Richards, J, & Saxberg, B (Eds.) Learn- ing Engineer-ing for Online Education: Theoretical Contexts and De-sign-Based Examples. Routledge (2018). 15. Johnson, C. I., & Priest, H. A.: The feedback principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 449–463). Cambridge: Cambridge University Press(2014). 16. Mayer, R. E.: Multimedia learning (2nd ed.). New York, NY: Cambridge University Press (2009). 17. Mayer, R. E.: Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. E. Mayer (Ed.). Cambridge handbook of multime- dia learning (2nd ed., pp. 345–368). New York, NY: Cambridge University Press (2014). 18. Normadhi, N. B. A., Shuib, L., Nasir, H. N. M., Bimba, A., Idris, N., & Balakrishnan, V.: Identification of personal traits in adaptive learning environment: Systematic literature re- view. Computers & Education, 130, 168-190 (2019). 19. Ou, C., Goetzel, W.: Learning with videos in face-to-face and online classes. EDUCAUSE Conference, Chicago, IL (2019) 14 20. Ou, C., Joyner, D.A., & Goel, A.K.: Designing and developing video lessons for online learning: A seven-principle model. Online Learning, 23(2), 82-104 (2019).. 21. Ou, C., & Urmanbetava, A.: Redesigning an open textbook by leveraging media, pedagogy, and student collaboration. Open Education Conference. Virtual (2020). 22. Quaresma, P.: Towards an Intelligent and Dynamic Geometry Book. Mathematics in Com- puter Science, 11(3-4), 427-437 (2017). 23. Singer, L. M., & Alexander, P. A.: Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal. Review of Educational Research, 87(6), 1007-1041(2017) 24. Vandewaetere, M., Desmet, P., & Clarebout, G.: The contribution of learner characteristics in the development of computer-based adaptive learning environments. Computers in Hu- man Behavior, 27(1), 118-130 (2011). 25. Weng, C., Otanga, S., Weng, A., & Cox, J. Effects of interactivity in E-textbooks on 7th graders science learning and cognitive load. Computers & Education, 120, 172–184 (2018). 26. Zarzour, H., & Sellami, M.: A linked data-based collaborative annotation system for increas- ing learning achievements. Educational Technology Research and Development, 65(2), 381–397(2017). 27. Zarzour, H., & Sellami, M.: An investigation into whether learning performance can be im- proved by CAALDT. Innovations in Education and Teaching International, 55(6), 625–632. (2018a). 28. Zarzour, H., & Sellami, M.: Effects of a linked data-based annotation approach on students’ learning achievement and cognitive load. Interactive Learning Environments, 26(8), 1090– 1099 (2018b).