Luna: A Dashboard for Teachers Using Intelligent Tutoring Systems

Abstract

Intelligent Tutoring Systems (ITS) generate a wealth of finegrained student interaction data. Although it seems likely that teachers could benefit from access to advanced analytics generated from these data, ITSs do not typically come with dashboards designed for teachers' needs. In this project, we follow a user-centered design approach to create a dashboard for teachers using ITSs.

-

• Human-centered computing~Human (HCI) • Applied computing~Education

1. THE LUNA DASHBOARD

Intelligent Tutoring Systems (ITS) [ 4, 8, 9 ] typically generate a wealth of fine-grained data about student progress and learning. The analytics that can be derived from these data include, for example, estimates of student knowledge, decomposed by skills and misconceptions within a domain, as well as time and progress on various activities within the ITS. Although it seems highly likely that human teachers could benefit from access to these analytics, ITSs do not typically come with teacher dashboards that are designed with a thorough understanding of teachers’ needs for actionable information in various contexts. More often, analytics from ITSs are used for research purposes or in Open Learner Models shown to the student. While some ITSs show reports that may be useful for teachers (e.g. [ 1, 2 ]), these are not typically designed specifically to address teachers’ needs.

In our project [3], our goal is to support teacher decision-making and self-reflection in blended learning environments that use ITSs in conjunction with classroom instruction, particularly in contexts where ITS-use and classroom instruction occur at separate times. We are in the process of creating a dashboard for an ITS that supports step-based problem solving (as many ITSs do [ 4 ]). As our initial test bed, we use Lynnette – a simple but highly effective ITS for basic equation solving, built in our lab [ 5, 6 ]. Lynnette has been used in a number of middle schools in our region, in research studies with students in grades 6 through 8 (1114 year olds). However, our goal is to create a dashboard that can be used with any ITS that, like Lynnette, is built within the CTAT/Tutorshop environment for authoring and deployment of ITSs [7]. This environment provides both efficient authoring tools as well as a system for web-based deployment with learning management facilities for teachers. The CTAT/Tutorshop environment has previously been used to build many ITSs that have been shown to be effective in classrooms.

In creating the Luna dashboard, we follow a user-centered design approach, grounded in data collected about teacher and student needs. So far, we have used Contextual Inquiry together with other design methods such as Interpretation Sessions and Affinity Diagramming to collect user data from middle-school teachers [ 10 ]. We created a medium-fidelity prototype, which we then used in a classroom study with real student data.

We are currently redesigning this early dashboard prototype (see Figure 1) based on extensive feedback and usage data from teachers. We then plan to move to an implementation, within the CTAT/Tutorshop environment, of a fully functioning initial version. During the workshop we will show some of the design iterations the dashboard underwent, and we will demo a functioning early prototype. In addition, we will introduce extensions we are making to the CTAT/TutorShop environment, which facilitate the iterative prototyping and deployment of learning analytics tools for ITSs.

Although the notion of a teacher dashboard for advanced learning technologies is not in itself new, our project may have a somewhat unique combination of characteristics. As mentioned, few ITSs have teacher dashboards. Further, we are carefully considering the unique needs of different usage-scenarios and designing to address them. Specifically, in our design process we are considering teachers’ needs in two usage scenarios within a single project: exploratory/reflective use (a dashboard that offers formative reports, accessible anytime by the teacher), as well as real-time decision support (a dashboard that helps teachers monitor and help their students during live, in-class use of ITSs). Each of these scenarios leads to different teacher needs and designs. Finally, we ultimately aim to study how teachers use these dashboards, and how student learning is affected by teachers’ use of the dashboards.

2. ACKNOWLEDGMENTS

We thank Gail Kusbit, Octav Popescu, Jonathan Sewall, Cindy Tipper, and all participating teachers for their help with this project. The research reported here was supported by NSF Award #1530726 and by the Institute of Education Sciences, U.S. Department of Education, through Grant R305B150008 to Carnegie Mellon University. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

3. REFERENCES

Woolf BP. Building intelligent interactive tutors: Studentcentered strategies for revolutionizing e-learning. Burlington, MA: Morgan Kaufmann; 2009.

References

[2] Arroyo , I. , Woolf , B. P. , Burleson , W. , Muldner , K. , Rai , D. , & Tai , M. ( 2014 ). A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect . International Journal of Artificial Intelligence in Education , 24 ( 4 ), 387 - 426 . doi: 10 .1007/s40593-014-0023-y [3] [10] Beyer , H. , & Holtzblatt , K. ( 1997 ). Contextual design: defining customer-centered systems . Elsevier.

[4] VanLehn , K. ( 2006 ). The behavior of tutoring systems . International Journal of Artificial Intelligence in Education , 16 ( 3 ), 227 - 265 .

Waalkens , M. , Aleven , V. , & Taatgen , N. ( 2013 ). Does supporting multiple student strategies lead to greater learning and motivation? Investigating a source of complexity in the architecture of intelligent tutoring systems . Computers & Education , 60 ( 1 ), 159 - 171 .

doi:10 .1016/j.compedu. 2012 . 07 .016

[6] Long , Y. , & Aleven , V. ( 2016 ). Mastery-Oriented shared student/system control over problem selection in a linear equation tutor . In A. Micarelli, J. Stamper , & K. Panourgia (Eds.), Proceedings of the 13th international conference on