The Generalized Intelligent Framework for Tutoring (GIFT) is a domain-independent framework for developing intelligent tutoring systems. While GIFT has primarily focused on individual learning, one of the ultimate goals of GIFT is to be used for team tutoring. As part of implementing team tutoring in GIFT there have been practical, authoring, and technological challenges. In this paper we discuss the goals of team tutoring in GIFT, progress that has been made, and the challenges that still remain in creating assessment and intervention during team tutoring. We also discuss the specific goals of this workshop.
Creating intelligent team tutoring systems (ITTSs) is a uniquely challenging research area. While there is a robust body of research on systems and frameworks for creating individual tutoring systems, there is comparatively much less research in the area of intelligent tutoring systems (ITSs) for teams. Part of the discrepancy lies in an almost factorial complexity of tasks and assessments that need to be configured for a team of individuals. These complexities include both technological considerations (e.g., intercomputer communication), and instructional challenges. One of the goals of the Assessment and Intervention during Team Tutoring Workshop is to provide a forum to discuss the different approaches that researchers have used, and plan to use, in tackling the difficult challenge of creating ITTSs. 1.1 For the purposes of this workshop, we define team tutoring as a computer-based tutoring event in which there are multiple individuals working collaboratively on a joint task. As such, work involving traditional teams -- as well as collaborative problem solving -- are covered. Many of the challenges in creating a tutor for a traditional team also exist in collaborative ITS based problem-solving situations. These challenges include: determining how computer systems can communicate data to each other; tracking individual performance; tracking team performance; determining the kinds of feedback (individual or team; explicit or motivational); ensuring and supporting team communication; and, if relevant, recording team communication for real-time or after-the-fact analysis. These, and other elements of both individual and collaborative problem solving situations, represent some of the salient, complex processes that need to be addressed both independently and concomitantly in order to build effective and useful ITTSs. 1.2
Goals of this Workshop The Assessment and Intervention during Team Tutoring Workshop covers both empirical and theoretical approaches to ITTSs. Given the widely acknowledged complexity of team tutoring, this workshop provides a space for theoretical models that are not yet implemented but could provide insight into approaches that can be used to realize future team models.
The papers received and presented as part of the workshop cover diverse approaches and topics. Included in the agenda are examples of collaborative problem-solving studies that have been conducted; examples of frameworks that can assist in collaborative learning and problem solving; approaches to examining team communications; and discussions of the specific challenges that are associated with team tutoring. One of the goals of this workshop is to determine commonalities that exist between these different approaches and implementations. This will help shape the steps forward for not only team tutoring in general, but also the implementation of team tutoring in the Generalized Intelligent Framework for Tutoring (GIFT) software project [1] that is being developed by the US Army Research Laboratory. 2
The Generalized Intelligent Framework for Tutoring and Team Tutoring GIFT is a domain-independent framework for creating ITSs. It includes components that are standard to ITSs (learner module, pedagogical module, domain module, tutoruser interface) as well as a sensor module and gateway module [1]. The gateway module allows GIFT to communicate with external programs (including PowerPoint and Virtual Battlespace 3), and to provide feedback based on actions that are taken in those programs. GIFT includes authoring tools that allow instructors or subject matter experts (SMEs) that do not have a background in computer science to create their own ITSs. GIFT includes a survey authoring system which allows for surveys, individual questions, and question banks to be created for use in GIFT courses.
Team Tutoring is an area in which initial work has begun in GIFT. Among the projects related to Team Tutoring are described in the following subsections: 2.1 Team Meta-Analysis. A large-scale literature search and meta-analysis was conducted to provide the theoretical background for the team tutoring implementation in GIFT. The results of the meta-analysis are reported in depth in a recent paper by Sottilare, Salas, Burke, Sinatra, Johnston, and Gilbert [2]. In addition to this comprehensive meta-analysis, behavioral markers and team member actions are identified, which in turn can contribute to developing team assessments in an ITTS. An anticipated next step is implementing these markers and actions into the GIFT framework by converting these elements into quantifiable measures that are calculated and used by GIFT in realtime. 2.2 Team Surveillance Task. The ability to create a tutor for small teams has been demonstrated in GIFT [3]. The task created in this effort was executed within the serious game Virtual Battlespace 2 (VBS2). In the initial implementation, two team members monitored a 180-degree area and communicated with each other when a threat was about to pass to their teammate’s sector. Assessment was occurring in real time on both individual and team performance measures, and it was demonstrated that feedback could be given to both an individual and a team. This task was then scaled up to be a three-person task that included an additional role and additional tasks to be engaged in by the team members. 2.3 Towards a GIFT Mission Command Team Training Model for USMA Cadets. In the 2017-2018 academic year, ARL researchers and instructors in the Department of Military Instruction (MS) have been laying the ground work for developing an effective team tutoring model that would support the classroom-based, MS-novice cadet at the United States Military Academy (USMA). The objective of this effort is to identify salient variables of role adoption that emerge in self-selected team formations during classroom assessments within MS.
Given the implicit and explicit efforts both inside and outside the classroom in team training for a platoon organization, ARL researchers and USMA MS instructors have decided to take a closer look at the dynamics of self-selected team configurations that emerge within the MS200 course. Based on qualitative observations within the classroom, this course was deemed as a valid starting point for developing a team training model that can be built in GIFT and used to evaluate both team assessments within an MS200 course, as well as identify effective and dysfunctional team formations. 2.4 Team Search and Rescue Task. Current work in the area of team tutoring in GIFT has included the development of a search and rescue task that will be implemented in Virtual Battlespace 3 (VBS3) [4]. The scenario that is being developed is expected to have a 9-person team (squad) that will need to work collaboratively to complete the search and rescue task, incorporating the use of sub-teams and different team roles within the scenario. Behavioral markers identified in Sottilare et al. [2] will be the basis for the team measures used during the task. This project is still early in development and will demonstrate the ability of GIFT to handle assessment and intervention for many team members simultaneously.
Implications of Team Tutoring Work to GIFT and other ITSs The prior and anticipated theoretical as well as practical work done in GIFT is an example of team tutoring in action in a generalized framework for ITSs. The lessons learned through analysis and research are relevant to developers of ITTSs and contribute to the development of a roadmap for future efforts. This roadmap includes work that still needs to be done in developing sound theoretical constructs as well as in obtaining empirical evidence that can validate these constructs. Following this careful analysis and validation, greater strides can then be made in subsequently implementing these validated team training models in real world scenarios to assess their efficacy. Additionally, more work will need to be done to address the generalizability of these ITTS models across a range of domains that have unique roles and performance task needs. Encompassing all these efforts, then, this roadmap ultimately includes insuring that validated team training modeling frameworks are inherently flexible for authoring, configuration, and scalability without losing effectiveness -- irrespective of domain distinctiveness.
Accordingly, the current workshop will cover a broad range of topics including: frameworks for team tutoring, communication during a team tutoring experiment, and theoretical implications of constructing team tutoring. In all, the main goal of the Assessment and Intervention during Team Tutoring Workshop is to examine and discuss existing ITTSs in different domains and configurations so that it can offer insight into the best approaches and most relevant challenges encountered in the ITTS landscape.
Acknowledgements. The research described herein has been sponsored by the U.S. Army Research Laboratory/NSRDEC Simulation and Training Technology Center (STTC). The statements and opinions expressed in this article do not necessarily reflect the position or the policy of the United States Government, and no official endorsement should be inferred. 4