=Paper=
{{Paper
|id=Vol-2501/shortpaper1
|storemode=property
|title=Introduction to Approaches and Challenges in Team Tutoring
|pdfUrl=https://ceur-ws.org/Vol-2501/shortpaper1.pdf
|volume=Vol-2501
|authors=Anne M. Sinatra,Jeanine A. DeFalco
|dblpUrl=https://dblp.org/rec/conf/aied/SinatraD19
}}
==Introduction to Approaches and Challenges in Team Tutoring==
https://ceur-ws.org/Vol-2501/shortpaper1.pdf
1
Introduction to Approaches and Challenges in
Team Tutoring
Anne M. Sinatra1 & Jeanine A. DeFalco1,2
1 US Army Combat Capabilities Development Command – Soldier Center –
Simulation & Training Technology Center, USA
2 Oak Ridge Associated Universities, USA
anne.m.sinatra.civ@mail.mil
jeanine.a.defalco.ctr@mail.mil
Abstract. There are many challenges that are associated with the field of team
tutoring. Intelligent tutoring for an individual is a difficult task, and it gets even
more complex when a solution is scaled up to support team tutoring. Complex
considerations include tracking of individual learners, team roles, overall team
performance, and feedback time/frequency/type. Intelligent tutoring system
(ITS) frameworks such as the Generalized Intelligent Framework for Tutoring
(GIFT) are currently working on team tutoring implementations in a domain-in-
dependent fashion. As GIFT is a research project, lessons learned from other team
tutoring systems are valuable and will help shape its development. The Ap-
proaches and Challenges in Team Tutoring Workshop included topics such as
team tutoring, natural language processing, and collaborative learning. Addition-
ally, there was discussion about what worked and did not work in existing imple-
mentations.
Keywords: Team Tutoring, Intelligent Tutoring Systems, Generalized Intelli-
gent Framework for Tutoring
1 Approaches and Challenges in Team Tutoring
In June 2018, we chaired a workshop titled Assessment and Intervention during Team
Tutoring in association with the Artificial Intelligence in Education (AIED) 2018 con-
ference. We had a very positive response to that workshop, and through discussion that
occurred during the workshop it was established that there were commonalities in the
approaches and challenges that were being encountered even in very different domain
areas. This led to the Approaches and Challenges in Team Tutoring Workshop at the
AIED 2019 conference. While our original workshop contained six presentations from
researchers, we had a very strong response to our 2019 workshop with nine papers se-
lected for presentation. This increase in submissions demonstrates that team tutoring is
an up-and-coming research area, and the submissions provide evidence that this area
includes many challenging considerations.
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
2
1.1 Team Tutoring
While developing intelligent tutoring systems (ITSs) is challenging, ITSs are often fo-
cused on tutoring an individual learner. It is typical that ITSs include common compo-
nents, such as a learner module, pedagogical module, domain module, and a tutor-user
interface [1]. Based on how it has been authored and configured, each ITS will differ
in the approaches to assessment, feedback types, and characteristics of the learner that
are being tracked and adapted upon. When altering individually oriented ITSs to ad-
dress team tutoring there are many complex design considerations. Generally, team tu-
toring is not as simple as just summing the individual performance of each of the learn-
ers to arrive at a composite team model. Rather, the specific domain, domain-specific
team structure, potentiality of team roles and use and purpose of feedback are all of
particular interest. For example, in the important case of tracking and assessing com-
munication amongst team members in real time, the methods to assess and subsequently
inform adaptive content in real time becomes very, very complex [2].
1.2 Goals of this Workshop
The Approaches and Challenges during Team Tutoring Workshop covered both empir-
ical and theoretical approaches to ITSs for teams. The papers included in these pro-
ceedings and presented at the workshop comprised many approaches and were
grounded in different educational domains. Specific topics of discussion following
these presentations included: mechanisms for enhancing an ITSs ability to process team
communications; individual differences and their impact on team tutoring; collabora-
tive problem solving; tools for assisting with authoring of team tutoring; and lessons
learned from building team tutors.
A goal of this workshop was to provide a forum for researchers who continue to
work in the field and expand their research, as well as new contributors in the field. The
workshop included three follow up papers that built on work from our previous work-
shop, and work from six new contributors. Topics included: a framework for team eval-
uation; the need for sensors; challenges of natural language processing; lessons learned
from large team training; tools for authors of team training; collaborative problem solv-
ing; characteristics of the individual that may impact team tutoring; and simulated stu-
dents. Even though these topics sound varied, each paper identifies common challenges
that are common in the domain of team tutoring systems. Some specific commonalities
identified the need to understand and process what individual team members are doing
in real time, and the capability of the team tutoring system to track individual states and
provide distinguishable feedback for both individuals and for the team. In particular,
the need to have an understanding of the communication that is occurring during the
team tutoring was of particular importance, and was repeatedly identified as a thorny
and difficult problem.
It is our intention that the outcomes of this workshop will help shape the steps for-
ward for the Generalized Intelligent Framework for Tutoring (GIFT) project as it con-
tinues to be developed by the US Army CCDC Soldier Center and to address team
tutoring while also providing valuable findings to the tutoring community at large.
3
2 The Generalized Intelligent Framework for Tutoring and
Team Tutoring
GIFT is an open source and domain independent framework for creating ITSs. GIFT
includes a number of authoring tools that allow instructors and subject matter experts
who do not have a background in computer science to be able to author complete ITSs.
GIFT includes the standard components of an ITS (i.e., learner, pedagogical, and
domain modules) in addition to a sensor module and a gateway module, which allows
communication with external computer programs or game environments [1]. Work has
been on-going in GIFT to support team tutoring. Current challenges that have been
encountered while making the shift from individual and team tutoring in GIFT are: how
to deal with/process semantic information from real-time communications; how to ac-
count for team members that have different roles; how to track/assess individual team
members as well as the team as a whole.
Initial work in this area includes a meta-analysis of the relevant team literature and the
identification of behavioral markers that would be relevant to implement in GIFT [2].
Additionally, initial team tutoring work has been demonstrated through a Virtual Bat-
tlespace 2 (VBS2) team surveillance task, providing feedback to team members while
they engaged in a scenario. The first iteration of the task included two team members
(in the same role), and the second version had three team members (in two different
roles) [3]. Current on-going work with GIFT has demonstrated team tutoring at the
Squad level with a series of Search and Rescue scenarios in Virtual Battlespace 3
(VBS3) that are designed for nine team members. These scenarios include overall team
assessments and after-action review feedback [4, 5]. Work is continuing on this project
to improve the assessment and feedback occurring in real-time during the scenarios.
3 Implications of Team Tutoring Work to GIFT and other ITSs
One of the challenges of developing GIFT is that it is domain-independent. All of the
procedures and authoring tools that are being developed need to work for multiple do-
mains. This is even more challenging when different team tasks have different charac-
teristics, team roles, and team configurations. Some of the lessons learned in varying
domains are highly applicable to GIFT as they provide examples of unique implemen-
tations of team tutoring that have encountered specific challenges and addressed them.
The lessons learned and outcomes of these works can help to influence the design of
GIFT, and ITSs for teams in general.
Acknowledgements. The research described herein has been sponsored by the U.S.
US CCDC Soldier Center and US Army Research Laboratory. The statements and
opinions expressed in this article do not necessarily reflect the position or the policy of
the United States Government, and no official endorsement should be inferred.
4
4 References
1. Sottilare, R., Brawner, K., Sinatra, A. & Johnston, J. (2017). An Updated Concept for a
Generalized Intelligent Framework for Tutoring (GIFT). Orlando, FL: US Army Re-
search Laboratory. May 2017. DOI: 10.13140/RG.2.2.12941.54244.
2. Sottilare, R. A., Burke, C. S., Salas, E., Sinatra, A. M., Johnston, J. H., & Gilbert, S. B.
Designing adaptive instruction for teams: A meta-analysis. International Journal of Arti-
ficial Intelligence in Education, 28(2), 225-264 (2018).
3. Gilbert, S. B., Slavina, A., Dorneich, M. C., Sinatra, A. M., Bonner, D., Johnston, J., ...
& Winer, E. Creating a team tutor using GIFT. International Journal of Artificial Intelli-
gence in Education, 28(2), 286-313 (2018).
4. McCormack, R.K., Kilcullen, T., Case, A., Wade, A., Howard, D., Brown, T., & Sinatra,
A.M. Training teamwork skills in an Intelligent Tutoring System. Proceedings of the
2019 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC).
(in press).
5. McCormack, R.K., Kilcullen, T., Sinatra, A.M., Case, A.., & Howard, D. Teamwork
training architecture, scenarios, and measures in GIFT.. Proceedings of the 7th Annual
GIFT Symposium, pp. 131- 139. (2018).