Sensing Team Interaction to Enhance Learning and Performance during Adaptive Instruction

Abstract

Adaptive instruction is any individual or collective learning experience guided by artificially-intelligent, computer-based systems that tailor instruction and recommendations based on their goals, needs and preferences. Research into adaptive instructional methods has gained prominence over the last five years with a greater understanding of the benefits that tailored training and educational experiences provide to learner. While the application of adaptive instruction to task domains for individual learners has been prevalent, there is a growing desire to realize the same benefits for team instruction. As with any instruction, there are a set of measures that determine progress toward a set of learning objectives. In this paper, we discuss the importance of measures related to the interaction of team members, how this data might be captured, and how it might be interpreted to identify trends, provide recommendations, and select optimal instructional actions (e.g., feedback, support, direction) for teams.

Keywords

• Adaptive Instruction
• Sensors
• Team Interaction
• Team Learning
• Team Performance

-

Building upon work by Burke [ 1 ], we define teams and contributions to their success as follows:  A team is set of two or more individuals, interacting interdependently and adaptively towards a common valued goal or set of goals.  Team members generally have defined roles and responsibilities, but their roles may overlap and in some cases be redundant.  Teams must master both taskwork and teamwork skills to be optimally effective  Team effectiveness is also influenced by the level of effort provided by team members, their performance strategies, and the individual and collective knowledge and skills in the task domain(s) they operate within.  Individual and team knowledge, skills and abilities are greatly influenced by the amount and effectiveness of deliberate practice (training).  Adaptive instructional systems (AISs) should tailor training to the capabilities of the team and its members, and provide relevant content and effective strategies in pursuing the goal of optimal team learning and performance.

AISs are artificially-intelligent, computer-based systems that guide learning experiences by tailoring instruction and recommendations based on the goals, needs, and preferences of each individual learner or team in the context of domain learning objectives [ 2 ]. The goal of adaptive instruction is to provide computer-guided, self-regulated experiences for individuals and groups that are equivalent to or better than instruction provided by an expert human tutor. AISs support technology-enhanced learning (TEL) which “aims to design, develop and test sociotechnical innovations that will support and enhance learning practices” [ 3 ]. AIS learning technologies include intelligent tutoring systems (ITSs), recommender systems, and other intelligent media that model the learner and tailor instruction based on that learner model (Figure 1).

ITSs are computer learning environments that help learners master knowledge and skills using intelligent algorithms that tailor to learner idiosyncrasies at a fine-grained level and that instantiate complex principles of learning [ 4-5 ]. ITSs normally work with one learner at a time, but emerging capabilities are targeted to support automated instruction for groups of collaborative learners or teams of learners. AIS architectures such as the Generalized Intelligent Framework for Tutoring (GIFT) [ 4, 6-7 ] are not specifically ITSs, but they do provide the building blocks (components, tools, and processes) needed to generate ITSs and instantiate the design principles that govern the delivery of automated instruction by their ITSs [ 6-7 ].

Recommender systems provide strategies or plans for the AIS’s next action based upon the learner’s state(s) or suggestions about what the learner should do next. Recommendations can include suggestions about where to find novel domain resources, identification of other learners with similar interests or optimal learning paths through the learning resources [ 3 ]. We have created the category of “intelligent media” as a catch-all for AISs that are not ITSs or recommender systems.

Automatically instructing individual learners is primarily focused on a process of acquiring data about a specific learner’s behaviors and physiology, and then using that data to classify current states and predict future states. Learner states (e.g., performance, emotional) are then used by an AIS to determine readiness to learn and gaps between the individual’s knowledge and skills and the instructional objectives. While this is difficult, automating the acquisition of individual learner behaviors and classifying their states to determine their progress toward learning objectives has received substantial attention and resulted in significant progress in both government and academic research during the past 10 years. This includes research on evidence-centered design [ 8 ], selfregulation [ 9 ], and stealth assessment [ 10 ]. Now the automated assessment of teams is gaining strong support.

One critical need recently identified by the US Army’s Synthetic Training Environment (STE) program is the capability to automatically facilitate the training and education of teams. Whether it is a fire team training to learn building clearing tactics, a squad working to collaboratively to solve problems in the field, or a staff working to collectively develop recommended courses of action for their commander, the members of a team act, but also interact (e.g., communication, coaching, cooperating) with each other, and this makes the modeling of teams considerably more complex than the modeling of individual learners.

Team instruction includes the need to understand not only the progress toward learning objectives, but also interactions (e.g., teamwork) that have second-order effects on learning. We contend that a lack of interaction data makes it difficult to detect, classify, and predict interactions between team members, and we propose that there exists a need to adapt or develop sensors that are capable of acquiring this interaction data to identify team behavioral markers as recommended by Sottilare et al. [ 11 ].

Most solutions to the team interaction process have been manual with human observers identifying interaction occurrences and determining their meaning. Many times, this observation process is time consuming and expensive, but also occasionally inaccurate. Ideally, we seek to automate this observation process, but technology (a tool or method) is required to capture data associated with these events in real-time to support efficient and effective training. This paper targets the automated specific team interactions and provides recommendations for sensors to support the detection, acquisition, and classification of team interactions.

Challenges

According to Sottilare et al. [ 11 ], a group interacting while under instruction may be categorized into one of three areas: team taskwork (a group learning to do a task together), teamwork (the interactions of group members working toward a shared goal), and collaborative learning (a group of learners with a shared learning goal or problem to solve) [ 12 ]. Taskwork is a subset of team training that is focused on developing proficiency in task domains required for a specific duty of one’s job [ 12 ]. Teamwork is the “coordination, cooperation, and communication among individuals to achieve a shared goal” [ 13 ]. Collaborative learning (also referred to as cooperative learning) is “a situation in which two or more people learn or attempt to learn something together” [ 14 ]. The interaction between team members is the key to understanding group instruction and the acquisition of interaction data is the key to identify teamwork behaviors. The three major challenges are: 1. Unobtrusive data acquisition - identifying or adapting existing sensors or creating new sensors to unobtrusively acquire data about team member interaction behaviors (primarily communication). 2. Teamwork state classification - applying appropriate machine learning methods to accurately classify teamwork states: communication, cooperation, coordination, cognition, coaching, conflict, and instructional conditions as identified by Salas, Dickinson, Converse, and Tannenbaum [ 15 ]. 3. Selecting optimal plans and actions - given an accurate picture of teamwork, selecting appropriate instructional strategies and tactics to optimize learning and performance. 3

Related Research

Examining the three challenges identified above, we now discuss related research for both individual learners and teams. We refer to the learning effect model as a basis for understanding AIS processes for both individuals and team assessments [ 11 ]. 3.1

Assessing Individual Learner States

Previous work in this area is related to our first challenge of acquiring learner data but is limited to low cost sensors for assessing individual learner states rather than team states. Carroll et al. [ 16 ] conducted a survey of low cost behavioral and physiological sensors including EEGs, heart rate monitors, breathing straps, pressure sensors, and low-cost eye trackers. Kokini et al. [ 17 ] evaluated the data acquired by low cost sensors to classify learner states including workload, attention, engagement, and emotions that mediate learning (e.g., frustration, anxiety, and boredom).

Assessing Teamwork States

Johnson, Sottilare, Sinatra and Burke [ 18 ] integrated several sources of research related to assessing teamwork states within intelligent team tutoring systems (ITTSs). Sottilare et al. [ 11 ] identified several team behavioral markers indicative of a variety of teamwork behaviors but did not specifically identify how that data would be acquired and assessed. So, what are the next steps needed to move us forward? Since the bulk of interaction data is verbal and non-verbal communication, we expect to investigate methods to acquire and interpret communication data.

DeCostanza, Gamble, Estrada and Orvis [ 19 ] identified unobtrusive assessment methods while suggesting sources of psychophysiological data (e.g., heart rate variability, eye tracking, neural responses) to support both automated team performance (taskwork) assessment and teamwork assessment. Taskwork represents the objectives, roles, responsibilities, and actions of the team and its members [ 20 ]. Teamwork describes the interaction behaviors of team members as they communicate, coordinate, cooperate, lead, and support each other [ 20 ].

Freeman and Zachary [ 21 ] identified challenges associated with the design of ITTSs including the processing of communications data and the lack of automated and generalizable measures of teamwork. They also identified several essential features including “the use of team training objectives, teamwork models, measures of teamwork, diagnostic capability, instructional strategies, and adaptation of training to team needs” [ 21 ].

Sinatra and Sottilare [ 22 ] also identified ITTS design features and challenges, including the ability of the ITTS to process and respond in near-real time. Currently, many of the behavioral markers (e.g., communications) needed for assessment and instructional management “rely heavily on human intervention, interpretation, and coding” [ 22 ]. A significant number of communication behavioral markers (over 100) have been identified [ 11 ], but this communication data must be processed to assess teamwork states and select optimal instructional strategies and tactics. In the next section, we begin to evaluate processes to automatically analyze communications with the goal of determining teamwork states. 3.3

Automated Analysis of Communications to Assess Teamwork

States

Communication, both verbal and non-verbal, is a common element of all team-based activities and is the single most influential process related to teamwork assessments. LeCouteur [ 23 ] highlighted the importance of maintaining high frequencies of communication between players during team sports, but the communication is important to successful performance in any coordinated team activity (e.g., military maneuvers). Empirically, communications account for 28% of the variance in team learning, and 13% of variance in team performance.

In this section, we examine methods to automatically analyze team communications as a basis for assessing teamwork states. The reason we want to automatically conduct this analysis is due to the expense of manual analysis. Emmert and Barker [ 24 ] identified a study in which manual communication analysis required 28 hours of transcription and encoding for every hour of communication. This analysis could be reduced to one hour with automated, real-time transcription and encoding processes. This section provides a review of some of the approaches that might facilitate automated analysis of team communications.

Foltz and Martin [ 25 ] describe two approaches to automated analysis of team communications to assess team performance: 1) theory-based and 2) model-based. In the theory-based approach to learning analytics, the researcher uses a cognitive, social or communication theory to identify key factors and then tests these factors to see how well the model accounts for the key factors. In the model-based approach to learning analytics, the researcher uses human-derived (identify by a subject matter expert) or objective team performance measures and evaluates the relationship between these measures and team performance. Patterns of communication provide information about the type and duration of interactions between team members. Latent semantic analysis (LSA) is used to analyze the content of communications by measuring and comparing semantic information in verbal interactions.

Epistemic Network Analysis (ENA) is a theory-based approach to learning analytics and is used to model and compare the structure of connections between elements in coded data [ 26 ]. For example, a set of 8 verbal response modes (VRMs) represents a generalized set of communication behaviors within a team [ 26 ]:  Disclosure - reveals thoughts, feelings, perceptions, intentions.  Advisement - attempts to guide behavior, suggestions, commands.  Edification - states objective information.  Confirmation - agreement, disagreement, shared experience or belief.  Question - requests information or guidance.  Interpretation - explains or labels the other, judgments or evaluations of behavior.  Reflection - repetition, restatements, puts other’s experience into words.  Acknowledgment - conveys receipt of communication.

ENA, uses communication data to construct models of learning that are visualized as network graphs (Figure 2) that are mathematical representations of patterns of connections [ 27 ]. When employing ENA, “it is essential to consider the semantic and conceptual content of what gets said during social interactions in addition to tracing the patterns of who talks to whom in a social network” [ 28 ].

Ryan et al. [ 29 ] examined automated methods to assess the verbal skills of clinicians and identified several metrics indicative of good discourse, written or verbal communication. We adapted these metrics for team instruction:  Speaker ratio - the equitable distribution of talking time among team members indicates a willingness to listen.  Turn-taking - similar to speaker ratio, turn-taking indicates a wiliness to listen  Overlapping talk - interruption or simultaneous talk may indicate a lack of respect for the contributions of others.  Pauses - number of pauses longer than 2 seconds invites team member contributions and indicates a willingness to listen to others.  Speed of speech - the pace of speech can influence comprehension and indicates a desire to be understood when it the speaker moderates their communications to allow the receiver(s) to fully understand intent.  Energy (pitch and tone) - influences receivers’ perceptions with respect to the engagement and empathy of the speaker.  Plain language – speakers should evaluate word choices, sentence length, and structure to be appropriate with the receivers’ capabilities.  Clinical jargon - the speaker’s choice of terminology and effort to explain technical words indicate a willingness to coach/mentor and be understood by other team members.  Shared decision making - effort to inform, elicit, and integrate preferences of others into decision-making processes.

Sottilare et al. [ 11 ] identified specific behavioral markers that could be identified by LSA. Below is a small sample of teamwork states, their definitions, and a subset of communication (verbal and non-verbal) behavioral markers that influence them:  Trust - the willingness of a team member to be vulnerable to the actions of another team member based on the expectation that the other team member will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other team member [ 30 ]. ─ Opinion-Seeking (verbal) - occurrences of team members actively seeking out the opinions of other team members regarding work tasks. ─ Information Sharing (verbal) - amount of task information share with fellow team members.  Collective efficacy - a shared belief in a group’s capabilities to organize and execute a course of action [ 31 ]. ─ Help-seeking (verbal) - team members actively request backup when needed. ─ Acknowledgment/Recognition (verbal) - team members acknowledge input from other teammates during taskwork, and incorporate their suggestions.  Conflict – the process resulting from tension between team members that is due to real or perceived differences [ 32 ]. ─ Frustration (non-verbal) – team members furrowed their brow, get red faced, or physically agitated. ─ Loudness (verbal) - team members raise their voice when talking with each other or otherwise communicate frustration with the team. ─ Withdrawal (non-verbal) - unwillingness to continue working with someone or on a task. 4

Assessing Teamwork States in GIFT based on Team

Communications

In this section, we review related work and methods for assessing team interactions within the GIFT architecture [ 6-7 ]. As noted in section 3.2, communications data is the key to teamwork assessment and also the most challenging to interpret. We begin the quest to assess teamwork within GIFT tutors by examining the evolution of GIFT team models: single and multi-level.

Single-level Modeling in GIFT

Recently, the US Army examined a simplified approach to modeling teams by using the existing GIFT architecture as a team model. Figure 3 shows the four primary ITS architectural components in green: learner, pedagogical (instructional), domain, and interface modules. The yellow elements define data/information passed between architectural components and to/from the individual learner. To use the GIFT architecture for team training, none of the architectural component nor the principles that govern them have been altered, but the data, shown in orange, is modified to reflect a team training scenario (Figure 4). The learner module remains intact but is relabeled as a team model. The team scenario includes assessment of team states based on initial team model and team data. Individual learner performance becomes an assessment of team performance based on team objectives, and the input of individual team members comprises team inputs to the tutor-user interface. GIFT principles driving strategy recommendations remain the same and are based upon best training practices found in the instructional literature. Tactic selection and presentation remain based on the context defined by the domain module. While this team model works surprising well for team taskwork, the lack of individual learner models for team members means the tutor has no knowledge of the interactions between members. In other words, the GIFT ITS only has knowledge of the team’s objectives, measures associated with those objectives, and progress toward the defined objectives. The goals associated with this team scenario allow for adaptation of content (primarily difficulty level), feedback, and support associated with the task, but not adaptation based upon the state of teamwork (e.g., communication, collaboration, coaching). 4.2

Multi-level Modeling in GIFT

Gilbert et al. [ 33 ] specifically modified the GIFT architecture to support both individual and team models (Figure 5), but simplified teamwork measures to register whether communication occurred/did not occur to enable a largely automated approach to team tutoring. No effort was directed at interpreting any of the communication behaviors and this concept largely focused on taskwork measures and assessments with the development of team domain knowledge file (DKF).

Recommended Practices for Assessing Team Interaction

To fully understand progress toward objectives defined by the task and how well the team is working together toward those objectives, we highly recommend a model of the team that includes assessment of the team communications. Just as team performance is assessed using measures related to taskwork objectives, we recommend team performance is also assessed using measures related to teamwork objectives (e.g., timely resolution of conflict). Critical teamwork assessments should include measures (e.g., behavioral markers) from team interactions in order to assess their impact on teamwork states.

If we think about the learning effect model (LEM) [ 11 ] for individuals and data flow in that model is approximated in Figure 6, then the we might adapt Sottilare’s LEM for teams and represent it as shown in Figure 7. In this updated model of teamwork, GIFT is able to capture interactions within the team under assessment and use their interactions to assess teamwork states.

If we breakdown the process for identifying team interactions and other behaviors, and examine using them as indicators of teamwork states, we need to address methods to acquire the data, process the data (classifying teamwork states) and formulating courses of action ranked by reward (as shown in Figure 8). Fig. 8. Recommended Process for Identifying Teamwork States and Selecting Appropriate AIS

Interventions.

The following provides a set of recommendations for future research and development:  Continue investigating methods to unobtrusively acquire team interaction data.  Improve the accuracy of classification methods to determine discrete teamwork states (e.g., low conflict or moderate workload conditions).  Develop instructional strategies rooted in best practices for teamwork, assess their effectiveness, and adapt strategies as needed.  Continue investigating machine learning methods to select AIS actions to optimize learning, performance, retention, and especially transfer of training per Baldwin and Ford [ 34 ].

Finally, in the spirit of moving forward with the teamwork assessment process, we introduce a concept, perceptual machine learning, which is the use of multiple sensors to acquire data (visual, aural, olfactory, haptic/tactile) about the team and their environment. This might seem to be much the same way that human observers capture information from the environment to classify or predict team states. However, we advocate the use of use this data to evaluate conditions in the environment through both separate sensory channels and through a data fusion process where each sensory channel reinforces the classification/prediction of team states from other sensory channels.

References

1. Burke , S. Using the Science of Teams to Inform GIFT Development . Presentation to the Army Research Laboratory Expert Workshop on Team Tutoring. University of Central Florida. ( 2017 ).

2. Sottilare , R. , Brawner , K. Component Interaction within the Generalized Intelligent Framework for Tutoring (GIFT) as a Model for Adaptive Instructional System Standards . In the Adaptive Instructional System (AIS) Standards Workshop of the 14th International Intelligent Tutoring Systems (ITS) Conference , Montreal, Quebec, Canada. ( 2018 ).

3. Manouselis , N. , Drachsler , H. , Vuorikari , R. , Hummel , H. , Koper , R. Recommender systems in technology enhanced learning . In Recommender systems handbook ,pp. 387 - 415 . Springer, Boston, MA. ( 2011 ).

4. Graesser , A. C. , Hu , X. , Nye , B. , Sottilare , R. Intelligent tutoring systems, serious games, and the Generalized Intelligent Framework for Tutoring (GIFT) . In H. F. O'Neil , E. L. Baker , and R. S. Perez (Eds.), Using games and simulation for teaching and assessment , pp. 58 - 79 . Abingdon, UK: Routledge. ( 2016 ).

5. Graesser , A. C. , Hu , X. , Sottilare , R. Intelligent tutoring systems . In International handbook of the learning sciences , pp. 246 - 255 . Routledge. ( 2018 ).

6. Sottilare , R.A. , Brawner , K.W. , Goldberg , B.S. Holden , H.K. The Generalized Intelligent Framework for Tutoring (GIFT) . Concept paper released as part of GIFT software documentation . Orlando, FL: US Army Research Laboratory - Human Research & Engineering Directorate (ARL-HRED). ( 2012 ). Retrieved from: https://gifttutoring.org/attachments/152/GIFTDescription_0.pdf

7. Sottilare , R. , Brawner , K. , Sinatra , A. Johnston , J. An Updated Concept for a Generalized Intelligent Framework for Tutoring (GIFT) . Orlando, FL: US Army Research Laboratory. ( 2017 ). DOI: 10.13140/RG.2.2.12941.54244.

8. Rupp , A. A. , Gushta , M. , Mislevy , R. J. , Shaffer , D. W. Evidence-centered design of epistemic games: Measurement principles for complex learning environments . The Journal of Technology, Learning and Assessment , 8 ( 4 ). ( 2010 ).

9. Schunk , D. H. , Zimmerman , B . (Eds.). Handbook of self-regulation of learning and performance . Taylor & Francis. ( 2011 ).

10. Shute , V. J. Stealth assessment in computer-based games to support learning . Computer games and instruction , 55 ( 2 ), pp. 503 - 524 . ( 2011 ).

11. Sottilare , R.A. , Burke , C.S. , Salas , E. , Sinatra , A.M. , Johnston , J.H. Gilbert , S.B. Designing Adaptive Instruction for Teams: A Meta-Analysis . International Journal of Artificial Intelligence in Education . ( 2017 ). DOI: 10 .1007/s40593-017-0146-z.

12. Cannon-Bowers , J. A. , Bowers , C. Team development and functioning . In S. Zedeck (Ed.), APA handbook of industrial and organizational psychology , Vol 1 : Building and developing the organization , pp. 597 - 650 . Washington, DC: American Psychological Association. ( 2011 ).

13. Salas , E. Team training essentials: A research-based guide . London: Routledge. ( 2015 ).

14. Dillenbourg , P. What do you mean by collaborative learning? Collaborative-learning: Cognitive and Computational Approaches , 1 , pp. 1 - 15 . ( 1999 ).

15. Salas , E. , Dickinson , T. L. , Converse , S. A. , Tannenbaum , S. I. Toward an understanding of team performance and training . In R. W. Swezey, E. Salas, R. W. Swezey , & E. Salas (Eds.), Teams: Their training and performance , pp. 3 - 29 . Westport: Ablex Publishing. ( 1992 ).

16. Carroll , M. Kokini , C. , Champney , R. , Fuchs , S. , Sottilare , R. , Goldberg , B. Modeling Trainee Affective and Cognitive State Using Low Cost Sensors . In Proceedings of the Interservice/Industry Training Simulation & Education Conference , Orlando, Florida, December 2011 . ( 2011 ).

17. Kokini , C. , Carroll , M. , Ramirez-Padron , R. , Wang , X. , Hale , K. , Sottilare , R. , Goldberg , B. Quantification of Trainee Affective and Cognitive State in Real-time . In Proceedings of the Interservice/Industry Training Simulation & Education Conference , Orlando, Florida, December 2012 . ( 2012 ).

18. Johnston , J. , Sottilare , R. , Sinatra , A. M. , Burke , C. S. (Eds.). Building intelligent tutoring systems for teams: What matters . Emerald Publishing Limited . ( 2018 ).

19. DeCostanza , A. H. , Gamble , K. R. , Estrada , A. X. , Orvis , K. L. Team measurement: Unobtrusive strategies for intelligent tutoring systems . In Building Intelligent Tutoring Systems for Teams: What Matters , pp. 101 - 130 . Emerald Publishing Limited. ( 2018 ).

20. Marks , M. A. , Mathieu , J. E. , Zaccaro , S. J. A temporally based framework and taxonomy of team processes . Academy of management review , 26 ( 3 ), pp. 356 - 376 . ( 2001 ).

21. Freeman , J. , Zachary , W. Intelligent tutoring for team training: Lessons learned from US military research . In Building Intelligent Tutoring Systems for Teams: What Matters , pp. 215 - 245 . Emerald Publishing Limited. ( 2018 ).

22. Sinatra , A. M. , Sottilare , R. Considerations in the Design of a Team Tutor . In Building Intelligent Tutoring Systems for Teams: What Matters , pp. 301 - 312 . Emerald Publishing Limited. ( 2018 ).

23. LeCouteur , A. , Feo , R. Real-time communication during play: Analysis of team-mates' talk and interaction . Psychology of sport and exercise , 12 ( 2 ), pp. 124 - 134 . ( 2011 ).

24. Emmert , P. , Barker , L. L. Measurement of communication behavior . Addison-Wesley Longman Ltd . ( 1989 ).

25. Foltz , P. W. , Martin , M. J . Automated communication analysis of teams. Team Effectiveness in Complex Organizations . ( 2008 ).

26. Sullivan , S. , Warner-Hillard , C. , Eagan , B. , Thompson , R. J. , Ruis , A. R. , Haines , K. , ... Jung , H. S. Using epistemic network analysis to identify targets for educational interventions in trauma team communication . Surgery , 163 ( 4 ), pp. 938 - 943 . ( 2018 ).

27. Shaffer , D. , Ruis , A. Epistemic network analysis: A worked example of theory-based learning analytics. Handbook of learning analytics . ( 2017 ).

28. Ruis , A.R. , Hampton , A.J. , Goldberg , B.S. , Shaffer , D.W.Modeling Processes of Enculturation in Team Training . In Sottilare, R., Graesser , A. , Hu , X. , & Sinatra , A . (Eds.). Design Recommendations for Intelligent Tutoring Systems: Volume 6 - Team Tutoring . Army Research Laboratory, Orlando, Florida. ( 2018 ). ISBN: 978-0-9977257-4-2.

29. Ryan , P. , Luz , S. , Albert , P. , Vogel , C. , Normand , C. , Elwyn , G. Using artificial intelligence to assess clinicians' communication skills . Bmj , 364 , pp. l161 . ( 2019 ).

30. Mayer , R. C. , Davis , J. H. , Schoorman , F. D. ( 1995 ). An integrative model of organizational trust . Academy of Management Review , 20 , pp. 709 - 734 . ( 1995 ).

31. Bandura , A. Self-efficacy: The exercise of control . London: Macmillan Publishers. ( 1997 ).

32. De Dreu , C. K. , Weingart , L. R. ( 2003 ). Task versus relationship conflict, team performance, and team member satisfaction: a meta-analysis . Journal of Applied Psychology , 88 ( 4 ), pp. 741 - 749 . ( 2003 ).

33. Gilbert , S. B. , Slavina , A. , Dorneich , M. C. , Sinatra , A. M. , Bonner , D. , Johnston , J. , ... Winer , E. Creating a team tutor using GIFT . International Journal of Artificial Intelligence in Education , 28 ( 2 ), pp. 286 - 313 . ( 2018 ).

34. Baldwin , T.T. , Ford , J.K. Transfer of training: A review and directions for future research . Personnel psychology , 41 ( 1 ), pp. 63 - 105 . ( 1988 ).