Children’s Socio-Emotional Learning Enhanced by Tangibles for Group Activities Mehdi Rizvi Faculty of Computer Science, Free University of Bozen-Bolzano Piazza Domenicani 3, 39100 Bolzano, Italy, srizvi@unibz.it Abstract. Teaching children how to behave socially and how to identify and react to emotions forms the basis of social-emotional learning. So far little research in interaction design has been devoted to tangible interac- tive objects that can support the scaffolding of children’s socio-emotional learning. This PhD research aims to design and evaluate tangible inter- active objects for enhancing children’s social-emotional learning experi- ence, working in close collaboration with an interdisciplinary team, with experts of interaction design and education. Keywords: Playful Experience Design; Socio-emotional Learning; Tan- gible Interaction 1 Introduction and Research Motivations Learning to communicate means fostering socio-emotional learning (SEL) for connecting with others, exchanging ideas and feelings. In the education domain, there are different SEL guidelines for promoting positive social interactions [14]. A number of them are based on activities for small groups, of 2 to 6 mem- bers. According to [20], 55% of teens spend time every day by exchanging text messages with friends, but only 25% of them spend time with friends outside of school, face-to-face. That and concerns voiced by educators and education researchers [14] indicate why children need SEL synchronous group activities, centered around face to face conversations, at the same time, in the same place. As substantiated in [29], interaction design (ID) can help tackle a number of challenges that are faced in such activities, by balancing usability, user ex- perience and instructional SEL goals, e.g., [3, 4]. In the areas of collaborative learning and cooperative work, ID researchers have extensively explored the role of computer or mobile based solutions for supporting human-to-human inter- action, usually asynchronous, e.g., [8, 9]. However, in educational contexts, SEL also requires unobtrusive support for conversation-based SEL synchronous group activities (SEL group activities, henceforth). In line with [13], tangible ID solu- tions (briefly, tangibles) can provide children with such support, by engaging different children and creating fair SEL opportunities for all. Moreover, tangi- bles also have the potential to assist educators and aid them in the scaffolding 37 of SEL group activities, e.g., by tracing relevant interaction data and displaying them conveniently. This paper and the related research are based on the idea that tangibles can support SEL group activities with children, and assist teachers in the instruc- tional process. Each tangible should target a main instructional goal, related to SEL, and at the same time be usable and enhance its users’ experience. The paper starts by reviewing related work, relevant for the reported research work. The literature analysis points to research questions and directions, which are explored next in the paper, together with the chosen research methodology. Then the paper presents a specific tangible as proof-of-concept ID solution for SEL group activities. The paper concludes by recapping its contributions and reflecting on possible future work. 2 Related Work In spite of their potentials for educational contexts, so far little research in ID has been devoted to tangibles for SEL group activities with children. Most work done is primarily focused on adults in work related meetings instead of children in educational settings. Even so, key ideas behind those ID solutions can aid in the design of tangibles for children. Many such solutions are based on tabletop devices, whereas others are wearables or ID alternative solutions for meetings with adults. Some solutions give direct feedback to groups whereas others choose not to do so. Specific solutions for synchronous group activities in educational contexts exist as well, and can be source of inspiration for the PhD research reported in the paper. All relevant solutions are briefly overviewed in the remainder of this section. 2.1 Interactive Solutions for Meetings Tabletop or screen-based solutions often implement visualisations concerning conversation progress. All such tabletop or screen-based solutions gather different complex data and visualise conversation patterns mainly for adults’ usage rather then focusing on children. Conversation Clock [6], in particular, shows in real-time the conversation history in the form of a spiral time-line projected on the conversation table. Similarly, Second Messenger uses and experimentally evaluated a variety of vi- sualisations to show adults’ participation in order to help them move towards a balanced conversation with equal participation [10]. It had both tabletop visu- alizations or visualizations projected on the wall. Solutions such as Collaborative Workspace or Reflect also visualise conversa- tion patterns. These were tested with adults in [21] and [5], respectively. They are essentially mirroring devices: they aim to inform users about their conversa- tion and leave it to the group (of adults) to reflect upon their feedback and see if anything needs to be changed in conversation behaviours. 38 In [30], authors present a prototype with visualization showing group dy- namics projected on a tabletop. These visualizations include a comparison of the total versus current talk time of each user. It also shows how much attention the speaker is giving to other participants while speaking and in turn how much attention other users are giving to the speaker. Although the system does not have tangible or interactive components, it provides a good way to self reflect and may encourage a user to change their conversation behaviour according to how it is being perceived by others. Besides tabletop or screen-based solutions, alternative solutions also exist in the literature. Grouper is one such solution for coordinating team work [28]. Developed as a proof-of-concept wireless wearable, it is a group coordinator: through sensory cues, it alerts its users to pay attention to the leader or cur- rent speaker of the group. However, users have to wear different modules to get sensory cues. Finally, Interactive Benches [26] also aim to promote collaboration and con- versation. Benches communicate with each other and light up to encourage peo- ple to sit closer. Users are awarded a symbolic reward, namely, a “light show”, in case all suggested benches are occupied by users. 2.2 Interactive Solutions for Meetings with Individual Feedback Some solutions, although still focused on group conversations, only provide feed- back to individuals rather than the whole group. Such solutions give personal feedback not visible to others in order to encourage users to participate more. In [27], a group-conversation device was designed to act as an automated fa- cilitator to support the flow of conversation in 4-member groups. Each group member sits facing a personal screen, indicating the current speaker. There is also an option to give direct textual cues about how to progress further. It is important to note that some “shy” users reported to feel uncomfortable with the visualisation of their under-participation, in line with the empirical results of [10]. Ambient Conversation Support System (ACSS) [17] has a focus on small synchronous group meetings but it is not designed specifically for children. ACSS uses micro-electronics and LEDs to mediate the conversation by signaling if a user has been silent for too long a time. Users may or may not see other users’ LEDs based on the ACSS mode, i.e., private or public. Most importantly, ACSS does not aim at enhancing self-reflection on conversation patterns and hence does not give users any visual or other kind of feedback on the conversation progress. Switch [2] is designed to promote quick and effective decision-making in meet- ings. The system identifies and records the actions, speech and body movements of each user. The chairs of the users have an embedded mechanism to change the temperature of the chair to cold or warm. This is intended to be used as a cue or feedback. It is based on the assumption that colder temperature would encourage calm behavior whereas a warmer temperature would encourage action 39 or debate. At the end of the session, the system notifies the users about their performance via email. 2.3 Technology Enhanced Solutions for Educational Contexts Few tabletop or screen-based solutions focus on kindergarten children and enable them to exchange and manipulate objects through their surface. For example, Ely the Explorer [1] is intended for school children for fostering collaboration through the exchange of physical objects. In this solution, each child uses physical objects such as dolls, rotary knobs and RFID cards to interact with the system, which interacts back with the children through animations. The animations are designed in such a manner to encourage discussion. All such solutions, however, do not aim at conveying specific conversation norms. Other technology-enhanced solutions for educational contexts tend to record and analyse the interaction but avoid giving any live feedback in order to avoid any possible hindrance in the main activity. Specifically, in collaborative learning studies, reported in [23, 24], university students used a shared surface or screen which is touch sensitive, while data, such as verbal communication and gestures, is recorded. Students collaborate through the shared work-space, but they are not given any live input or feedback for their collaboration. In the later of the two studies [24], the teacher however received live information on collaboration through her hand held device called the teacher’s dashboard. As inclusiveness and accessibility for different users is also important, espe- cially for children who are at risk of isolation in social activities at school, the literature of inclusive and accessible design counts several interactive solutions, which are often designed according to their specific users’ needs. For example, in a study [19], head-mounted displays were used to help deaf users localize sound in order to be effectively part of a group conversation or activity. SocialMirror [18] is instead focused on helping individuals with autism by displaying comments and suggestions by family and friends, but the idea can be transferred to team-work settings. Another tool called whatsup [7], although not specifically designed for con- versation based activities, has the potential to be used as such. It is intended to be used as an individual assistive kit for children with autism by promoting ex- plorative learning, experience sharing, time-management etc. Use of such devices in a group setting can be beneficial especially for children who are physically or mentally more challenged as compared to others. 3 Research Questions and Methodology 3.1 Research Questions The goal of the PhD research, as outlined in this paper, is to design tangibles for SEL group activities with children. It is expected that each tangible has its main SEL goal for educational contexts, that require group conversation. The 40 research work has thus been framed around the following two main questions, one considering children, the other considering educators and their requirements for such tangibles. First Research Question. How can we design tangibles that enhance children’s experience with SEL group activities, focusing on conversation? Answering the question requires to set requirements and guidelines for the design of tangibles that can enhance the experience of different children in SEL group activities, revolving around conversation, in educational contexts. Second Research Question. How can we design tangibles that enhance edu- cators’ experience with SEL group activities focusing on conversation? Normally the focus of SEL is on children, and within a classroom environment. Designing tangibles that take care of educators and enhance their experience of children’s SEL requirements means involving educators as well in the design process, and considering their needs. For this reason, educators, besides education experts, were involved in the context of analysis of tangibles and in their evaluation. 3.2 Research Methodology and Initial Research Answers This PhD work follows the User eXperience design methodology with a Lean Ap- proach. Starting with the context of use analysis, it would consist of successive cycles of prototyping and evaluation until satisfactory results are achieved. Keep- ing this in mind, a number of prototypes, each having a certain SEL goal, are being rapidly prototyped, evaluated with domain-experts then evaluated with users. Changes are made in the design according to evaluation results, and the cycle continues. As this research work is focused on technology for children and education contexts, it poses a number of specific challenges and requirements different than those faced by ID solutions for adults and work or leisure contexts. Critical requirements were initially formulated on the basis of the context of use analysis, based on previous studies, e.g., [15] as well as contextual inquiries with educa- tion experts. The requirements were revised in light of evaluation results, and can be distinguished into: usability requirements; user experience requirements; instructional requirements depending on the SEL goals of tangibles. They are briefly listed in the following. Relevant usability requirements for tangibles is that they should be under- standable and effective in teaching SEL goals to children, according to their age group. They should be easy to learn, with minimum training [22]. Tangibles should at the same time enhance the experience of their users with SEL group activities. In the case of children, that means that tangibles should be perceived playful by children in order to make their use engaging for children. Gamification can then be used, as in [12, 11, 15], in order to make children perceive tangibles as playful and engage them in SEL group activities. In the case of teachers, tangibles should be perceived helpful, e.g., they should 41 help teachers assess group conversation behaviours and possible improvements, as well as areas of intervention. Other requirements depend crucially on the SEL goals of tangibles. In gen- eral, since SEL group activities are usually centered around conversation, all tangibles should aid in the achievement of positive conversation habits, but what these are and how they can be helpful, depends on their specific SEL goal. The remainder of this paper advances the design of TurnTalk, a tangible for SEL group activities requiring conversation. The paper uses it as a design case to show how requirements and the above research questions were translated into design choices. 4 The Turn-Talk Design Case The turn-sharing in talking is a base norm for conversation in SEL group activi- ties, but it takes considerable scaffolding support by teachers, mainly. TurnTalk is a tangible that aims to promote children’s awareness of the norm and reflec- tion on conversation patterns so as to stir a balanced conversation, in which all children are engaged. See Fig. 1. In line with the requirements listed above and its design goal, TurnTalk was designed so as to be unobtrusive and playful for children, as well as affordable for them. Details of a TurnTalk prototype are in [25]. The main physical components of TurnTalk are: play-cards; a pentagon- shaped tabletop device; rewards, e.g., plastic coins. The turn-card is a 3D printed card, personalised for each group member and identifying him or her. Each mem- ber plays his or her play-card on the pentagon-shaped tabletop device in Fig. 1 to take his/her turn in talking. The pentagon device hides economical microelec- tronics components: an Arduino UNO micro-controller, sensors and actuators. They enable two things: (1) the interaction with children; (2) the storage and processing of social interaction data, starting with the number of turns taken by each member. In particular, the pentagon displays visual feedback, through LED strips, concerning how often each child has taken turns in speaking with his/her card, and showing the child’s progression in the conversation. The visual feedback is designed so as to be playful, unobtrusive (e.g., without sound as it may interrupt conversations) and yet usable for teachers, watching the group from a distance or analyzing the data later. Moreover, its timing can be adapted by teachers. For instance, the visual feedback could be given during a conversa- tion, at specific moments chosen by the teacher; or it could be delivered only at the end of the conversation-based group activity. Moreover, TurnTalk was designed so as to be portable device for a single educator of a class, and not-expensive for being easily replicated for an entire class using 3D printing and off-the-shelf electronic components. As per the research methodology (explained in section 3.2) selected for this research work, the TurnTalk prototype is already evolving by going through repeated cycles of design and evaluation. The device was initially tested with 42 Fig. 1. The initial design of TurnTalk prototype and its implementation experts of ID and child development and then it was tested with children in a controlled study. 5 Conclusions and Expected Research Advancements This paper gives the research motivations, context and questions of my PhD. In order to make my research questions concrete, this paper also presents the design case of TurnTalk, a tangible for the scaffolding of the turn-sharing norm in groups of children. The research so far conducted for my PhD, published in [16, 25] shows that unobtrusive technology and play can assist in supporting and enhancing SEL group activities with children. Forthcoming evaluation studies with children will allow me to evaluate design choices and directions further. The research so far conducted, however, has already highlighted the impor- tance of an even more general and ambitious goal for my PhD research and tangible design in general: how to design tangibles that “teach and disappear” so that the benefits of tangibles are still available if the support of technology progressively fades, in line with the challenges for ID put forward in [29]. References 1. Africano, D., Berg, S., Lindbergh, K., Lundholm, P., Nilbrink, F., Persson, A.: Designing tangible interfaces for children’s collaboration. In: CHI ’04 Extended Abstracts on Human Factors in Computing Systems. pp. 853–868. CHI EA ’04, ACM, New York, NY, USA (2004), http://doi.acm.org/10.1145/985921.985945 2. Alessandrini, L., Ben-Haim, N., Choi, R.J., Soler, E.: Switch. Retrieved online from http://elodiesoler.com/#/swiitch/ (2016) 3. Alrifai, M., Gennari, R., Vittorini, P.: Adapting with Evidence: The Adaptive Model and the Stimulation Plan of TERENCE, pp. 75–82. Springer Berlin Heidel- berg, Berlin, Heidelberg (2012) 4. Alrifai, M. and Gennari, R. and Tifrea, O. and Vittorini, P.: The User and Domain Models of the TERENCE Adaptive Learning System, pp. 83–90. Springer Berlin Heidelberg, Berlin, Heidelberg (2012) 43 5. Bachour, K., Kaplan, F., Dillenbourg, P.: An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning. IEEE Transactions on Learning Technologies 3(3), 203–213 (July 2010) 6. Bergstrom, T., Karahalios, K.: Conversation clock: Visualizing audio patterns in co-located groups. In: Proceedings of the 40th Annual Hawaii International Confer- ence on System Sciences. pp. 78–. HICSS ’07, IEEE Computer Society, Washington, DC, USA (2007), http://dx.doi.org/10.1109/HICSS.2007.151 7. Chang, H.T., Wang, Y., Jensen, M.T.L.: Whatsup. Retrieved on- line from http://ciid.dk/education/portfolio/idp11/courses/tangible-user- interface/projects/whatsup/ (2016) 8. Collazos, C.A., Guerrero, L.A., Pino, J.A., Ochoa, S.F.: A Method for Evaluating Computer-Supported Collaborative Learning Pro- cesses. Int. J. Comput. Appl. Technol. 19(3/4), 151–161 (May 2004), http://dx.doi.org/10.1504/IJCAT.2004.004044 9. De Marsico, M., Sterbini, A., Temperini, M.: Assessing Group Composition in e- learning According to Vygotskij’s Zone of Proximal Development, pp. 277–288. Springer International Publishing, Cham (2014) 10. DiMicco, J.M., Hollenbach, K.J., Pandolfo, A., Bender, W.: The Impact of In- creased Awareness While Face-to-face. Hum.-Comput. Interact. 22(1), 47–96 (May 2007), http://dl.acm.org/citation.cfm?id=1466595.1466598 11. Dodero, G., Gennari, R., Melonio, A., Torello, S.: Towards tangible gamified co- design at school: Two studies in primary schools. In: Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play. CHI PLAY ’14, ACM, New York, NY, USA (2014) 12. Dodero, G., Gennari, R., Melonio, A., Torello, S.: Gamified Co-design with Co- operative Learning. In: CHI ’14 Extended Abstracts on Human Factors in Com- puting Systems. pp. 707–718. CHI EA ’14, ACM, New York, NY, USA (2014), http://doi.acm.org/10.1145/2559206.2578870 13. Dourish, P.: Embodied Interaction: Exploring the Founda- tions of a new Approach to HCI. Retrieved online from http://www.ics.uci.edu/ jpd/publications/misc/embodied.pdf (1999) 14. Durlak, J.A., Weissberg, R.P., Dymnicki, A.B., Taylor, R.D., Schellinger, K.B.: The impact of enhancing student’s social and emotional learning: A meta-analysis of school-based universal interventions. Child Development 82(1), 405–432 (2011), http://dx.doi.org/10.1111/j.1467-8624.2010.01564.x 15. Gennari, R., Melonio, A., Torello, S.: Gamified Probes for Cooperative Learn- ing: a Case Study. Multimedia Tools and Applications pp. 1–25 (2016), http://dx.doi.org/10.1007/s11042-016-3543-7 16. Gennari, R., Pavani, F., Rizvi, M.: Tangible design for inclusive conversations with?deaf or hard-of-hearing children. In Proceedings of the 1st International Sym- posium on Emerging Technologies for Education (SETE 2016), LNCS, Springer, Rome (2016) 17. Hirai, Y., Kaneko, K.: Ambient conversation support in small face-to-face group meetings. In: Proceedings of the Sixth International Symposium on Information and Communication Technology. pp. 239–246. SoICT 2015, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2833258.2833274 18. Hong, H., Kim, J.G., Abowd, G.D., Arriaga, R.I.: Designing a social net- work to support the independence of young adults with autism. In: Pro- ceedings of the ACM 2012 Conference on Computer Supported Coopera- tive Work. pp. 627–636. CSCW ’12, ACM, New York, NY, USA (2012), http://doi.acm.org/10.1145/2145204.2145300 44 19. Jain, D., Findlater, L., Gilkeson, J., Holland, B., Duraiswami, R., Zotkin, D., Vogler, C., Froehlich, J.E.: Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 241–250. CHI ’15, ACM, New York, NY, USA (2015), http://doi.acm.org/10.1145/2702123.2702393 20. Lenhart., A.: Teens, technology and friendship, pew research center. Re- trieved online from http://www.pewinternet.org/2015/08/06/teens-technology- and-friendships/ (2016) 21. Leonardi, C., Pianesi, F., Tomasini, D., Zancanaro, M.: Computers in the Human Interaction Loop, pp. 187–205. Springer London, London (2009) 22. Levin Gelman, D.: Design for Kids. Rosenfeld Media (2014) 23. Martinez, R., Kay, J., Wallace, J.R., Yacef, K.: Modelling Symmetry of Activity as an Indicator of Collocated Group Collaboration, pp. 207–218. Springer Berlin Heidelberg, Berlin, Heidelberg (2011) 24. Martinez Maldonado, R., Kay, J., Yacef, K., Schwendimann, B.: An Interactive Teacher’s Dashboard for Monitoring Groups in a Multi-tabletop Learning Envi- ronment, pp. 482–492. Springer Berlin Heidelberg, Berlin, Heidelberg (2012) 25. Melonio, A., Rizvi, M.: The design of turntalk for the scaffolding of balanced con- versations in groups of children. In Proceedings of the 1st International Symposium on Emerging Technologies for Education (SETE 2016), LNCS, Springer, Rome (2016) 26. Remin, J.S., Ignac, M., Kjaergard, S.: Interactive benches. Retrieved online from http://ciid.dk/education/portfolio/py/courses/physical- computing/projects/interactive-benches/ (2016) 27. Schiavo, G., Cappelletti, A., Mencarini, E., Stock, O., Zancanaro, M.: Overt or subtle? supporting group conversations with automatically targeted di- rectives. In: Proceedings of the 19th International Conference on Intelligent User Interfaces. pp. 225–234. IUI ’14, ACM, New York, NY, USA (2014), http://doi.acm.org/10.1145/2557500.2557507 28. Shaw, F.W., Klavins, E.: Grouper: a Proof-of-Concept Wearable Wireless Group Coordinator. In: Proceedings of the 12th ACM International Conference Adjunct Papers on Ubiquitous Computing. pp. 379–380. ACM (2010) 29. Slovák, P., Fitzpatrick, G.: Teaching and developing social and emotional skills with technology. ACM Trans. Comput.-Hum. Interact. 22(4), 19:1–19:34 (Jun 2015), http://doi.acm.org/10.1145/2744195 30. Terken, J., Sturm, J.: Multimodal support for social dynamics in co- located meetings. Personal Ubiquitous Comput. 14(8), 703–714 (Dec 2010), http://dx.doi.org/10.1007/s00779-010-0284-x 45