The aim of this experiment was to evaluate a tangible environment called PrisMe (Collective attention led by a Mediated environment) developed during the e-TAC project. The methodology for creating this environment was based on an end-user-centered approach. Its role is to support collective work and foster collaborative learning in an educational context (i.e., primary and secondary school). More specifically, this set of artifacts must promote noise regulation during group activities, facilitate pupils' time management and support classroom management for teachers. The evaluation of PrisMe took place with pupils aged 9 to 10 in a real school context. Pupils worked in groups of 4 and completed mathematics problem-solving activities lasting approximately 20 minutes. At the end of these activities, they were given two user experience questionnaires, the AttrakDiff and the meCUE. The user feedback allowed us to validate a significant number of interactions and brought out points of vigilance for the final development phase. The results of the questionnaires are encouraging and seem to indicate that PrisMe provides a satisfactory user experience to pupils.
In line with UNESCO or OECD recommendations, K-16 educational programs stress worldwide the
need to build the 21st century cross-disciplinary competencies needed by all for personal fulfillment
and development, social inclusion, and active citizenship [
Nault and Fijalkow [20] describes classroom management as “the way in which interactions between a teacher and students in the classroom are regularized in a formal or informal way, even when they are unexpected situations or reactions.” This definition, which is broader than just disciplinary phenomena, encompasses what Nault and Fijalkow [20] have identified as the building blocks of classroom management. The e-TAC project aims to develop technologies that respond to real needs identified by teachers and learners, the end users. By developing technologies that offer a positive user experience, we aim to facilitate their implementation and adoption. By evaluating our technical solutions at an early stage of design, we want to validate the interactions we will develop in the next iterative loops. To support classroom management during group activities, we propose PrisMe, a set of tangible artifacts designed to encourage the regulation and autonomy of students working together.
We aim to develop a solution located close to the students and which provides direct feedback,
making the consequences of their individual actions on the group perceptible to everyone [
Many previous works evidenced the potential of TUI to support learning or collaboration (e.g., [16,
24]). A recent study using eye tracker devices showed that participants working collaboratively on
problem-solving showed more visual attention when working with tangible objects than with a more
conventional mouse-screen-keyboard system [25]. Kim and Maher [15] showed that the collaboration
fostered by TUI could improve creative results compared to a Graphical User Interface (GUIs). In
comparison to PrisME, a software such as classroomscreen.com, is designed to support student
selfregulation while facilitating pedagogical orchestration [
PrisMe is a tangible environment that might support self-regulation during the group work at school and thus facilitate the classroom management.
Our design approach relies heavily on the expertise of the teachers we work with. One of the first steps in our work has been to organize focus groups to gather students' group work experiences. We were able to identify four main difficulties of regulation among the students. These difficulties have an impact on classroom management. The first is that the pupils have difficulty to have a dialogue that allows them to make decisions collectively. There is no real group dynamic and individualistic behaviors are observed. This leads to the second difficulty, which is the over-solicitation of teachers. When a student has a question, he or she prefers to ask it directly to the teacher while the answers are often within the groups themselves. The third problem is that the pupils often find themselves rushing to get to the end in order to be on time. Teachers must constantly remind students to keep moving forward. The last challenge is that pupils are not aware of the noise they make. As time passes, the noise level increases. The teachers have to stop the activity until the noise level is low in order to resume under bearable conditions. 3.2.
The first recommendation we proposed is that the system should help pupils to become aware of the noise they make so that they can regulate themselves. If a tolerance level is reached, the system must require them to make less noise. Then, the tool must help students make decisions collectively by listening and respecting everyone's opinion. They must be able to vote and agree before calling the teacher or returning their work. To encourage them to ask fewer but more relevant questions, the system should limit the number of questions they can ask. Finally, our system must make them aware of the time allocated to an activity so that they can organize their work accordingly.
This section presents all the devices’ early design we have proposed. These artefacts should allow us to validate the interactions we had imagined before going through all the development phases. Based on the teachers’ feedback, we imagined a set of artifacts that could support group work called PrisMe. It is composed of six elements. Each one is related to one of the issues raised by the teachers. • Prisms (figure 3): allows students, in a group, to work together to take a decision, vote, call the teacher, or signal that the work is finished. • A lamp (figure 4): allows the teacher to know a group of students' status: everything is okay, needs help, finished work... • A Bluetooth relay (figure 4): serves as a bridge between each group member's prism and the lamp. • An interactive sound levels meter (figure 5): allows a group to self-regulate itself concerning noise during work. • A question coupon (figure 4): allows the teacher to define the number of questions the students are allowed to ask. The teacher or the students themselves can choose this number. •
A stopwatch (figure 4): allows the teacher to set the time remaining in the activity. The teacher or the students themselves can choose the value.
Each pupil has a prism (figure 3) allowing him/her to vote on actions involving important collective decisions. To start an activity the pupils, symbolize their belonging to the group by putting their objects beside each other. They can then use them to take part in the group's decisions by voting for or against their peers' proposals. Pupils can perform four actions with their prisms using four touch zones: (1) I agree with the proposal made by a classmate (2) I do not agree with the proposal made by a classmate (3) They can also jointly call the teacher to limit individual questions (4) Finally, they can agree to declare their activity finished when it suits everyone.
The second element is the Bluetooth relay (figure 4) whose role is to make the connection between all the students' prisms and the lamp. So, when the students call the teacher or declare that they have finished the activity via their prisms, the lamp lights up to signal this. The second element is a mechanical stopwatch (figure 4) with two functions. The first one is to define the time the activity will last and the second one is to define a red zone representing the time left before the end of the activity. Finally, the colored marbles (figure 4) is a question coupon that represents the number of questions the students are allowed to ask during the activity. In this way, they are encouraged to find the answers to their questions within the group.
Finally, the pumpkin-shaped sound levels meter (Figure 5) allows to have color feedback according to the sound level. These ambient feedbacks should make it easier to become aware of the noise generated during the activity. If this is not enough, the pumpkin emits a sound signal to indicate that the maximum noise level tolerated by the teacher has been exceeded.
This contribution aims to assess the usability of this TUI through a preliminary study in real school context. We wanted to ensure that several PrisMe are usable by the pupils in group work, and that they are able to implement the related interactions (agree among themselves, call the teacher and declare that they have finished the activity) in an effective, efficient and satisfactory way. Before measuring its impacts in terms of learning and regulation of activities, it was necessary to ensure that students would not be blocked in using PrisMe to regulate themselves in their work.
Participants. 12 pupils (5 girls, 7 boys; aged 9-11 years; Mage=9.57, SDage=0.5) from a primary school were involved in a test day. The teacher formed 4 groups of 3 pupils similar in age and grade level.
Tasks & activities. The task assigned to the students was to solve two mathematical problems dealing with concepts expected from the French curriculum. They were judged by their teacher to be within the reach of the students involved. These problems were selected in consultation with the working group on primary school of the “Institut de Recherche sur l'Enseignement des Mathématiques” in the Lorraine region in France.
Experimental protocol. Each group spent 20 minutes doing one activity, took a break, then 20 minutes doing a second activity. To counterbalance the order of activities, half of the groups carried out Activity 1 and then Activity 2 and, the other half did the reverse. In each group, identical instructions preceded the activities. After each activity with PrisMe, the short version of the AttrakDiff [12] and the meCUE [18] were completed by the pupils.
AttrakDiff is one of the most common for evaluating UX. For Hassenzahl [
meCUE is based on the user experience model of Thüring and Mahlke [28]. In order to limit the fatigue of the pupils, we have chosen to pass only module 1 “Product perceptions” and 2 “Emotions.” These dimensions complete the AttraDiff results, as the user’s status is evaluated by the meCUE. It questions the impact of the use of the device on the perception that others have of the user. As PrisMe is designed to change the place of individuals within the group, these dimensions seem particularly relevant. 4.2.
AttrakDiff. The results show that the device is well rated in terms of hedonic and pragmatic qualities. The testers gave the device as many hedonic as pragmatic qualities. The confidence interval shows that the system can be classified as desirable (Figure 6 top). The scores for hedonic qualities (1.73), pragmatic qualities (1.56) and attractiveness (1.75) are above average and can be qualified as positive.
meCUE. The meCUE data confirm the results of the AttrakDiff. The averages for “usability,” “usefulness,” and “visual aesthetics” are 4.62 (1.66), 5.70 (1.22), and 5.58 (1.28) respectively (see figure 6). They are therefore all above average and confirm the positive evaluation of our system with room for improvement. A very encouraging result provided by the meCUE compared to the AttrakDiff is the "status" score, which allows us to know whether users felt that using our system changes the point of view of others about their status within the group (Figure 6 below). Indeed, the average score on the status dimension is 4.49 (1.08), indicating that children who have used PrisMe perceived their status within the group more favorably because of its use.
We have put in place a design process focused on user needs for the development of PrisME. This process allowed us to make recommendations that resulted in the creation of tangible artifacts to support the regulation of group work. Preliminary results seem to indicate that pupils enjoy interacting with our artefacts. This evaluation at an early design stage allowed us to validate the interactions we proposed. However, our work should be taken in perspective since several biases were identified. The first being the desirability bias of pupils who like anything technology-related and like anything out of the ordinary at school. In addition, it is very difficult to do an evaluation in a school context. The equipment available, the size of the rooms, the habits of the pupils, makes the exercise perilous. These results allow us to positively conclude this first iteration that aimed at validating user satisfaction during a user experience study with pupils in a real use context.
For now, some of the PrisME elements are interconnected which means that they cannot be used independently of one another. Other elements are not connected. This is the case for example with the marbles. Our future work will consist first in transforming these passive objects into connected ones, and second to make interconnected objects eventually independent of one other. In this case, a teacher will be able to use only one device without having to use all the environment. PrisME would thus represent a modular tangible interfaces that we make available to teachers. Moreover, PrisME would allow to record interaction data that will allow us to provide pupils’ behavioral indicators for teachers. They will thus be able to know the sound level of a group during an activity, the number of questions asked, or whether they succeeded in managing their time efficiently.
This study is part of the e-TAC project supported by the French Ministry of National Education and Youth within the framework of the eFRAN funding of the Future Investment Plan. Thanks to the pupils and their teacher with the support of the academic inspection and of the head of the school. [12] Hassenzahl, M., Burmester, M. and Koller, F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. Springer, City, 2003. [13] Hornecker, E. and Buur, J. Getting a grip on tangible interaction: a framework on physical space and social interaction. ACM, City, 2006. [14] Ishii, H. and Ullmer, B. Tangible bits: towards seamless interfaces between people, bits and atoms. ACM, City, 1997. [15] Kim, M. J. and Maher, M. L. The impact of tangible user interfaces on spatial cognition during collaborative design. Design Studies, 29, 3 (2008), 222-253. [16] Konkel, M. K., Ullmer, B., Shaer, O. and Mazalek, A. Envisioning tangibles and display-rich interfaces for co-located and distributed genomics collaborations. In Proceedings of the Proceedings of the 8th ACM International Symposium on Pervasive Displays (Palermo, Italy, 2019). Association for Computing Machinery, [insert City of Publication],[insert 2019 of Publication]. [17] Marshall, P. Do tangible interfaces enhance learning? ACM, City, 2007. [18] Minge, M. and Riedel, L. meCUE-Ein modularer fragebogen zur erfassung des nutzungserlebens. Mensch & Computer 2013: Interaktive Vielfalt (2013). [19] Montroy, J. J., Bowles, R. P., Skibbe, L. E. and Foster, T. D. Social skills and problem behaviors as mediators of the relationship between behavioral self-regulation and academic achievement. Early Childhood Research Quarterly, 29, 3 (2014/07/01/ 2014), 298-309. [20] Nault, T. and Fijalkow, J. Introduction. La gestion de la classe: d’hier à demain. Revue des sciences de l’éducation, 25, 3 (1999), 451-466. [21] Neuenschwander, R., Röthlisberger, M., Cimeli, P. and Roebers, C. M. How do different aspects of self-regulation predict successful adaptation to school? Journal of Experimental Child Psychology, 113, 3 (2012/11/01/ 2012), 353-371. [22] OECD. Students, Computers and Learning. OECD Publishing, 2015. [23] Resnick, L. B., Levine, J. M. and Behrend, S. D. Socially shared cognition. American Psychological Association Washington, DC, 1991. [24] Scharf, F., Winkler, T., Hahn, C., Wolters, C. and Herczeg, M. Tangicons 3.0: an educational noncompetitive collaborative game. ACM, City, 2012. [25] Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P. and Pea, R. D. 3D tangibles facilitate joint visual attention in dyads. City, 2015. [26] Scott, C. L. The Futures of learning 2: what kind of learning for the 21st century? - UNESCO Digital Library. Education, research and foresight: working papers https://unesdoc.unesco.org/ark:/48223/pf0000242996_fre (2015), 14. [27] Suthers, D. D. Representational guidance for collaborative learning. Amsterdam: IOS Press, City, 2003. [28] Thüring, M. and Mahlke, S. Usability, aesthetics and emotions in human–technology interaction. International journal of psychology, 42, 4 (2007), 253-264. [29] Wagener, B. L’autorégulation conjointe de la cognition et des émotions: quel impact sur les apprentissages? Voix Plurielles, 12, 1 (2015), 82-103.