Adoption of teaching analytics systems at early school levels is scarce. To increase the uptake of these systems by teachers, there is a need of initiatives that enable teachers and designers work together, to allow designers understand teachers' needs. This paper reports on the initial findings of a study where nineteen teachers were interviewed to elicit their demands in the context of a technologically enabled classroom setup. The results show that teachers demand indicators related to their daily tasks in the context of formal education, and suggest design features that resemble their current practices.
In spite of the increasing interest in the field of learning analytics, there is a scarcity
of research on this field for elementary and early secondary education [
The design, development and evaluation of learning analytics systems aimed for
teachers to understand learning and teaching processes is known as teaching analytics
[
The project Epa’T (Espace Protégé pour l’Apprentissage en ses Traces) [
The rest of the paper is structured as follows: the next sections reviews relevant literature related to teachers’ adoption of learning analytics tools. Then, Section 3 introduces the project Epa’T, that sets the context of the study and the long-term research goals. Section 4 describes the implementation of the study and its results. Finally, section 5 presents the conclusions and future work lines. 2
The adoption and integration of technology in schools is a complex endeavor,
comprising many entangled factors. Models such as TPACK (Technological Pedagogical
and Content Knowledge) [
Being able to reach the highest levels of integration depends on a number of
intertwined personal, institutional and technological factors [
However, there are symptoms that the field of learning analytics is not offering this
kind of tools to teachers, specially those at the initial educational levels. In spite of the
increasing investment in learning analytics, the presence of real-world examples of
primary and secondary school teacher’s uptake of this technology is almost non
existent. For example, in the last international conference on learning analytics (LAK’17)
[
There is therefore, a need for investigating and reporting on the users’ needs for teaching analytics’ systems, based on projects that facilitate co-design processes where researchers sit together with teachers to understand their needs. The project Epa’T which is briefly introduced below, was proposed with this vision. 3
The project Epa’T (Oct. 2015 – Sept. 2016) [
The project takes a bottom-up perspective based on action-research, with the overall goal of analyzing the consequences of the tools’ use in everyday teaching practices and their acceptability by teachers. 21 teachers (7 in primary, 14 in secondary) and 251 pupils (153 in primary, 98 in secondary) were involved in this pilot.
The study presented in this paper is part of the initial, exploratory phases of the project, aiming to identify teachers’ points of view about the aforementioned topics. 4 4.1
The study [
At the moment when the study was carried out, the technical infrastructure and applications envisioned in the program had not been deployed yet. Therefore, the study did not aim at evaluating any specific feature of the technical infrastructure that was being deployed in the schools. It aimed at identifying the needs expressed by the teachers in a more general way, and more concretely, to identify the needs of teachers for implementing personalized learning activities in their classrooms. 4.2
Methods and Research Question
To take into account the practices and needs of teachers in the personalization of
learning, the research team chose to follow a DBR (Design-Based Research)
approach, which involves an iterative approach to both testing and defining innovative
learning environments responding to real needs, while defining new principles. This
study is part of the first, exploratory phase of the first DBR cycle [
The interviews were conducted with teachers from the two aforementioned academies, with the following distribution: for Academy 1, eleven teachers from the first year of the same secondary school were interviewed, although one of these interviews could not be processed later on and was discarded. All the teachers were part of the aforementioned implementation program. For Academy 2, eight teachers from different elementary schools were interviewed, six of which were participants in the implementation program, while the other two were external to the program. These two teachers were chosen to check if the needs they expressed were different from those of the teachers that where involved in the project.
All teachers participated on a voluntary basis in the study. The interviews with
teachers from Academy 1 took 30 minutes, because the time for the interview was
shared with another research being done in the frame of the Epa’T project, while the
interviews in Academy 2 where fully devoted to this study, and lasted one hour in
average. The responses were summarized in grids organized by question and by
teacher [
The interviews consisted of three parts: an introduction, with close questions about the school, the class, and the teacher; a second part aimed at eliciting the teachers’ current practices with and without technology; and a last part where teachers were asked about the information they would like to have to help them personalize activities to the needs of their learners. In Academy 2, a protocol that aimed at helping teachers externalize their needs was followed: the teachers were asked to propose a situation of personalization with an application devoted to the design of lesson plans for the development of competences. Then, teachers were asked about which information they considered important for implementing this situation.
To answer the research question posed in this paper, i.e., what kind of support do teachers expect from a computerized classroom? we analyzed the summaries of the responses focusing on the following issues: a) What kind of contribution do teachers expect from a technological setup in their classrooms? b) What information do the teachers demand to follow their learners’ activities and assess their competences? We were also interested in finding out whether it was possible to observe differences in the type of information requested by the teachers that were participating in the implementation program and those that were not. In spite of their very exploratory nature, we consider these issues as important to understand better teachers’ views regarding the use of learning analytics tools in their classrooms. 4.3
This section summarizes the findings from the interviews, organized according to the three issues that structured the analysis (see section 4.2). No attempt to quantify the results is presented, in accordance with the qualitative and exploratory nature of the study. For each result presented, we point out which teachers mentioned that aspect in the interview. With this, we aim to provide supporting evidence for the results. We will use the code A1_n for teachers of Academy 1, and A2_n for teachers of the Academy 2, being n the number of that teacher within each group.
some teachers, specially in Academy 1 (A1_5, A1_6, and A1_8) expressed their personal doubts about the utility of digital tools for their classes, due to the amount of preparation work required, and poor mastery of tools), most of the teachers expressed positive expectations about the role of technology for: • Adapting the difficulty of the activities according to the learner's profile (A1_1,
A1_2, A1_4, A1_5, A1_6, A1_8, A1_9, A2_1, A2_5). • Adapting the type of activity to the learners’ profile, for example by adapting the texts for dyslexics (A1_2, A1_4, A2_2, A2_3). • Monitoring their learners' activities (in and out of class), including successes or failures and progression, evaluations and behavior (individual or collaborative) (A2_1, A2_2, A2_5, A2_6). • Intervening directly on the production of learners by sending personalized messages (A1_6, A1_11, A2_6, A2_7).
expressed by teachers can be classified according to the following three axes: supervision of the activities inside and outside the classroom, assessment of competences, and proposals specific to a subject matter.
Regarding the supervision of the activities, teachers envisioned the need of the following information: • For the activities carried out at the classroom: progression of the learners through the activities, including: number of exercises made (and not made) (A1_2, A1_3, A2_4, A2_5, A2_6, A2_8); number of attempts to make an exercise (A2_2, A2_8); time spent in an exercise (A1_3, A2_2, A2_8); visualization and remediation of mistakes (A1_1, A1_2, A1_3, A2_1, A2_2, A2_3, A2_4, A2_5). Regarding mistakes, some teachers suggested to show a synopsis table to visualize the main errors encountered together with the number of pupils concerned (in order to estimate whether it is necessary to make common or more individual corrections) (A2_1, A2_3, A2_5). Two teachers stated that the review of the activities was to be done off-line, during the evening (A1_9, A1_11). Another set of requirements focused on the need of controlling the learners’ activity while using computers in the classroom, by supervising their screens (A1_8, A1_9, A1_11), visualizing whether they were actually connected and working in the activities (A1_9, A1_11), and enabling direct interaction with the learners through messages (A1_11). • For the out-of-classroom activities (lessons to be reviewed at home, videos with questionnaires for the flipped classroom, or classroom blog), the teachers that were already using this approach demanded to know which activities had been carried out (A1_3, A2_6), length of time that the learners had been connected (A2_5), and whether the families had accessed the classroom blog (A1_8). • Several proposals were focused on time: time spent actually working on the task (A1_2, A2_2, A2_4, A2_8); time elapsed between the activity had been proposed and the activity was carried out (A2_8). • Few teachers expressed the need to measure activities made in groups using codes of participation (by learner and by group) (A1_6, A2_4, A2_5).
As regards the assessment of competences, the participant teachers proposed different formats of organizing the information: • For each subject, visualize the progress for the different competences of each learner (A1_1, A1_3, A1_5, A1_6, A2_4), or of the whole classroom (A1_4, A2_1, A2_4). • Visualize the level of acquisition of a competence by learners compared with their class (A1_4, A1_5) and by a class, compared with the average of the school (A1_5).
Overall, teachers requested to be able to follow the evolution of the competences by different criteria (by learner, by subject, by cycle). Some teachers that were already using systems based on paper (such a color code: green / yellow / red), suggested to use a similar code for the envisioned system (A2_1, A2_8).
Finally, some teachers demanded more specific information, related to their subject matter. For example, one teacher of Physical Education (A1_5), suggested to get information of the cardiac rhythm of the pupils while doing exercise and the distance run; one language teacher (A1_1) proposed to record the learners’ oral exercises and make them aware of their mistakes; another one (A2_5) expressed the need that the system monitored the quality of the pupils’ written productions (use of punctuation, structure of sentences, ...).
Overall, we can observe that teachers demanded indicators that helped them supervise the activity of their learners, in ways that would help attend the requests of the formal educational system (supervision of the activities, errors’ remediation, assessment of competences, etc.). Other demands point out to the need of controlling what happens when computers are used in the classroom (e.g., know what learners actually do at their computers, either at the classroom or at their homes). It is interesting that some teachers pointed out that the supervision of the activities should be done off-line during the evening. This points out to a critical issue: the lack of time that teachers have while working with the learners at the classroom.
Differences between participant and non-participant teachers. Against what could have been expected, we could not observe any meaningful difference between the kind of responses given by teachers already participating in the program and the external teachers from Academy 2 that participated in the interview. One likely reason for this lack of differentiation is the fact that the implementation phase of the project had not been yet started and therefore, the participant teachers had not had the opportunity to develop new ideas about the support from technology. It remains as future work to carry out similar comparisons, when the participant teachers had already worked with the technological setup. 5
In order to gain further impact in schools, specially at the primary and middle educational levels, learning analytics has to invest in field work in which teachers and designers develop a common understanding on the possibilities of technology to support teachers’ work.
The results of the exploratory work presented in this paper show that teachers
demand information tightly connected to the tasks they have to fulfill within the formal
educational system. Teachers envision ways of visualizing information that are close
to their current practices, like e.g., the use of color-codes that match the ones they
already use without computers. It is also important to take into account the tight time
restrictions teachers have while they are working with their learners, as well as the
resources they have available, to propose solutions that are affordable for them. These
findings are aligned with the conclusions reported in [
Following the DBR cycle initiated by the study presented in this paper, future work plans include defining a set of indicators derived from the results, and testing them with the teachers participating in the program. A second line of future research will be to analyze whether these teachers, after working with the implementation of the program and the proposed indicators, are able to envision new ways in which technology can support their practices, and help them implement them. The long-term goal is to analyze whether initiatives like the Epa’T project allow teachers to move from a model where technology substitutes existing methods to a model where technology is used to modify or redefine their pedagogy -using the terms of the SAMR model-, leading to more efficient use of ICT and of learning analytics in the schools.
Acknowledgements The authors thank the participant teachers for their fruitful collaboration. This research has been partially supported by a grant as invited professor offered by the University of Franche-Comté to Alejandra Martínez Monés, by a grant from the Ministry of Education, Spain (PRX16/00480), by Junta de Castilla y León (VA082U16), and Ministerio de Economía y Competitividad (TIN2014-53199-C3-2R, and TIN2015-71669-REDT). Data and research material have been collected thanks to the financial support of the Digital National Education (DNE) of the French Ministry of Education.