=Paper=
{{Paper
|id=Vol-1925/paper02
|storemode=property
|title=What Information Do Teachers Demand from a Computerized Classroom? An Exploratory Analysis
|pdfUrl=https://ceur-ws.org/Vol-1925/paper02.pdf
|volume=Vol-1925
|authors=Alejandra Martínez-Monés,Christophe Reffay,Gwénaëlle Lécuyer-Cabioch,Vanda Luengo
|dblpUrl=https://dblp.org/rec/conf/lasi-spain/Martinez-MonesR17
}}
==What Information Do Teachers Demand from a Computerized Classroom? An Exploratory Analysis==
https://ceur-ws.org/Vol-1925/paper02.pdf
What information do teachers demand from a
computerized classroom? An exploratory analysis
Alejandra Martínez-Monés1,2, Christophe Reffay1, Gwénaëlle Lécuyer-Cabioch3 and
Vanda Luengo3
ELLIADD, FR-EDUC, University Bourgogne Franche-Comté
1
2
GSIC-EMIC. Universidad de Valladolid, Valladolid, Spain
3 MOCAH. LIP6 - Laboratoire d'Informatique de Paris 6, Paris, France
amartine@infor.uva.es, christophe.reffay@univ-fcomte.fr,
vanda.luengo@lip6.fr
Abstract. Adoption of teaching analytics systems at early school levels is
scarce. To increase the uptake of these systems by teachers, there is a need of
initiatives that enable teachers and designers work together, to allow designers
understand teachers’ needs. This paper reports on the initial findings of a study
where nineteen teachers were interviewed to elicit their demands in the context
of a technologically enabled classroom setup. The results show that teachers
demand indicators related to their daily tasks in the context of formal education,
and suggest design features that resemble their current practices.
Keywords: Teaching Analytics · Teachers’ ICT adoption · Primary Education ·
Early Secondary Education
1 Introduction
In spite of the increasing interest in the field of learning analytics, there is a scarcity
of research on this field for elementary and early secondary education [1]. Apart from
some exceptions (see e.g., [2][3]), there are very seldom cases reported of successful
use of learning analytic tools at these educational levels. One likely reason for this
lack of adoption is the complexity of implementing such technologies in a way that
supports teachers’ pedagogical needs and adapts to their restrictions, which is also a
limitation for the overall use of ICT, specially at early school levels [4].
The design, development and evaluation of learning analytics systems aimed for
teachers to understand learning and teaching processes is known as teaching analytics
[5]. As stated by researchers in this area [5][6], to gain further impact, there is a need
of joint work by teachers and designers, that would facilitate a better understanding of
the teachers’ needs by learning analytics designers, and the initiation by the teachers
of cycles of innovation that would lead them to integration strategies where technolo-
gy is used to improve their pedagogical practices.
The project Epa’T (Espace Protégé pour l’Apprentissage en ses Traces) [7] was set
up with the goal of providing teachers with efficient ICT support based on usable
Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
technology enriched with learning analytics, with the overall intention of helping to
fill the distance between what is being proposed nowadays by the learning analytics
tool designers and current school practices. This paper reports on the initial findings
of a study that took place shortly after the project started. The objective of the study
was to identify with teachers of primary and early secondary education the indicators
that would allow to personalize the learning of their students in a computerized envi-
ronment. Contextualized interviews with nineteen primary and middle school teachers
of two academic regions in France were conducted to explore their ideas on how
technology could help them personalize learning in their classes. Based on the teach-
ers’ responses to these interviews, but taking a broader perspective, the research ques-
tion addressed in this paper is: What kind of support do teachers expect from a com-
puterized classroom?
The rest of the paper is structured as follows: the next sections reviews relevant lit-
erature related to teachers’ adoption of learning analytics tools. Then, Section 3 intro-
duces the project Epa’T, that sets the context of the study and the long-term research
goals. Section 4 describes the implementation of the study and its results. Finally,
section 5 presents the conclusions and future work lines.
2 Adoption of Learning Analytics at Early School Levels
The adoption and integration of technology in schools is a complex endeavor, com-
prising many entangled factors. Models such as TPACK (Technological Pedagogical
and Content Knowledge) [8] or SAMR (Substitution, Augmentation, Modification
and Redefinition) [9] define elements for successful adoption, as well as different
levels of integration of ICT into teachers’ practices. According to these models, it is
feasible to think that the road for a successful uptake of technology by teachers can
start from experiences where technology adapts to their current practices, helping
teachers understand the possibilities of the design space. This will eventually enable
them to envision more innovative practices supported by ICT, and thus, achieve a
more efficient and effective integration of digital technologies in their classrooms.
Being able to reach the highest levels of integration depends on a number of inter-
twined personal, institutional and technological factors [10]. Among all these factors,
teachers’ beliefs about teaching and their expectations on the role of technology have
been considered as the most influential for successful uptake [11]. In turn, these ex-
pectations depend, among other factors, on what is offered to them by the available
technology. If teachers perceive ICT as a beneficial tool, compatible with their current
activities, easy to use and with observable outcomes, they will demonstrate positive
attitudes towards ICT [1] [12].
However, there are symptoms that the field of learning analytics is not offering this
kind of tools to teachers, specially those at the initial educational levels. In spite of the
increasing investment in learning analytics, the presence of real-world examples of
primary and secondary school teacher’s uptake of this technology is almost non exist-
ent. For example, in the last international conference on learning analytics (LAK’17)
[13], only 1 out of 65 research papers reported a study of a learning analytics tool
being used by a teacher in a primary school [3]. One likely reason for this problem, as
stated by Fergusson et al., [1] is the fact that the offer is still based on the supply side,
i.e., it consists of tools and methods provided by learning analytics’ experts, which do
not take sufficiently into account the many questions and restrictions met by teachers
in their practice [5]. This gap has been also noted by Holstein et al., [6] in their recent
work towards identifying teachers’ needs for the design of Intelligent Tutoring Sys-
tems (ITS) that overcome the difficulties that these systems pose for teachers’ adop-
tion.
There is therefore, a need for investigating and reporting on the users’ needs for
teaching analytics’ systems, based on projects that facilitate co-design processes
where researchers sit together with teachers to understand their needs. The project
Epa’T which is briefly introduced below, was proposed with this vision.
3 The Project Epa’T
The project Epa’T (Oct. 2015 – Sept. 2016) [7] concerns an experimentation, in pilot
primary and early secondary schools of one of the two academies involved in the
innovation program (Academy 1), for providing Chromebooks equipped with Google
Classroom1 integrated with Hapara2 as a monitoring tool. Administrative territories’
technical services where actively involved in the technical support of the hardware
infrastructure (networks’ connections, Chromebooks management and software con-
figuration). Pedagogical support was provided by the DANE (Délégation Académique
du Numérique Educatif).
The project takes a bottom-up perspective based on action-research, with the over-
all goal of analyzing the consequences of the tools’ use in everyday teaching practices
and their acceptability by teachers. 21 teachers (7 in primary, 14 in secondary) and
251 pupils (153 in primary, 98 in secondary) were involved in this pilot.
The study presented in this paper is part of the initial, exploratory phases of the
project, aiming to identify teachers’ points of view about the aforementioned topics.
4 Investigating Teachers’ Desires and Needs
4.1 Context of the Study
The study [14] was carried out in two administrative regions (academies) in France,
involving 19 teachers from the third cycle of the French educational system (com-
prising learners between 9 and 12 years). The two academies were engaged in a pilot
program of use of digital tools by pupils for school learning, from which the project
Epa’T (see Section 3) was a concrete implementation.
1
Google classroom website: https://classroom.google.com Last visit 12-4-2017
2
Hapara web site: https://hapara.com/products/g-suite/ Last visit 12-4-2017
At the moment when the study was carried out, the technical infrastructure and ap-
plications envisioned in the program had not been deployed yet. Therefore, the study
did not aim at evaluating any specific feature of the technical infrastructure that was
being deployed in the schools. It aimed at identifying the needs expressed by the
teachers in a more general way, and more concretely, to identify the needs of teachers
for implementing personalized learning activities in their classrooms.
4.2 Methods and Research Question
To take into account the practices and needs of teachers in the personalization of
learning, the research team chose to follow a DBR (Design-Based Research) ap-
proach, which involves an iterative approach to both testing and defining innovative
learning environments responding to real needs, while defining new principles. This
study is part of the first, exploratory phase of the first DBR cycle [15]. The overall
approach is qualitative, involving a small number of teachers, and the focus was to
understand teachers’ point of view, not to obtain generalizable results.
The interviews were conducted with teachers from the two aforementioned acade-
mies, with the following distribution: for Academy 1, eleven teachers from the first
year of the same secondary school were interviewed, although one of these interviews
could not be processed later on and was discarded. All the teachers were part of the
aforementioned implementation program. For Academy 2, eight teachers from differ-
ent elementary schools were interviewed, six of which were participants in the im-
plementation program, while the other two were external to the program. These two
teachers were chosen to check if the needs they expressed were different from those
of the teachers that where involved in the project.
All teachers participated on a voluntary basis in the study. The interviews with
teachers from Academy 1 took 30 minutes, because the time for the interview was
shared with another research being done in the frame of the Epa’T project, while the
interviews in Academy 2 where fully devoted to this study, and lasted one hour in
average. The responses were summarized in grids organized by question and by
teacher [14].
The interviews consisted of three parts: an introduction, with close questions about
the school, the class, and the teacher; a second part aimed at eliciting the teachers’
current practices with and without technology; and a last part where teachers were
asked about the information they would like to have to help them personalize activi-
ties to the needs of their learners. In Academy 2, a protocol that aimed at helping
teachers externalize their needs was followed: the teachers were asked to propose a
situation of personalization with an application devoted to the design of lesson plans
for the development of competences. Then, teachers were asked about which infor-
mation they considered important for implementing this situation.
To answer the research question posed in this paper, i.e., what kind of support do
teachers expect from a computerized classroom? we analyzed the summaries of the
responses focusing on the following issues: a) What kind of contribution do teachers
expect from a technological setup in their classrooms? b) What information do the
teachers demand to follow their learners’ activities and assess their competences? We
were also interested in finding out whether it was possible to observe differences in
the type of information requested by the teachers that were participating in the im-
plementation program and those that were not. In spite of their very exploratory na-
ture, we consider these issues as important to understand better teachers’ views re-
garding the use of learning analytics tools in their classrooms.
4.3 Results
This section summarizes the findings from the interviews, organized according to the
three issues that structured the analysis (see section 4.2). No attempt to quantify the
results is presented, in accordance with the qualitative and exploratory nature of the
study. For each result presented, we point out which teachers mentioned that aspect in
the interview. With this, we aim to provide supporting evidence for the results. We
will use the code A1_n for teachers of Academy 1, and A2_n for teachers of the
Academy 2, being n the number of that teacher within each group.
What contribution do teachers expect from a technological setup? Although
some teachers, specially in Academy 1 (A1_5, A1_6, and A1_8) expressed their per-
sonal doubts about the utility of digital tools for their classes, due to the amount of
preparation work required, and poor mastery of tools), most of the teachers expressed
positive expectations about the role of technology for:
• Adapting the difficulty of the activities according to the learner's profile (A1_1,
A1_2, A1_4, A1_5, A1_6, A1_8, A1_9, A2_1, A2_5).
• Adapting the type of activity to the learners’ profile, for example by adapting the
texts for dyslexics (A1_2, A1_4, A2_2, A2_3).
• Monitoring their learners' activities (in and out of class), including successes or
failures and progression, evaluations and behavior (individual or collaborative)
(A2_1, A2_2, A2_5, A2_6).
• Intervening directly on the production of learners by sending personalized messag-
es (A1_6, A1_11, A2_6, A2_7).
What kind of information do the teachers demand from the system? The needs
expressed by teachers can be classified according to the following three axes: supervi-
sion of the activities inside and outside the classroom, assessment of competences,
and proposals specific to a subject matter.
Regarding the supervision of the activities, teachers envisioned the need of the fol-
lowing information:
• For the activities carried out at the classroom: progression of the learners through
the activities, including: number of exercises made (and not made) (A1_2, A1_3,
A2_4, A2_5, A2_6, A2_8); number of attempts to make an exercise (A2_2, A2_8);
time spent in an exercise (A1_3, A2_2, A2_8); visualization and remediation of
mistakes (A1_1, A1_2, A1_3, A2_1, A2_2, A2_3, A2_4, A2_5). Regarding mis-
takes, some teachers suggested to show a synopsis table to visualize the main er-
rors encountered together with the number of pupils concerned (in order to esti-
mate whether it is necessary to make common or more individual corrections)
(A2_1, A2_3, A2_5). Two teachers stated that the review of the activities was to be
done off-line, during the evening (A1_9, A1_11). Another set of requirements fo-
cused on the need of controlling the learners’ activity while using computers in the
classroom, by supervising their screens (A1_8, A1_9, A1_11), visualizing whether
they were actually connected and working in the activities (A1_9, A1_11), and en-
abling direct interaction with the learners through messages (A1_11).
• For the out-of-classroom activities (lessons to be reviewed at home, videos with
questionnaires for the flipped classroom, or classroom blog), the teachers that were
already using this approach demanded to know which activities had been carried
out (A1_3, A2_6), length of time that the learners had been connected (A2_5), and
whether the families had accessed the classroom blog (A1_8).
• Several proposals were focused on time: time spent actually working on the task
(A1_2, A2_2, A2_4, A2_8); time elapsed between the activity had been proposed
and the activity was carried out (A2_8).
• Few teachers expressed the need to measure activities made in groups using codes
of participation (by learner and by group) (A1_6, A2_4, A2_5).
As regards the assessment of competences, the participant teachers proposed dif-
ferent formats of organizing the information:
• For each subject, visualize the progress for the different competences of each
learner (A1_1, A1_3, A1_5, A1_6, A2_4), or of the whole classroom (A1_4,
A2_1, A2_4).
• Visualize the level of acquisition of a competence by learners compared with their
class (A1_4, A1_5) and by a class, compared with the average of the school
(A1_5).
Overall, teachers requested to be able to follow the evolution of the competences
by different criteria (by learner, by subject, by cycle). Some teachers that were al-
ready using systems based on paper (such a color code: green / yellow / red), suggest-
ed to use a similar code for the envisioned system (A2_1, A2_8).
Finally, some teachers demanded more specific information, related to their subject
matter. For example, one teacher of Physical Education (A1_5), suggested to get in-
formation of the cardiac rhythm of the pupils while doing exercise and the distance
run; one language teacher (A1_1) proposed to record the learners’ oral exercises and
make them aware of their mistakes; another one (A2_5) expressed the need that the
system monitored the quality of the pupils’ written productions (use of punctuation,
structure of sentences, ...).
Overall, we can observe that teachers demanded indicators that helped them super-
vise the activity of their learners, in ways that would help attend the requests of the
formal educational system (supervision of the activities, errors’ remediation, assess-
ment of competences, etc.). Other demands point out to the need of controlling what
happens when computers are used in the classroom (e.g., know what learners actually
do at their computers, either at the classroom or at their homes). It is interesting that
some teachers pointed out that the supervision of the activities should be done off-line
during the evening. This points out to a critical issue: the lack of time that teachers
have while working with the learners at the classroom.
Differences between participant and non-participant teachers. Against what
could have been expected, we could not observe any meaningful difference between
the kind of responses given by teachers already participating in the program and the
external teachers from Academy 2 that participated in the interview. One likely reason
for this lack of differentiation is the fact that the implementation phase of the project
had not been yet started and therefore, the participant teachers had not had the oppor-
tunity to develop new ideas about the support from technology. It remains as future
work to carry out similar comparisons, when the participant teachers had already
worked with the technological setup.
5 Conclusions and Future Work
In order to gain further impact in schools, specially at the primary and middle educa-
tional levels, learning analytics has to invest in field work in which teachers and de-
signers develop a common understanding on the possibilities of technology to support
teachers’ work.
The results of the exploratory work presented in this paper show that teachers de-
mand information tightly connected to the tasks they have to fulfill within the formal
educational system. Teachers envision ways of visualizing information that are close
to their current practices, like e.g., the use of color-codes that match the ones they
already use without computers. It is also important to take into account the tight time
restrictions teachers have while they are working with their learners, as well as the
resources they have available, to propose solutions that are affordable for them. These
findings are aligned with the conclusions reported in [6] for ITS systems and in [17]
for inquiry-based learning, and underline the need for more holistic approaches to the
implementation of ICT at these elementary levels, taking also into account the specif-
ic data privacy issues that appear when working with minors, as also noted by [16].
Following the DBR cycle initiated by the study presented in this paper, future work
plans include defining a set of indicators derived from the results, and testing them
with the teachers participating in the program. A second line of future research will be
to analyze whether these teachers, after working with the implementation of the pro-
gram and the proposed indicators, are able to envision new ways in which technology
can support their practices, and help them implement them. The long-term goal is to
analyze whether initiatives like the Epa’T project allow teachers to move from a mod-
el where technology substitutes existing methods to a model where technology is used
to modify or redefine their pedagogy -using the terms of the SAMR model-, leading to
more efficient use of ICT and of learning analytics in the schools.
Acknowledgements The authors thank the participant teachers for their fruitful colla-
boration. This research has been partially supported by a grant as invited professor
offered by the University of Franche-Comté to Alejandra Martínez Monés, by a grant
from the Ministry of Education, Spain (PRX16/00480), by Junta de Castilla y León
(VA082U16), and Ministerio de Economía y Competitividad (TIN2014-53199-C3-2-
R, and TIN2015-71669-REDT). Data and research material have been collected
thanks to the financial support of the Digital National Education (DNE) of the French
Ministry of Education.
References
1. Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B.,
Ullmann, T., Vuorikari, R.: Research Evidence on the Use of Learning Analytics: Implica-
tions for Education Policy. (2016).
2. Ebner, M., Schön, M.: Why learning analytics in primary education matters. Bull. Tech.
Comm. Learn. Technol. 15, 14–17 (2013).
3. Mutahi, J., Kinai, A., Bore, N., Diriye, A., Weldemariam, K.: Studying engagement and
performance with learning technology in an African classroom. In: Proceedings of the
Seventh International Learning Analytics & Knowledge Conference. pp. 148–152. ACM
(2017).
4. Somekh, B.: Factors affecting teachers’ pedagogical adoption of ICT. Int. Handb. Inf.
Technol. Prim. Second. Educ. 449–460 (2008).
5. Dyckhoff, A.L., Zielke, D., Bültmann, M., Chatti, M.A., Schroeder, U.: Design and im-
plementation of a learning analytics toolkit for teachers. Educ. Technol. Soc. 15, 58–76
(2012).
6. Holstein, K., McLaren, B.M., Aleven, V.: Intelligent tutors as teachers’ aides: exploring
teacher needs for real-time analytics in blended classrooms. In: Proceedings of the Seventh
International Learning Analytics & Knowledge Conference. pp. 257–266. ACM (2017).
7. Schneewele, M., Reffay, C.: Bilan du projet EPA’T : Espace Protégé pour l’Apprentissage
en ses Traces. Université de Franche-Comté, Université Bourgogne Franche-Comté
(2017).
8. Mishra, P., Koehler, M.J.: Technological pedagogical content knowledge: A framework
for teacher knowledge. Teach. Coll. Rec. 108, 1017 (2006).
9. Puentedura, R.: Transformation, Technology, and Education,
http://hippasus.com/resources/tte/, (2006).
10. Buabeng-Andoh, C.: Factors influencing teachers’ adoption and integration of information
and communication technology into teaching: A review of the literature. Int. J. Educ. Dev.
Using Inf. Commun. Technol. 8, 136 (2012).
11. Hermans, R., Tondeur, J., van Braak, J., Valcke, M.: The impact of primary school teach-
ers’ educational beliefs on the classroom use of computers. Comput. Educ. 51, 1499–1509
(2008).
12. Rogers, E.M.: Diffusion of innovations. Simon and Schuster (2010).
13. Wise, A., Winne, P.H., Lynch, G. eds: LAK ’17: Proceedings of the Seventh International
Learning Analytics & Knowledge Conference. ACM, New York, NY, USA (2017).
14. Lécuyer-Cabioch, G.: Quels indicateurs pour personnaliser les apprentissages dans une
école numérique ?, (2016).
15. Reeves, T.C.: Design research from a technology perspective. Educ. Des. Res. 1, 52–66
(2006).
16. Rodríguez-Triana, M.J., Martínez-Monés, A., Villagrá-Sobrino, S.: Learning analytics in
small-scale teacher-led innovations: Ethical and data privacy issues. J. Learn. Anal. 3, 43–
65 (2016).
17. Rodríguez-Triana, M.J., Holzer, A., Vozniuk, A., Gillet, D.: Orchestrating inquiry-based
learning spaces: An analysis of teacher needs. In: International Conference on Web-Based
Learning. pp. 131–142. Springer (2015).