=Paper=
{{Paper
|id=Vol-2712/paper7
|storemode=property
|title=How Educators Value Design Analytics for Blended Learning (extended abstract)
|pdfUrl=https://ceur-ws.org/Vol-2712/paper07.pdf
|volume=Vol-2712
|authors=Laia Albó,Davinia Hernández-Leo
|dblpUrl=https://dblp.org/rec/conf/ectel/AlboH19
}}
==How Educators Value Design Analytics for Blended Learning (extended abstract)==
https://ceur-ws.org/Vol-2712/paper07.pdf
How Educators Value Design Analytics for
Blended Learning
Laia Albó1[0000-0002-7568-9178] and Davinia Hernández-Leo1[0000-0003-0548-7455]
1
Universitat Pompeu Fabra, Barcelona, Spain
laia.albo@upf.edu,davinia.hernandez-leo@upf.edu
Extended abstract
Keywords: Design Analytics, Blended Learning, Authoring tool, Learning Design
With the increased use of technology in education, educational data science has
become more accessible, providing tools and techniques for making sense of
educational data [1]–[3]. On the other hand, blended educational practices are
complex and combine face to face -f2f- instruction with online activities supported by
technology. Thus, nowadays, educators and researchers could benefit from data
extracted from blended educational practices for analysing and reflecting about the
characteristics of the educational settings they have designed [3]. The data collected
can serve different purposes, depending on their nature. Whereas data about students’
interactions can be used to understand their learning processes and experiences
(learning analytics), metrics of design decisions and related aspects characterizing
learning designs (design analytics) can provide awareness and reflection on decisions
made during the learning design process as well as inform future design decisions
[1][4]. Moreover, the metrics and patterns of design activity within a community of
teachers and related stakeholders (community analytics) can provide awareness and
reflection about individual and collective design activities and trigger orientation and
inspiration about how to improve the design practices [1], [5].
In this paper, we explore the opinions of teachers regarding the use of design
analytics, probably the least explored type of data in educational technology together
with community analytics. Specifically, we analyse the value that design analytics can
offer in authoring experiences using the edCrumble learning design tool [6].
Three workshops were carried out in different contexts to give participants the
opportunity of interacting with the tool and evaluating the design analytics provided.
The first workshop took place on a teaching innovation conference (April 2018) and
participants were 23 teachers, researchers and educational technology-related
stakeholders (9 female). Whereas the second and third workshops were placed within
a collaboration with two schools in the frame of a research project (May-June 2018).
In this case, participants were 14 high school teachers, half of them from each school
community. Each workshop (of 90 minutes) consisted of a role-playing game where
participants were organized in groups of 2-4 people. Each group of participants
represented an imaginary school and each participant of each group represented a
teacher in charge of a subject matter (simulating different educational communities).
The role-playing game had two main parts (individual and in groups). Each part had
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
54
three steps. The individual activity (at “imaginary” teacher-role level) consisted of the
following tasks: (1) Design of a short teaching unit with the edCrumble online
version—a printed LD was provided by the researchers for each teacher role; (2)
Analyse the data resulting from the elaborated design, and (3) Share the design
created within the edCrumble community. Whereas the group activity (at “imaginary”
school-role level) implied: (1) Grouping several designs to generate community
analytics; (2) Solving an educational challenge; and (3) Discussing results with all
participants.
The education challenge proposed at the group level was asking teachers to use
community analytics for balancing the out-of-class workload between the different
designs they created which were part of a whole curriculum to be worked with the
same cohort of students (within the simulated school community). The objective was
first, to analyse how many hours of homework they had given to the students in total,
counting all the designs generated by the school (the community analytics allowed the
participants to see the aggregated out-of-class hours of all the designs automatically).
And second, they were invited to reduce the total hours of work outside the classroom
(if there were many) to a certain number that they would consider appropriate,
debating what strategy to follow.
At the end of the workshop in one of the schools (n=7), researchers asked
participants to fill in a research questionnaire individually (the questionnaire could not
be used in the other workshops due to lack of time). The questionnaire (which was
anonymous) had three main questions for evaluating the design analytics of
edCrumble and, for each of them, participants indicated their level of agreement
(using a five-point Likert scale). They were invited to provide an optional comment or
open response. Moreover, participants of both schools (n=14) were asked to evaluate
design analytics (individually) based on several factors (following a five-point Likert
scale of agreement). Finally, the design artefacts resulting from all workshops were
analysed, checking whether participants were able to generate the community
analytics correctly and whether they successfully solved the educational challenge
proposed.
After analysing the results, we can state that design analytics have the potential of
supporting teachers during the design process facilitating data-based decision making.
The results have shown positive attitudes and opinions of teachers regarding the use
of different types of design analytics as they think analytics can help them to improve
their designs. Moreover, teachers valued that design analytics could facilitate the
reflection during the design process as well as provide them support for achieving a
design continuity between the in-class and out-of-class activities. Lastly, aggregated
design analytics from multiple designs across educators in a community (community
analytics) have also been valued offering design support and awareness at the school
community level, with the potential of facilitating the design coordination among
teachers of the same students’ cohort.
Acknowledgements. The authors would like to thank all the participants who
participated in the workshops as well as Alex Young Pedersen for revising the
manuscript. This work has been partially funded “la Caixa Foundation” (CoT project,
100010434) and FEDER, the National Research Agency of the Spanish Ministry of
55
Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R,
TIN2017-85179-C3-3-R. DHL is a Serra Húnter Fellow.
References
[1] D. Hernández-Leo, R. Martinez-Maldonado, A. Pardo, J. A. Muñoz-Cristóbal, and
M. J. Rodríguez-Triana, “Analytics for learning design: A layered framework and
tools,” Br. J. Educ. Technol., vol. 51, no. 1, pp. 139–152, 2019.
[2] I. Amarasinghe, D. Hernández-Leo, and A. Jonsson, “Data-informed design
parameters for adaptive collaborative scripting in across-spaces learning
situations,” User Model. User-adapt. Interact., in press.
[3] K. Michos, D. Hernández-Leo, and L. Albó, “Teacher-led inquiry in technology-
supported school communities,” Br. J. Educ. Technol., vol. 49, no. 6, pp. 1077–
1095, 2018.
[4] L. Albó, J. Barria-pineda, P. Brusilovsky, and D. Hernández-Leo, “Concept-Level
Design Analytics for Blended Courses,” in ECTEL2019, in press.
[5] K. Michos and D. Hernández-Leo, “Supporting awareness in communities of
learning design practice,” Comput. Human Behav., vol. 85, pp. 255–270, 2018.
[6] L. Albó and D. Hernández-Leo, “edCrumble: designing for learning with data
analytics,” in Lifelong Technology-Enhanced Learning. EC-TEL 2018. Lecture
Notes in Computer Science, vol 11082., 2018, pp. 605–608.