=Paper=
{{Paper
|id=Vol-1596/paper6
|storemode=property
|title=Data Literacy and Use for Learning When Using Learning Analytics for Learners
|pdfUrl=https://ceur-ws.org/Vol-1596/paper6.pdf
|volume=Vol-1596
|authors=Barbara Wasson,Cecilie Hansen,Grete Netteland
|dblpUrl=https://dblp.org/rec/conf/lak/WassonHN16
}}
==Data Literacy and Use for Learning When Using Learning Analytics for Learners==
https://ceur-ws.org/Vol-1596/paper6.pdf
Data Literacy and Use for Learning
when using Learning Analytics for Learners
Barbara Wasson Cecilie Hansen Grete Netteland
Centre for the Science of Learning and Technology for Practice Department of Social Sciences
Technology (SLATE) & InfoMedia Uni Research Health Sogn and Fjordane University College
University of Bergen, Bergen, Norway Bergen Norway Sogndal, Norway
+4755585420 +4755584191 +4757676000
barbara.wasson@uib.no cecilie.hansen@uni.no grete.netteland@hisf.no
ABSTRACT 2. BACKGROUND
In this position paper we suggest that Learning Analytics for In our perspective there are three areas of research are relevant for
Learners (LAL) requires new digital competences of learners. LAL: Open Learner Models; Learning Analytics; and, Visual
The use of Open Learner Models, Learning Analytics, and Visual Analytics. An Open Learner Model (OLM) [7] is an artefact that
Analytics creates new opportunities for learners, but our own is well suited to visualise competence models of student’s
research indicates that digital competence to take advantage of learning, and learning analytics can create the evidence to be
these opportunities needs to be extended to specifically address modelled in the learner model that the OLM visualised, and can
data literacy and use. We outline how our framework for data also be used to analyse students’ use of the OLM. Each of these
literacy and use for teaching can guide research to develop a areas is briefly presented below in the context of the recently
framework for data literacy and use for learning, and identify a completed EU NEXT-TELL project, our nationally funded project
number of research questions. iComPAss, and our focus on data literacy and use for learning in
our research Centre for the Science of Learning and Technology
CCS Concepts (SLATE).
•Applied computer→ Education •Human-centred computing→
Visualisation→ Visualization application domains •General and
reference→ •Empirical studies •Design •Information systems→ Open Learning Models (OLM): In the Next-Tell project we
•Data mining •Human centered computing→ •User models •User used an Open Learner Model [7] to facilitate both self-reflection
studies •Usability testing •HCI theory, concepts and models on the part of learners, and teacher planning and decision- making
•Interactive systems and tools •Visual analytics •Student ([8], [9]). An OLM is similar to student progress and performance
assessment reports, however, it does more than report progress, it models and
externalises competences and skills. Built on a learner model
(LM), an OLM traditionally is a representation of a learner’s skill
Keywords and abilities inferred while the learner interacts with learning
Open Learning Models; Learning Analytics; Visual Analytics; material in an intelligent tutoring system (ITS). State-of-the-art
Competences; Data literacy and use. research is extending the use of OLMs (beyond ITSs) to situations
where there are multiple data sources, both automatic and manual,
feeding its learner model ([10], [11], [2]), and visualising the
1. INTRODUCTION learner’s competences in various ways ([2]; [8]). The OLM thus
Technology rich classrooms (e.g., [1]) offer new pedagogical facilitates reflection, planning, self-assessment and self-directed
possibilities, new ways of learning, and generate new types of learning [7], In our iComPAss project we investigate the use of
data that can be used both for assessment and for improving an OLM in professional competence development for firefighters
teaching and learning [2]. While these technology and and in the education of leaders for healthcare management, and in
information-rich classrooms enable 21st Century pedagogy (e.g., SLATE we continue the work from NEXT-TELL in investigating
[3], [4]), they also place new demands and require new the use of OLMs with students.
competence of teachers and learners. That is, these data-rich work
environments require new knowledge, skills, and abilities to lever Learning Analytics and Data Mining: Learning Analytics (LA)
the possibilities in, and beyond these classrooms. Accordingly, “is the use of intelligent data, learner-produced data, and analysis
teacher capacity development for using ICT and data for their models to discover information and social connections, and to
students’ learning and for their own professional development
predict and advise on learning.” and as such has a predictive
([5], [6]) needs to be fostered. Furthermore, there are new nature. LA has been used to provide data for various stakeholders,
demands on students to use and understand data and its such as teachers, school leaders, policy makers, and even for
visualisation, for learning. Open Learner Models, Learning learners themselves. Important for LA to be useful for leaners is
Analytics, and Visual Analytics can be used to provide evidence visualisations. In NEXT-TELL Kickmeir-Rust et al. [12] used LA
(data) to students for self-reflection and this position paper to analyse several learning tools and input the evidence in the
outlines what we mean needs to be explored with respect to data OLM.
literacy and use for learners when these models and methods are
taken into use with learners.
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes.
This volume is published and copyrighted by its editors. LAL 2016 workshop at LAK '16, April 26, 2016, Edinburgh, Scotland.
Visual Analytics: Visual analytics is “the science of analytical use these tools various types of data that are generated through the
reasoning facilitated by interactive visual interfaces.” ([13], [14] ). use of the tools are tracked and stored as evidence of use. These
Applying visual analytics methods to education and training has might include log files such as Chat logs from OpenSim or traces
the potential to enable decision makers, including learners, to see of who wrote what in a Google doc, a document such as the final
and explore learner data in order to have evidence for decision text written in Google doc, the item responses (i.e., answers
making. In iComPAss and SLATE we investigate this potential. given) to a test, a video saved to an e-portfolio in Moodle, or a
There are multitudes of ways in which data can be visualised and grid of responses to a Repertory Grid exercise (e.g., RGFA tool
our research will focus on how best to visual the competences to [15]), or an activity visualisation of one’s participation in
support self-reflection in an individual learner. OpenSim. The collected evidence can either be used directly or
be processed electronically by an assessment analysis tool such as
PRONIFA that uses the CbKST algorithm ([12]) or manually by
We identify another important challenge that LAL raises – the teacher (e.g., evaluate the video created by a student)
understanding the new competence demands on learners when transforming the evidence into information that can be used to
using LA for learning. Thus, we are interested in a dialogue at update the student’s competence model in the NEXT-TELL
LAL on the competences that learners need in order to understand Independent Open Learner Model ([16]). Ideally as much as
OLMs, the use of LA, and the visualisations that are presented to possible is automated.
them either through dashboards or other means.
In SLATE we will be carrying out research to identify these
competences in order to understand better data literacy and use
related to using data for learning. We are building on the work we
have carried out in NEXT-TELL with respect to data literacy and
use for teaching [5]. This is presented briefly after the
introduction of the NEXT-TELL vision of the information and
technology rich classroom.
3. THE NEXT-TELL CLASSROOM
As indicated in the introduction of this paper, the insurgence of
use of technology for learning has increased the amount and types
of data available in technology and information rich classrooms,
and tools to handle this data are emerging. A NEXT-TELL
classroom is such a classroom where a plethora of data is
generated through the use of digital tools and services. Figure 1
can be used to explain this vision.
Figure 2. Teacher’s view of a NEXT-TELL Classroom
We can look at this process from a teacher perspective, see figure
2. The teacher uses a planner to specify learning activities and
assessments (fully digital if possible). The activities and
assessments get delivered automatically to the students, or the
teacher manually delivers them to the students (e.g., a test on
paper). As the student engages in the learning activity or
assessments data is being recorded for further analysis by either
another tool or the teacher manually. The analysis of the data is
PR feed into the independent OLM where the student’s digital
competence model is updated and visualised ([16]). The teacher
can use this student information to adjust their pedagogical
practice to an individual student (view an individual competence
model) or to the entire class (view the competence model of the
class). Using this teacher perspective and our experience in
working with teachers we identified and identify data literacy and
use skills that teachers would have to develop for a NEXT-TELL
classroom.
Figure 1. NEXT-TELL Data Flow
4. DATA LITERACY AND USE FOR
A Next-Tell classroom is also a learning environment where tools TEACHING AND LEARNING
such as Moodle (LMS), Google Docs, Immersive Environments
(such as Second Life (secondlife.com) or OpenSim From our experience in using NEXT-TELL tools with high school
(opensimulator.org)), formative assessment tools, digital science teachers and their students over the past two years we
educational games, or electronic tests can be found. As students
suggest that the existing definitions of digital competence need to certainly relevant, especially when these are complex issues,
be extended to specifically address data literacy and use. which students and teachers may never entirely understand and
In NEXT-TELL we explored the knowledge, skills, and abilities thus protection must be provided by policy.
required to make effective use of the new kinds of data and
information available for teaching, assessment, and diagnosing
learning in the technology and information rich classroom. Based 5. CONCLUSIONS
on these findings we developed a framework that encompasses the As education begins to embrace new trends such as educational
various aspects of data literacy and use required by teachers, and data mining, learning analytics, visual analytics, and big data,
illustrate these using examples from the NEXT-TELL project [5]. there will be even more intricate data literacy and use skills that
learners (and teachers) will need to develop. Our future research
In Wasson & Hansen [5] we present a framework for data literacy
on the role of these approaches in education should be cognisant
and use for teaching that encompasses the teacher’s need to have
of the impact on digital competence for learners and can use the
an understanding of
framework for data literacy and use for learning to identify how
(1) how the configuration of technology tools or leaners need to be trained on the new approaches and the tools
applications impacts the data generated (conceptually that embrace them. Finally, research is needed to understand the
and technically), impact on using student data for learners; does it improve
(2) what and how data is generated by a tool or learning?
application,
(3) how the data is analysed (if it is done automatically), We believe that these issues are important for those participating
(4) how the data/data interpretations can be used in a in the LAL workshop to discuss.
pedagogical manner (for both teaching and formative
assessment), and
(5) how the data and data interpretations can be shared. 6. ACKNOWLEDGMENTS
This research is funded by the Research Council of Norway grant
number 246765/H20 and the Centre for the Science of Learning
We propose that this framework serve as the basis for developing and Technology (SLATE), University of Bergen, Norway.
a framework for data literacy and use for learning. The points in
the existing framework will be taken one by one and examined for
its relevance for a framework for data literacy and use for
learning. For example, with respect to (1) we would need to use 7. REFERENCES
the technology tools (e.g., an OLM or an assessment tool) with [1] Groff, J. (2013). Technology Rich Innovative Learning
learners and identify the competence the learner needs in order to Environments. OECD Report.
understand how various configurations of the tools impacts the http://www.oecd.org/edu/ceri/Technology-
data that is generated. For (4) we would have to examine what Rich%20Innovative%20Learning%20Environments%20by%
competence is need to use the available data / data interpretations 20Jennifer%20Groff.pdf
for learning. In addition, we need to study students using OLMs, [2] Reimann, P., Bull, S., Kickmeir-Rust, M., Vatrapu, R. &
LA, and visualisations in their learning and identify the Wasson, B. (Eds.) (2016). Measuring and visualising
competence needed. Building on the existing framework already competence development in the information‐rich classroom.
raises questions such as: London: Routledge.
[3] Redecker, Christine, Leis, Miriam , Leendertse, Matthijs ,
What do students need to know about data as evidence? Punie, Yves, Gijsbers, Govert, Kirschner, Paul , Stoyanov,
How do the tools the students use for learning generate data Slavi and Hoogveld, Bert (2011). The Future of Learning:
that can be used as evidence of their learning? Preparing for Change. JRC IPTS. Seville, Spain.
http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=
What do students need to know about how learning web&cd=1&ved=0CB8QFjAA&url=http%3A%2F%2Fftp.jrc.es
analytics uses their student data to understand and visualise %2FEURdoc%2FJRC66836.pdf&ei=Rm0hVMKQFYeAywPMgI
their learning?
GgCA&usg=AFQjCNFMqUolTcs_vPPVEbvSFj3mVIcAGA&sig2
What competence is required by learners in order to use the =MkAWEbOrh9fVhG3niklN7A&bvm=bv.75775273,d.bGQ
visualisation data for self-reflection, planning, self-directed
learning, etc.? [4] Sharples, M., McAndrew, P., Weller, M., Ferguson, R.,
FitzGerald, E., Hirst, T. & Gaved, M. (2013). Innovating
What competence is required in order for students to carry Pedagogy 2013. Open University Innovation Report 2.
out their own learning analytics on their own student data? Milton Keynes: The Open University. ISBN 978-1-78007-
What visualisations are best understood by students? 937-0
Are students concerned about ethical and privacy issues [5] Wasson, B., & Hansen, C. (2016). Data literacy and use for
related to the use of their learning data? Should they be? teaching. In P. Reimann, S. Bull, M. Kickmeir-Rust, R.
Vatrapu & B. Wasson (Eds.). Measuring and visualising
competence development in the information‐rich classroom,
We imagine that the LAL workshop will raise many more such 56-73. London: Routledge.
questions that are important to investigate in order that we can
[6] Luckin, R., Hansen, C., Wasson, B., Avramides, K., Hunter,
develop a framework for data literacy and use for learning. For
J. & Oliver, M. (2016). Teacher inquiry into students’
example, issues related to privacy, ethics, and data protection are
learning: Research pedagogical innovations. In P. Reimann, [12] Kickmeier-Rust, M. and Albert, D. (2016). Competence-
S. Bull, M. Kickmeir-Rust, R. Vatrapu & B. Wasson (Eds.). Based Knowledge Space Theory: Options for the 21st
Measuring and visualising competence development in the Century ClassroomIn P. Reimann, S. Bull, M. Kickmeir-
information‐rich classroom, 74-91. London: Routledge. Rust, R. Vatrapu & B. Wasson (Eds.). Measuring and
[7] Bull, S. & Kay, J. (2007). Student Models that Invite the visualising competence development in the information‐rich
Learner In: The SMILI Open Learner Modelling Framework, classroom, 74-91. London: Routledge.
International Journal of Artificial Intelligence in Education [13] Thomas, J. and Kielman, J. (2009). Challenges for visual
17(2), 89-120. analytics. Information Visualization, 8(4), 309-314.
[8] Bull, S., Wasson, B., Kickmeier-Rust, M., Johnson, M.D., [14] Thomas, J.J. & Cook, K.A. (Eds)(2005). Illuminating the
Moe, E., Hansen, C., Meissl-Egghart, G. & Hammermüller, path: The research and development agenda for visual
K. (2012). Assessing English as a Second Language: From analytics. IEEE Computer Society, 2005.
Classroom Data to a Competence-Based Open Learner [15] Vatrapu, R. Reimann, P., Hussain, A & Pantazos, K. (2016).
Model, ICCE 2012, November, Singapore. Repertory Grids for Formative Assessment: Concept,
[9] Bull, S. and Wasson, B. (2016). Competence Visualisation: Method, Tools, and Findings. In P. Reimann, S. Bull, M.
Making Sense of Data from 21st Century Technologies in Kickmeir-Rust, R. Vatrapu & B. Wasson (Eds.). Measuring
Language Learning. ReCALL, 28(02):147-165. and visualising competence development in the information‐
[10] Morales, R., Van Labeke, N., Brna, P. & Chan, M.E. (2009). rich classroom, 121-136. London: Routledge.
Open Learner Modelling as the Keystone of the Next [16] Johnson, M.D. & Bull, S. (2016). Designing for Visualisation
Generation of Adaptive Learning Environments. In C. of Formative Information on Learning. In P. Reimann, S.
Mourlas & P. Germanakos (eds), Intelligent User Interfaces, Bull, M. Kickmeir-Rust, R. Vatrapu & B. Wasson (Eds.).
288-312, London: ICI. Measuring and visualising competence development in the
[11] Mazzola, L. & Mazza, R. (2010). GVIS: A Facility for information‐rich classroom, 221-234. London: Routledge.
Adaptively Mashing Up and Representing Open Learner
Models, In M. Wolpers et al (eds), EC-TEL 2010, Berlin:
Springer Verlag, 554-559.