=Paper=
{{Paper
|id=Vol-2209/paper2
|storemode=property
|title=Challenges in Developing Automatic Learning Guidance in Relation to an Information Literacy Curriculum
|pdfUrl=https://ceur-ws.org/Vol-2209/paper2.pdf
|volume=Vol-2209
|authors=Angela Fessl,Alfred Wertner,Viktoria Pammer-Schindler
|dblpUrl=https://dblp.org/rec/conf/ectel/FesslWP18a
}}
==Challenges in Developing Automatic Learning Guidance in Relation to an Information Literacy Curriculum==
https://ceur-ws.org/Vol-2209/paper2.pdf
Challenges in Developing Automatic Learning Guidance
in Relation to an Information Literacy Curriculum
Angela Fessl1, Alfred Wertner1 and Viktoria Pammer-Schindler2
1 Know-Center GmbH, Inffeldgasse 13, 8010 Graz, Austria
2 Institute for Interactive Systems and Data Science, Graz University of Technology, Austria
{afessl,awertner,vpammer}@know-center.at
Abstract. Becoming a data-savvy professional requires skills and competences
in information literacy, communication and collaboration, and content creation
in digital environments. In this paper, we present a concept for automatic learning
guidance in relation to an information literacy curriculum. The learning guidance
concept has three components: Firstly, an open learner model in terms of an in-
formation literacy curriculum is created. Based on the data collected in the learner
model, learning analytics is used in combination with a corresponding visualiza-
tion to present the current learning status of the learner. Secondly, reflection
prompts in form of sentence starters or reflective questions adaptive to the learner
model aim to guide learning. Thirdly, learning resources are suggested that are
structured along learning goals to motivate learners to progress. The main con-
tribution of this paper is to discuss what we see as main research challenges with
respect to existing literature on open learner modeling, learning analytics, recom-
mender systems for learning, and learning guidance.
Keywords: Reflection guidance, information literacy, digital competency, open
learner models, learning analytics.
1 Introduction
Information literacy and the access to and use of knowledge are becoming a precondi-
tion for individuals to actively take part in social, economic, cultural and political life
in societies of the 21st century. Information literacy must be considered as a fundamen-
tal competency like the ability to read, write and calculate. The UNESCO considers it
“a basic human right” [4] while the American Library Association (ALA) [1] calls it a
“survival skill in the information age”. Therefore, the education of professionals to be-
come data-savvy becomes more and more important, especially during work, where the
time and place for learning is often neglected due to a high workload and time pressure.
To be able to educate data-savvy professionals in informal learning settings like the
workplace, we developed a concept for an automatic learning guidance. The goal of
this learning guidance is to support users to become information-savvy professionals
with regard to a curriculum developed for information literacy and digital competency
[5]. To do so, we use open learner models as underlying approach. We relate this learner
2
model strongly to the information literacy curriculum in order to store the learning sta-
tus and progress of each learner. Learning analytics is used to analyze the data in the
learner model and to visualize the learning status and progress in a sophisticated way.
Learning guidance [9] is implemented in form of reflective prompts consisting of re-
flective questions or sentence starters to motivate people to reflect about their compe-
tence status and learning progress. In addition, learning resources are recommended to
the learner in relation to the learning status aiming at motivating the learner to contin-
uously pursue their learning goals with regard to the curriculum.
The main contribution of this paper is to discuss what we see as main research chal-
lenges for the development of a concept for an automatic learning guidance with respect
to existing literature on open learner modeling, learning analytics, learning guidance in
form of reflective interventions and recommender systems for learning.
2 Background and Related Work
Our present work draws on background and related work from information literacy and
digital competences, as that is the domain of learning (that which is learned) which we
investigate. Our present work also draws on research on open learner modelling and
learning analytics, as in these fields, log data created by learners are analyzed and pre-
sented to users as basis for learning. Reflection guidance based on such data goes one
step further and provides explicit guidance for using such data to reflect on it with the
purpose of learning. Finally, we briefly relate to work on recommender systems in
learning, as content recommendation will be one functionality that complements the
widget under development.
Information Literacy and Digital Competence: Since the emergence of Web 2.0,
information literacy needs to be reconsidered in the context of participatory environ-
ments as “students have grown up in a digital age, wherein social media platforms are
playing a central role in defining the ways they interact with information” [18]. As a
consequence, there needs to be a paradigm shift from formal to informal learning within
such participatory environments to educate students’ as well as European citizens’ sur-
vival skills in the information age to become lifelong, autonomous learners. Moreover,
the European Commission sees information literacy and digital competence as funda-
mental competences in the 21st century: “Digital competence – or the confident and
critical use of ICT tools in these areas – is vital for participation in today's society and
economy” [17]. Even though information literacy and digital competence are not the
same, but they necessarily complement each other particularly in web-based infor-
mation systems. Thus, the European Commission developed “The European Digital
Competence Framework for Citizens” (DigComp2.1) [5] that offers a tool to improve
citizens’ digital competence.
In our present work, we focus on three major modules of the DigComp 2.1 curricu-
lum: “Information and Data Literacy”, “Communication & Collaboration” and “Con-
tent Creation”. Each of these modules consists of several sub-competences, e.g. one of
the sub-competences for the Information and Data Literacy module is “Browsing,
3
searching, filtering information, data and digital content”. Each sub-competence con-
sists of different competency levels such as beginner, intermediate and expert.
Open Learner Modeling and Learning Analytics: In open learner modeling, user
models (that what a computer knows about the user) are made available to users as basis
for learning. User profiles are models that computer systems have about their users [10].
The data stored in such user models are often automatically captured by the system (e.g.
activity tracking tools) and are often used in learning environments (see e.g., [7, 10]).
User models established in learning environment systems for modelling the learner and
the corresponding learning activities are called learner models. Such models typically
contain information such as “knowledge, interest, goals, background and individual
traits” [3]. These models are not only used by computers to adapt their behavior or
information representation to the user but also to track and store the learning activities
of the user. If these models are made accessible and manageable to the learners, they
have been termed open learner models. Such learner models can serve as basis for re-
flection on one's own learning activities, or the progress towards the individual learning
goals which was explicitly suggested by several works [11, 12].
Similarly, learning analytics researches methods and usage of data analysis and pat-
tern mining on data collected from educational settings or learning environments about
the learner. Explicit traces (e.g. the learner’s entries in a chat or a discussion forum)
and implicit traces (e.g. the learner entering a course or clicking on a document or but-
ton) stored in the corresponding open learner model serve as basis for the aggregation
and visualization of the gathered data. These explicit and implicit traces can be used to
provide personalized access to learning material [6], which can be specifically prepared
for such learning needs [15]. Thus, the focus of learning analytics is on providing sup-
port for the learners in formal as well as informal learning settings. Approaches like
learning dashboards for example described in [7, 14] present an overview of the
learner’s own learning activities and learning progress often in relation to colleagues at
one glance. Such visualizations support self-monitoring of learners and awareness for
teachers and empower the learners to reflect on their own (learning) activity and that of
their peers. In the present work, we draw on literature from open learner modeling and
learning analytics in that we aim to infer activities and learning progress towards learn-
ing goals defined in the information literacy curriculum by analyzing log data created
while using a search platform.
Reflection Guidance: During a learning process, independent whether formal or
informal, reflective learning can play a significant role. Reflective learning is a viable
mean to re-evaluate past experiences in order to learn from them to guide future behav-
ior [2]. In literature, there exists different types of technologies like, diaries, journals,
e-portfolios, as well as prompts or visuals [8] that aim to actively foster and guide re-
flective learning. Diaries, journals and e-portfolios are very time consuming to be kept
and maintained, thus, they are mostly used in formal learning settings [9]. Therefore,
we will mainly focus on visuals as reported in the learning analytics paragraph above
and reflective prompts. Reflective prompts, which we understand as interventions (or
triggers) that consist of small text messages or questions trying to motivate a user to
reflect. In learning environments prompts are well investigated, because the learning
activities and tasks are well known beforehand, thus prompts can be well designed and
4
tailored to the learning tasks. In contrast, at the workplace, learning activities are not
always known beforehand or only vaguely known therefore it is not so easy to design
prompts according to the learners’ activities [9]. Additionally, it is still a challenge to
decide the right timing for presenting prompts in order to not disrupt the current work-
flow of a user. However, reflective prompts are still seen as very promising approach
to stimulate reflection when presented at the right time and with the right content [9].
In the present work, reflection guidance on the curriculum modules constitutes the core
learning guidance functionality, complemented by recommendation of suitable learning
materials.
By recommender systems in this context of learning, we understand functionality
that suggest items related to learning goals to users [13]. Therefore, recommender sys-
tems are widely common in the area of technology enhanced learning, as for some
learners “it is difficult to express specific learning requirements through keywords” as
stated by [15], thus meaning that for some learners it is difficult to express or formulate
their exact learning needs. Especially, learning environments often provide access to
learning resources without ensuring if a learner or teacher really used the suggested
resources [16]. In contrast, adaptive learning environments track the learner’s activities
on a learning environment to provide personalized access to learning material [6]. In
the present work, we use such a personalized recommendation of learning resources to
facilitate the achievement of the information literacy curriculum.
Below we will first develop the concept of a widget for automatically guiding learn-
ing with respect to information literacy on a search platform; and then close the paper
by discussing what are research questions that are not answered by existing literature,
and challenging in development.
3 Bringing it all Together: Widget for Automatic
Learning Guidance
Based on the above literature, we have designed a concept for the following widget to
provide automatic learning guidance with respect to the information literacy curricu-
lum. The goal of the automatic learning guidance is to raise the learner’s competence
level for each competence to the expert level. Therefore, the widget is designed to be
placed next to a search interface on a newly developed search platform, such that search
activities can be used to feed the open learner model. As learning activities performed
on the platform, we see all activities that are related to the curriculum, like for example
reading a document recommended for pursuing the curriculum or answering a reflective
question.
Open Learner Model & Learning Analytics: To compute the current status of the
learning progress, we follow a two-step approach: First, when a user registers herself
on the search platform the user has to self-assess her competence status with respect to
the information literacy curriculum. This self-assessment initializes the learner model.
Second, all search activities on the platform (e.g. entering search terms, open learning
resources) are tracked and stored, and are used to update the stored user competence
5
w.r.t. curriculum learning goals. Following the results from learning analytics research,
the current learner profile is visualized in the widget representing the current learning
status and progress of the learner w.r.t. the curriculum. For example, Fig. 1, left, shows
that the current user has already completed 45% of the module “Information and Data
Literacy”. When clicking on one of the modules, the detailed status per sub-competence
is presented, as shown in Fig. 1, middle.
Reflection guidance: For designing learning guidance according to the user’s needs
and with regard to the curriculum, we developed a pool of reflective questions and sen-
tence starters on different levels (beginner, intermediate and expert). These prompts
will be presented adapted to the user’s learning status and competence level. The
prompts contain placeholders with regard to the competences of the curriculum per
module in order to be adapted to the content and competence the user is currently learn-
ing e.g. “How did the document ‘Filtering of data on a search platform.pdf’ help you
to improve your search behavior?” (Beginner’s level) or “How could the topic ‘filtering
of data’ impact your search behavior?” (expert’s level). In addition, the prompts will
consist of different difficulty levels depending on the user’s competence status and pro-
gress. Furthermore, which prompts were presented to the learner, which of them were
answered by the user will be stored in the corresponding learner model.
Learning resource recommendation: We will manually define learning resources
for each curriculum module and sub-competences. These learning resources have been
explicitly produced for the curriculum modules and sub-competences. They will be
tagged with corresponding keywords, in order to know to which module and sub-com-
petence the resource is belonging to and which competence level it addresses. These
learning resources will be used as input to a recommender system that recommends
additional, related resources. Learning resources are then recommended in adaptation
to the user’s competence status stored in the learner model (see Fig. 1, right).
Fig. 1. Learning status w.r.t. the curriculum modules (left) and sub-competences (middle), and
learning resource recommendation and reflection guidance (right).
6
4 Research Questions and Outlook
We perceive the main challenges as lying in automatic creation of the open learner
model in terms of the curriculum; and the design of learning guidance. We therefore
formulate the following research questions as guide for our future work, and also as
open challenges for other researchers with shared interests:
RQ1: For every learning goal in the EU information literacy curriculum: How
accurately can users’ competence w.r.t. the learning goal be assessed automati-
cally? In our future work, we aim to base the automatic assessment on user
search behavior.
RQ2: How can reflection prompts be phrased for different learner competence
status such that they can be understood, are perceived as appropriate w.r.t. the
users’ expertise, and lead to reflection; and how does the appearance of prompts
need to be such that they are not perceived as interruptive.
Our next steps are twofold: on the one hand, the widget is currently implemented,
on the other hand we aim to set up experimental field studies with social sciences uni-
versity students in order to answer the above research questions.
Acknowledgement
The project “MOVING - TraininG towards a society of data-saVvy inforMation prO-
fessionals to enable open leadership iNnovation” is funded under the Horizon 2020 of
the European Commission (project number 693092). The Know-Center is funded
within the Austrian COMET Program - Competence Centers for Excellent Technolo-
gies - under the auspices of the Austrian Federal Ministry of Transport, Innovation and
Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the
State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.
References
1. American Library Association (ALA): Presidential Committee on Information Literacy: Fi-
nal Report. http://www.ala.org/acrl/publications/whitepapers/presidential last accessed
2018/3/23.
2. Boud, D. Keogh, R., Walker, D. Promoting Reflection in Learning: a Model. In Reflection:
Turning Experience into Learning, 18-40. New York: Routledge Farmer (1985).
3. Brusilovsky, P., Millan, E. User Models for Adaptive Hypermedia and Adaptive Educational
Systems. Berlin, Germany: Springer, pp. 3–53, (2007)
4. Catts, R., Lau, J.: Towards Information Literacy Indicators. Paris: UNESCO,
https://www.ifla.org/files/assets/information-literacy/publications/towards-information-lit-
eracy_2008-en.pdf, last accessed 2018/3/23.
5. Carretero, S., Vuorikari, R., Punie, Y.. DigComp 2.1: The Digital Competence Framework
for Citizens with eight proficiency levels and examples of use (2017).
7
6. De Bra, P., Pechenizkiy, M., van der Sluijs, K., Smits, D.. GRAPPLE: Integrating Adaptive
Learning into Learning Management Systems. In J. Luca and E.R. Weippl, editors, Proceed-
ings of World Conference on Educational Multimedia, Hypermedia and Telecommunica-
tions, 5183–5188, Vienna, Austria, June 2008. AACE (2008).
7. Duval, E.: Attention please!: Learning analytics for visualization and recommendation. In
Proceedings of the 1st International Conference on Learning Analytics and Knowledge,
LAK ’11, ACM, 9-17. New York, NY, USA (2011).
8. Fessl, A., Blunk, O., Prilla, M., Pammer, V.: The known universe of reflection guidance: a
literature review. Int. J. of Technology Enhanced Learning, 9(2-3), 103-125 (2016).
9. Fessl A., Wesiak, G., Rivera-Pelayo, V., Feyertag, S., Pammer V.: In-app Reflection Guid-
ance: Lessons Learned across Four Field Trials at the Workplace. In IEEE Transactions on
Learning Technologies, 10(4), 488-501 (2017).
10. Fischer, G.: User modeling in human-computer interaction. In User Modeling and
UserAdapted Interaction (UMUAI), 11, 65-86 (2001).
11. Kay, J.: Learner Control. In: User Modeling and User-Adapted Interaction, Kluwer Aca-
demic Publishers, 11, 111-127 (2001)
12. Kay, J., Kummerfield, B. Lifelong learner modelling. In: Adaptive Technologies for Train-
ing and Education. Cambridge University Press (2011).
13. Nabizadeh, A. H., Jorge, A. M., Leal, J. P. . Long Term Goal Oriented Recommender Sys-
tems. In WEBIST, pp. 552-557. (2015).
14. Santos, J. L., Verbert, K., Govaerts, S., & Duval, E.: Addressing learner issues with stepup!:
An evaluation. In Proceedings of the Third International Conference on Learning Analytics
and Knowledge, LAK ’13, ACM, 14-22. New York, NY, USA, (2013).
15. Thalmann, S. Adaptation criteria for the personalized delivery of learning materials: A
multi-stage empirical investigation. Australasian Journal of Educational Technology, 30(1).
(2014).
16. Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., Duval, E.:
Context-aware recommender systems for learning: a survey and future challenges. IEEE
Transactions on Learning Technologies, 5(4), 318-335 (2012).
17. Vuorikari, R., Punie, Y., Carretero Gomez, S., Van Den Brande, L.. DigComp 2.0: The Dig-
ital Competence Framework for Citizens: Update Phase 1: The Conceptual Reference
Model. Luxembourg (2016).
18. Witek, D. & Grettano, T.: Teaching metaliteracy: a new paradigm in action. Reference Ser-
vices Review, 42(2), 188-208, (2014).