Challenges in Developing Automatic Learning Guidance in Relation to an Information Literacy Curriculum

Abstract

Becoming a data-savvy professional requires skills and competences in information literacy, communication and collaboration, and content creation in digital environments. In this paper, we present a concept for automatic learning guidance in relation to an information literacy curriculum. The learning guidance concept has three components: Firstly, an open learner model in terms of an information literacy curriculum is created. Based on the data collected in the learner model, learning analytics is used in combination with a corresponding visualization to present the current learning status of the learner. Secondly, reflection prompts in form of sentence starters or reflective questions adaptive to the learner model aim to guide learning. Thirdly, learning resources are suggested that are structured along learning goals to motivate learners to progress. The main contribution of this paper is to discuss what we see as main research challenges with respect to existing literature on open learner modeling, learning analytics, recommender systems for learning, and learning guidance.

Keywords

• Reflection guidance
• information literacy
• digital competency
• open learner models
• learning analytics

-

Information literacy and the access to and use of knowledge are becoming a precondition for individuals to actively take part in social, economic, cultural and political life in societies of the 21st century. Information literacy must be considered as a fundamental competency like the ability to read, write and calculate. The UNESCO considers it “a basic human right” [ 4 ] while the American Library Association (ALA) [ 1 ] calls it a “survival skill in the information age”. Therefore, the education of professionals to become data-savvy becomes more and more important, especially during work, where the time and place for learning is often neglected due to a high workload and time pressure.

To be able to educate data-savvy professionals in informal learning settings like the workplace, we developed a concept for an automatic learning guidance. The goal of this learning guidance is to support users to become information-savvy professionals with regard to a curriculum developed for information literacy and digital competency [ 5 ]. To do so, we use open learner models as underlying approach. We relate this learner model strongly to the information literacy curriculum in order to store the learning status and progress of each learner. Learning analytics is used to analyze the data in the learner model and to visualize the learning status and progress in a sophisticated way. Learning guidance [ 9 ] is implemented in form of reflective prompts consisting of reflective questions or sentence starters to motivate people to reflect about their competence status and learning progress. In addition, learning resources are recommended to the learner in relation to the learning status aiming at motivating the learner to continuously pursue their learning goals with regard to the curriculum.

The main contribution of this paper is to discuss what we see as main research challenges for the development of a concept for an automatic learning guidance with respect to existing literature on open learner modeling, learning analytics, learning guidance in form of reflective interventions and recommender systems for learning. 2

Background and Related Work

Our present work draws on background and related work from information literacy and digital competences, as that is the domain of learning (that which is learned) which we investigate. Our present work also draws on research on open learner modelling and learning analytics, as in these fields, log data created by learners are analyzed and presented to users as basis for learning. Reflection guidance based on such data goes one step further and provides explicit guidance for using such data to reflect on it with the purpose of learning. Finally, we briefly relate to work on recommender systems in learning, as content recommendation will be one functionality that complements the widget under development.

Information Literacy and Digital Competence: Since the emergence of Web 2.0, information literacy needs to be reconsidered in the context of participatory environments as “students have grown up in a digital age, wherein social media platforms are playing a central role in defining the ways they interact with information” [ 18 ]. As a consequence, there needs to be a paradigm shift from formal to informal learning within such participatory environments to educate students’ as well as European citizens’ survival skills in the information age to become lifelong, autonomous learners. Moreover, the European Commission sees information literacy and digital competence as fundamental competences in the 21st century: “Digital competence – or the confident and critical use of ICT tools in these areas – is vital for participation in today's society and economy” [ 17 ]. Even though information literacy and digital competence are not the same, but they necessarily complement each other particularly in web-based information systems. Thus, the European Commission developed “The European Digital Competence Framework for Citizens” (DigComp2.1) [ 5 ] that offers a tool to improve citizens’ digital competence.

In our present work, we focus on three major modules of the DigComp 2.1 curriculum: “Information and Data Literacy”, “Communication & Collaboration” and “Content Creation”. Each of these modules consists of several sub-competences, e.g. one of the sub-competences for the Information and Data Literacy module is “Browsing, searching, filtering information, data and digital content”. Each sub-competence consists of different competency levels such as beginner, intermediate and expert.

Open Learner Modeling and Learning Analytics: In open learner modeling, user

models (that what a computer knows about the user) are made available to users as basis for learning. User profiles are models that computer systems have about their users [ 10 ]. The data stored in such user models are often automatically captured by the system (e.g. activity tracking tools) and are often used in learning environments (see e.g., [ 7, 10 ]). User models established in learning environment systems for modelling the learner and the corresponding learning activities are called learner models. Such models typically contain information such as “knowledge, interest, goals, background and individual traits” [ 3 ]. These models are not only used by computers to adapt their behavior or information representation to the user but also to track and store the learning activities of the user. If these models are made accessible and manageable to the learners, they have been termed open learner models. Such learner models can serve as basis for reflection on one's own learning activities, or the progress towards the individual learning goals which was explicitly suggested by several works [ 11, 12 ].

Similarly, learning analytics researches methods and usage of data analysis and pattern mining on data collected from educational settings or learning environments about the learner. Explicit traces (e.g. the learner’s entries in a chat or a discussion forum) and implicit traces (e.g. the learner entering a course or clicking on a document or button) stored in the corresponding open learner model serve as basis for the aggregation and visualization of the gathered data. These explicit and implicit traces can be used to provide personalized access to learning material [ 6 ], which can be specifically prepared for such learning needs [ 15 ]. Thus, the focus of learning analytics is on providing support for the learners in formal as well as informal learning settings. Approaches like learning dashboards for example described in [ 7, 14 ] present an overview of the learner’s own learning activities and learning progress often in relation to colleagues at one glance. Such visualizations support self-monitoring of learners and awareness for teachers and empower the learners to reflect on their own (learning) activity and that of their peers. In the present work, we draw on literature from open learner modeling and learning analytics in that we aim to infer activities and learning progress towards learning goals defined in the information literacy curriculum by analyzing log data created while using a search platform.

Reflection Guidance: During a learning process, independent whether formal or informal, reflective learning can play a significant role. Reflective learning is a viable mean to re-evaluate past experiences in order to learn from them to guide future behavior [ 2 ]. In literature, there exists different types of technologies like, diaries, journals, e-portfolios, as well as prompts or visuals [ 8 ] that aim to actively foster and guide reflective learning. Diaries, journals and e-portfolios are very time consuming to be kept and maintained, thus, they are mostly used in formal learning settings [ 9 ]. Therefore, we will mainly focus on visuals as reported in the learning analytics paragraph above and reflective prompts. Reflective prompts, which we understand as interventions (or triggers) that consist of small text messages or questions trying to motivate a user to reflect. In learning environments prompts are well investigated, because the learning activities and tasks are well known beforehand, thus prompts can be well designed and tailored to the learning tasks. In contrast, at the workplace, learning activities are not always known beforehand or only vaguely known therefore it is not so easy to design prompts according to the learners’ activities [ 9 ]. Additionally, it is still a challenge to decide the right timing for presenting prompts in order to not disrupt the current workflow of a user. However, reflective prompts are still seen as very promising approach to stimulate reflection when presented at the right time and with the right content [ 9 ]. In the present work, reflection guidance on the curriculum modules constitutes the core learning guidance functionality, complemented by recommendation of suitable learning materials.

By recommender systems in this context of learning, we understand functionality that suggest items related to learning goals to users [ 13 ]. Therefore, recommender systems are widely common in the area of technology enhanced learning, as for some learners “it is difficult to express specific learning requirements through keywords” as stated by [ 15 ], thus meaning that for some learners it is difficult to express or formulate their exact learning needs. Especially, learning environments often provide access to learning resources without ensuring if a learner or teacher really used the suggested resources [ 16 ]. In contrast, adaptive learning environments track the learner’s activities on a learning environment to provide personalized access to learning material [ 6 ]. In the present work, we use such a personalized recommendation of learning resources to facilitate the achievement of the information literacy curriculum.

Below we will first develop the concept of a widget for automatically guiding learning with respect to information literacy on a search platform; and then close the paper by discussing what are research questions that are not answered by existing literature, and challenging in development. 3

Bringing it all Together: Widget for Automatic

Learning Guidance

Based on the above literature, we have designed a concept for the following widget to provide automatic learning guidance with respect to the information literacy curriculum. The goal of the automatic learning guidance is to raise the learner’s competence level for each competence to the expert level. Therefore, the widget is designed to be placed next to a search interface on a newly developed search platform, such that search activities can be used to feed the open learner model. As learning activities performed on the platform, we see all activities that are related to the curriculum, like for example reading a document recommended for pursuing the curriculum or answering a reflective question.

Open Learner Model & Learning Analytics: To compute the current status of the

learning progress, we follow a two-step approach: First, when a user registers herself on the search platform the user has to self-assess her competence status with respect to the information literacy curriculum. This self-assessment initializes the learner model. Second, all search activities on the platform (e.g. entering search terms, open learning resources) are tracked and stored, and are used to update the stored user competence w.r.t. curriculum learning goals. Following the results from learning analytics research, the current learner profile is visualized in the widget representing the current learning status and progress of the learner w.r.t. the curriculum. For example, Fig. 1, left, shows that the current user has already completed 45% of the module “Information and Data Literacy”. When clicking on one of the modules, the detailed status per sub-competence is presented, as shown in Fig. 1, middle.

Reflection guidance: For designing learning guidance according to the user’s needs and with regard to the curriculum, we developed a pool of reflective questions and sentence starters on different levels (beginner, intermediate and expert). These prompts will be presented adapted to the user’s learning status and competence level. The prompts contain placeholders with regard to the competences of the curriculum per module in order to be adapted to the content and competence the user is currently learning e.g. “How did the document ‘Filtering of data on a search platform.pdf’ help you to improve your search behavior?” (Beginner’s level) or “How could the topic ‘filtering of data’ impact your search behavior?” (expert’s level). In addition, the prompts will consist of different difficulty levels depending on the user’s competence status and progress. Furthermore, which prompts were presented to the learner, which of them were answered by the user will be stored in the corresponding learner model.

Learning resource recommendation: We will manually define learning resources for each curriculum module and sub-competences. These learning resources have been explicitly produced for the curriculum modules and sub-competences. They will be tagged with corresponding keywords, in order to know to which module and sub-competence the resource is belonging to and which competence level it addresses. These learning resources will be used as input to a recommender system that recommends additional, related resources. Learning resources are then recommended in adaptation to the user’s competence status stored in the learner model (see Fig. 1, right).

Research Questions and Outlook

We perceive the main challenges as lying in automatic creation of the open learner model in terms of the curriculum; and the design of learning guidance. We therefore formulate the following research questions as guide for our future work, and also as open challenges for other researchers with shared interests:

RQ1: For every learning goal in the EU information literacy curriculum: How accurately can users’ competence w.r.t. the learning goal be assessed automatically? In our future work, we aim to base the automatic assessment on user search behavior.

RQ2: How can reflection prompts be phrased for different learner competence status such that they can be understood, are perceived as appropriate w.r.t. the users’ expertise, and lead to reflection; and how does the appearance of prompts need to be such that they are not perceived as interruptive.

Our next steps are twofold: on the one hand, the widget is currently implemented, on the other hand we aim to set up experimental field studies with social sciences university students in order to answer the above research questions.

Acknowledgement

The project “MOVING - TraininG towards a society of data-saVvy inforMation prOfessionals to enable open leadership iNnovation” is funded under the Horizon 2020 of the European Commission (project number 693092). The Know-Center is funded within the Austrian COMET Program - Competence Centers for Excellent Technologies - under the auspices of the Austrian Federal Ministry of Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.

References

1. American Library Association (ALA): Presidential Committee on Information Literacy: Final Report . http://www.ala.org/acrl/publications/whitepapers/presidential last accessed 2018 /3/23.

2. Boud , D. Keogh , R. , Walker , D. Promoting Reflection in Learning: a Model . In Reflection: Turning Experience into Learning , 18 - 40 . New York: Routledge Farmer ( 1985 ).

3. Brusilovsky , P. , Millan , E. User Models for Adaptive Hypermedia and Adaptive Educational Systems . Berlin, Germany: Springer, pp. 3 - 53 , ( 2007 )

4. Catts , R. , Lau , J.: Towards Information Literacy Indicators . Paris: UNESCO, https://www.ifla.org/files/assets/information-literacy/publications/towards-information-literacy_ 2008 -en .pdf, last accessed 2018 /3/23.

5. Carretero , S. , Vuorikari , R. , Punie , Y.. DigComp 2 . 1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use ( 2017 ).

6. De Bra , P. , Pechenizkiy , M., van der Sluijs , K. , Smits , D.. GRAPPLE: Integrating Adaptive Learning into Learning Management Systems . In J. Luca and E.R. Weippl , editors, Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications , 5183 - 5188 , Vienna, Austria, June 2008 . AACE ( 2008 ).

7. Duval , E.: Attention please!: Learning analytics for visualization and recommendation . In Proceedings of the 1st International Conference on Learning Analytics and Knowledge , LAK '11, ACM , 9 - 17 . New York, NY, USA ( 2011 ).

8. Fessl , A. , Blunk , O. , Prilla , M. , Pammer , V. : The known universe of reflection guidance: a literature review . Int. J. of Technology Enhanced Learning , 9 ( 2-3 ), 103 - 125 ( 2016 ).

9. Fessl A. , Wesiak , G. , Rivera-Pelayo , V. , Feyertag , S. , Pammer V. : In-app Reflection Guidance: Lessons Learned across Four Field Trials at the Workplace . In IEEE Transactions on Learning Technologies , 10 ( 4 ), 488 - 501 ( 2017 ).

10. Fischer , G. : User modeling in human-computer interaction . In User Modeling and UserAdapted Interaction (UMUAI) , 11 , 65 - 86 ( 2001 ).

11. Kay , J.: Learner Control . In: User Modeling and User-Adapted Interaction , Kluwer Academic Publishers, 11 , 111 - 127 ( 2001 )

12. Kay , J. , Kummerfield , B. Lifelong learner modelling . In: Adaptive Technologies for Training and Education . Cambridge University Press ( 2011 ).

13. Nabizadeh , A. H. , Jorge , A. M. , Leal , J. P. . Long Term Goal Oriented Recommender Systems . In WEBIST, pp. 552 - 557 . ( 2015 ).

14. Santos , J. L. , Verbert , K. , Govaerts , S. , & Duval , E. : Addressing learner issues with stepup!: An evaluation . In Proceedings of the Third International Conference on Learning Analytics and Knowledge , LAK '13, ACM , 14 - 22 . New York, NY, USA, ( 2013 ).

15. Thalmann , S. Adaptation criteria for the personalized delivery of learning materials: A multi-stage empirical investigation . Australasian Journal of Educational Technology , 30 ( 1 ). ( 2014 ).

16. Verbert , K. , Manouselis , N. , Ochoa , X. , Wolpers , M. , Drachsler , H. , Bosnic , I. , Duval , E.: Context-aware recommender systems for learning: a survey and future challenges . IEEE Transactions on Learning Technologies , 5 ( 4 ), 318 - 335 ( 2012 ).

17. Vuorikari , R. , Punie , Y. , Carretero Gomez , S. , Van Den Brande, L.. DigComp 2 . 0: The Digital Competence Framework for Citizens: Update Phase 1: The Conceptual Reference Model . Luxembourg ( 2016 ).

18. Witek , D. & Grettano , T. : Teaching metaliteracy: a new paradigm in action . Reference Services Review , 42 ( 2 ), 188 - 208 , ( 2014 ).