=Paper=
{{Paper
|id=Vol-2209/paper5
|storemode=property
|title=Analytics for Everyday Learning from two Perspectives: Knowledge Workers and Teachers.
|pdfUrl=https://ceur-ws.org/Vol-2209/paper5.pdf
|volume=Vol-2209
|authors=Angela Fessl,Dominik Kowald,Susana López Sola,Ana Moreno,Ricardo Alonso Maturana,Stefan Thalmann
|dblpUrl=https://dblp.org/rec/conf/ectel/FesslKLMMT18
}}
==Analytics for Everyday Learning from two Perspectives: Knowledge Workers and Teachers.==
https://ceur-ws.org/Vol-2209/paper5.pdf
Analytics for Everyday Learning from two Perspectives:
Knowledge Workers and Teachers
Angela Fessl1, Dominik Kowald1, Susana López Sola3, Ana Moreno3, Ricardo Alonso
Maturana3and Stefan Thalmann1,2
1 Know-Center, Inffeldgasse 13, 8010 Graz, Austria
2 Institute of Interactive Systems and Data Science, Graz University of Technology, Austria
3 GNOSS – RIAM INTELEARNING LAB S.L, Piqueras 3-4th floor,
26006 Logroño-La Rioja, Spain
{afessl, dkowald, sthalmann}@know-center.at
{susanalopez, anamoreno, riam}@gnoss.com
Abstract. Learning analytics deals with tools and methods for analyzing and
detecting patterns in order to support learners while learning in formal as well
as informal learning settings. In this work, we present the results of two focus
groups in which the effects of a learning resource recommender system and a
dashboard based on analytics for everyday learning were discussed from two
perspectives: (1) knowledge workers as self-regulated everyday learners (i.e.,
informal learning) and (2) teachers who serve as instructors for learners (i.e.,
formal learning). Our findings show that the advantages of analytics for every-
day learning are three-fold: (1) it can enhance the motivation to learn, (2) it can
make learning easier and broadens the scope of learning, and (3) it helps to or-
ganize and to systematize everyday learning.
Keywords: Learning analytics, recommender systems, focus group, informal
learning, formal learning.
1 Introduction
“Learning analytics is the measurement, collection, analysis and reporting of data
about learners and their contexts, for purposes of understanding and optimizing learn-
ing and the environments in which it occurs.” as stated by [16]. Thus, learning analyt-
ics deals beside others also with methods for analyzing and detecting patterns within
digital data collected within educational settings or learning environments about the
learner in order to support them. For applying learning analytics to everyday learners,
the data collected encompasses implicit and explicit traces a learner has left on the
Web, e.g., on Facebook or Twitter. These traces can also be mined for personalized
recommendations [3], which is very helpful for learners to filter information such as
the number of learning resources, learning activities etc. significantly increases. Such
2
a recommender typically suggests further learning resources out of an overwhelming
variety of choices adapted and personalized to the learner’s interests and learning
needs. While recommenders are well investigated in the area of technology-enhanced
learning especially with respect to formal learning settings [15], less research has
been addressed for recommenders based on analytics for everyday learning (i.e., in-
formal learning).
In this work, we present the results of two focus groups that were conducted in two
different settings focusing on a learning resource recommender based on analytics for
everyday learning. The first focus group was conducted together with knowledge
workers to discuss how a recommender can improve everyday learning while working
and which indicators are relevant for measuring the improvements. The second focus
group was conducted with teachers to discuss. After presenting the results of each
focus group individually, we finally compare the results from both focus groups and
discuss them from the two different perspectives, namely from the perspective of
knowledge workers (i.e., self-regulated everyday learners) and teachers who serve as
instructors for learners. As a result, the following three main claims are derived: ana-
lytics for everyday learning (1) can enhance the motivation to learn, (2) can make
learning easier and broadens the scope of learning, and (3) helps to organize and to
systematize everyday learning.
2 Related Work
Learning analytics deals with methods for analyzing and detecting patterns within
data collected from educational settings or learning environments about the learner,
and leverage those methods to support adaptation, personalization, reflection as well
as recommendation.
Siemens [12] defines learning analytics as “the use of intelligent data, learner-
produced data, and analysis models to discover information and social connections,
and to predict and advise on learning”. The focus of learning analytics is on provid-
ing support for the learner in formal as well as informal learning settings, thus, also
for everyday learning. Approaches like learning dashboards for example described in
[3,9] present an overview of the learner’s own learning activities and learning pro-
gress often in relation to colleagues at one glance. Such combined visualizations sup-
port self-monitoring of learners and awareness for teachers and empower the learners
to reflect on their own activity and that of their peers. Explicit traces (e.g. the learn-
er’s entries in a chat or a discussion forum) and implicit traces (e.g. the learner enter-
ing a course or clicking on a document or button) stored in the corresponding learner
profiles serve as basis for the aggregation and visualization of the gathered data.
These explicit and implicit traces can be used to provide personalized access to learn-
ing material [1], which can be specifically prepared for such learning needs [14].
Learning analytics can provide guidance, especially in informal learning situations.
This is relevant as most of the learning happens informally and as this type of learning
3
becomes more and more important [7]. Informal learning is either intentional or unin-
tentional, non-institutional as well as not pre-structured, experiential and primarily
under the control of the learner, and, furthermore, its outcomes are difficult to predict
a priori [13]. In regard of informal learning, analytics is important to prompt reflec-
tion about the own behavior [4], which is very helpful for learners as the number of
resources, learning activities etc. significantly increased, so that “learners may find it
hard to get an overview of the available learning activities and to identify the most
suitable ones” [2].
Additionally, analyzing the data streams can be used to provide suitable recommenda-
tions to the learners in informal learning situations [10]. The major goal of recom-
menders is to assist users to find items of their interest or related to their learning goal
[8] by suggesting potential useful items to a learner. As a result, recommender sys-
tems are widely common in the area of technology-enhanced learning, as “it is diffi-
cult to express specific learning requirements through keywords” [15]. Therefore,
learning environments often provide recommendations to learning resources but do
not ensure if a learner or teacher really use the suggested resources [15]. However,
from the content perspective, tools for selecting and preparing learning material are
needed [11]
The major goal of all these approaches is to use learning analytics as underlying ap-
proach to support learners while learning, irrespective of whether it is formal or in-
formal learning. Our focus thereby is to provide support by reducing the abundance of
information and by providing learning recommendations tailored to the learner’s
needs and preferences. Therefore, in this work, we want to address, which improve-
ments for learning can be achieved with recommendations and which indicators might
be useful to measure the usefulness of the recommendation. We will discuss this in
the remainder of this paper.
3 Procedure
We conducted two different focus groups following nearly the same procedure to
answer more or less the same research questions but from two different perspectives.
Additionally, the first focus group the focus was put solely on the resource recom-
mender system, while in the second focus group also a dashboard for teachers was
discussed. Therefore, two separately conducted focus groups were necessary because
of two reasons. First, parts of the results gained from the focus group conducted first
were taken as initial starting point for the second focus group. Second, we want to
shed light on our research questions from two different perspectives: from the per-
spective of knowledge workers as everyday learners and from the perspective of ac-
tive teachers who serve as instructors for learners.
3.1 Research Context
Our research took place in a project focusing on analytics for everyday learners. One
of the tools developed in this project is a mobile application that includes a learning
4
resource recommender for everyday learning. In general, the app allows learners to
monitor their own learning progress in the range of topics they are learning.
The resource recommender aims to provide suggestions of learning resources to users
using the app. Therefore, the overall goal of the recommender is to support the users
in discovering information, which, in some cases, the user is not even aware of s/he is
looking for. These recommendations are calculated based on the identified and ex-
tracted features indicative of learning activities. This includes user-based features
such as user-resource relations (e.g., clicks on resources) as well as resource-based
features such as resource popularity or resource topics. Specifically, the task of our
recommender system is to provide a personalized list of 10 learning resources for a
specific user.
The other tool within the focus of our interest was an analytics tool presenting a dash-
board for teachers. This dashboard visualizes the students’ activities (if students have
accepted sharing their information with the teacher). Additionally, the teacher would
see the aggregated data of the group (average), and the information of a specific stu-
dent to be able to compare him/her with the group. The dashboard for teachers, which
is based on the learner dashboard, was not developed by the time of the focus group
II. Therefore the teachers saw the tools developed for learners and were told about the
plans to release a further version for teachers with the previous additional features. In
the session, a demo of both tools learner dashboard and recommender was presented,
together with some functions developed in the mobile application. The objective of
the application is to provide feedback to the learners on their learning activities in the
various topics they are addressing, and to enable them to set learning goals and track
their own learning activities. The main features of the mobile application are:
Identification of user’s learning scopes based on the visited topics.
Learning dashboard that includes graphic visualizations:
o Front-page with two sections: 1) learning scopes, with possibility of
selecting one of them to see the activity in this topic; 2) general
view of activity in the Didactalia games that showed games played
by time spent, number of tries or score.
o Visualization for a given scope, which presents: 1) the state of
learning indicators for the scope (the defined indicators are: intensi-
ty, complexity, diversity, coverage); 2) identification of best time in
the week in which the user has improved in every indicator; 3) rec-
ommended resources.
o Visualization for a given game.
Function to ‘set goals’ and track their evolution based on indicators, for ex-
ample: increase work daily or increase coverage monthly.
Integration of the recommender tool.
The mobile application has been integrated with the plugin ‘browsing history’, which
registers the user activity on the internet, and into the Didactalia educational platform.
Didactalia.net (the collection and core of Didactalia platform) is a large educational
community based on semantics for students, teachers and parents, powered by
GNOSS. At present, Didactalia's content collection has more than 100,000 education-
5
al resources from various sources and authors, as well as self-created content such as
games (550, now in Geography and Anatomy) and lessons (400). Its registered users
can create, share and discover educational content, and also promote learning com-
munities and classes. In 2017 Didactalia had 12M sessions, 32M pages served,
350,000 members and more than 12M of educational plays (up to 150.000 plays/day).
In Focus group II, we presented the app version in Didactalia. The next figure pre-
sents some screenshots of the app
Figure 1: Screenshots of the AFEL mobile application, including front-page (a), part of the
front-page showing games (b), visualization for a given game (c), visualization for a given
scope (d), recommendations for a given scope (e), existing goals for a given scope (f) and set-
ting of new goal for a given scope (g).
3.2 Research Goals
Altogether we conducted two focus groups, involving 15 participants. The first focus
group (Focus group I) was conducted to get initial feedback about the resource rec-
ommender integrated in the mobile learning app. The goal was to answer the follow-
ing research questions from the perspective of knowledge workers.
RQ1: How does the learning resource recommender help you to improve
your everyday learning?
6
RQ2: Which indicators, measures or dimensions would be useful with regard
to the defined improvements?
The goal of the second focus group (Focus group II) was to get the perspective of
teachers for both tools, the dashboard and learning resource recommender and to an-
swer the following research questions:
RQ3: How/in which aspect do you think that the two tools would help you,
as a teacher, to improve the teaching/learning process with your students?
RQ4: Which indicators, measures or dimensions would be useful with regard
to predefined clusters?
3.3 Focus Group I: Knowledge Workers
Participants: We recruited 9 knowledge workers (2 females, 7 males) using a volun-
tary snowball sampling approach for the focus group. The participants are active pro-
fessionals and have backgrounds in computer science, engineering, controlling and
human resources.
Focus Group: With the focus group, we addressed RQ1 and RQ2 stated above.
Therefore, we divided the two hours into three parts: (i) a general introductory part to
ensure that all participants are equally informed about the app and especially the
learning resource recommender, (ii) Round I to infer improvements of learning with
regard to the recommender (= RQ1) and (iii) Round II to derive indicators to measure
the improvements (=RQ2).
At the beginning of the focus group, the participants were given an introduction to the
related project in general and a more detailed introduction about the mobile app and
its recommender, including a live demo. Then, the participants were asked to pose
questions to eliminate uncertainties about the project and the app. Afterwards input
from the participants was collected in two discussion rounds:
Round I: The goal of the first round was to identify potential application areas for
analytics for everyday learning and specifically for the app and the recommender. To
guide this exploratory brainstorming phase, the following four dimensions were writ-
ten on a blackboard to serve as starting point for the discussion: Professional learning,
competence development, private learning and learning management system. All
participants were asked to write at least one application scenario and one expected
improvement when using the app per topic on a post-it and put it on the blackboard
giving a short explanation. The idea behind was to give every participant a voice and
to reduce the influence of key actors in the group. Then the collected thoughts were
thematically clustered in a moderated group discussion. Finally, the participants were
asked to vote for their favorite clusters for the “gold” (1st place), “silver” (2nd place),
and “bronze” (3rd place). This was done, to achieve convergence by collecting the
final feedback and to get feedback on the importance.
Round II: The goal of the second round was to infer useful indicators for measuring
the success of the app for the identified application scenarios and expected benefits.
Thus, the participants were asked which indicators, measures or dimensions could be
7
used to assess the impact on the defined clusters. This time the participants were
asked to write the indicators directly on the blackboard and to justify their proposal.
3.4 Focus Group II: Teachers
Participants: We recruited 6 teachers (3 females, 3 males) for the focus group. The
participants are active teachers in different subjects: Spanish and Classical Languages,
English, Mathematics and Physical Education.
Focus Group: With the focus group, we addressed RQ3 and RQ4 stated above. The
focus group was carried out after focus group I and followed more or less the same
procedure but with some deviations that will be described below. In the introductory
part not only the project and the resource recommender were presented but also the
dashboard for teachers / administrators were introduced.
Round I: This discussion round started with the clusters that emerged from the first
discussion round of focus group I, with the following differences:
The Cluster ‘Personalization’ was added, as we wanted to find out how
teachers can personalize the activities with their students and to help them to
learn better.
We removed a cluster called “Social Learning” and we merged the “Evalua-
tion/Certificate” cluster with the “Tracking” cluster.
Summing up, the initial clusters were: motivation, discovery, tracking, system-
ic/organization, learning goal and personalization. In this case all participants were
asked to add at least one achievement per cluster. After the collection of thoughts, it
was identified that ‘evaluation’ had an important role according to participant’s opin-
ion, thus it was added as a cluster and participants were asked to reassign their contri-
butions considering this new cluster, if necessary. Afterwards again the same voting
as in focus group I was conducted.
Round II: This round followed the same rules as the second round in focus group I.
4 Results
4.1 Focus Group I
RQ 1: Improvement of learning with the learning resource recommender
Round I: Altogether 37 expected achievements were put on the blackboard by the
participants for the four initial topics. After having them sorted out thematically, the
following clusters emerged with 3 to 9 improvements each: Motivation (5), Discovery
(9), Tracking (4), Systemic (6), Feedback & Certificate (4), Social learning (6), Goal
(3).
Motivation cluster: Participants assumed that the analytics for everyday learning and
the app in particular will increase the motivation to learn. The cluster contains
achievements like increasing and maintaining the motivation to learn as well as to
have a tool that supports a learner in conducting regular learning activities, including
the presentation to automatically presented reminders.
8
Discovery cluster: Participants stated that analytics for everyday learning and the app
will support the discovery of suitable learning resources. The participants expect that
users will get easier interdisciplinary views on a topic, make easier new connections
between topics, get a quick and focused overview up to a quick orientation for new
topics and find new sets of learning resources and improvement of learning cycles.
Tracking cluster: Participants expect that analytics for everyday learning and the app
will support learners in tracking their own learning progress. This covers improve-
ments regarding remembering resources, tracking the progress and making the learn-
ing path visible.
Systemic cluster: Participants expect that the analytics for everyday learning will sys-
tematize the informal learning and provide suitable guidance to the learners. Specifi-
cally, improvements like being more flexible in the learning schedule, getting a better
organization for the learning process by becoming more systematic, getting a better
focus on a topic and being able of multi-topic learning at one place are expected.
Feedback & Certificate cluster: The focus group participants expect that learners are
more satisfied with the feedback in informal learning and that this could support certi-
fication of informal learning activities. In this regard improvements like the easy pro-
duction of reports, the ability to support certification of informal learning processes
and to make the learning visible and tangible, were discussed.
Social learning cluster: The participants expected that the app and specifically the
recommender based on group data will increase social learning. Specifically, im-
provements in learning together across time zones and languages are expected and the
app could serve as enabling factor that motivates to learn and to achieve something in
the group. Also the gamification component was mentioned and the possibility to
better reflect together on the learning and to expand learning experience of others.
Goal cluster: The participants expected that the app will support learners in defining
their learning goals and tracking their status. Within the focus group concept-driven
learning to set learning goals for informal learning was discussed.
After the clustering, the voting was conducted and the importance of the clusters was
ranked as follows: Discovery (1st place), Social Learning (2nd place) and Goal (3rd
place). A detailed voting results can be found in Table 1.
Table 1. Focus group I: Detailed voting results per cluster.
Cluster Gold Silver Bronze
Discovery 7 1
Social Learning 1 1 1
Goal 1
Systemic 3 2
Motivation 2 5
Tracking 2 1
Certificate
9
RQ2: Indicators for measuring the improvements
Round II: Based on the voting in the first round, we selected the four most relevant
clusters for this round: motivation, discovery, goal and social learning cluster.
For the motivation, discovery and goal cluster, we received 2 indicators per cluster,
for the social learning cluster we got 4 indicators as depicted in Table 2. The most
relevant with respect to the recommender are the following two: usage of recommen-
dation and the diversity indicator. The first indicator measures if the suggested rec-
ommendations are really used, while the second measures how participants refer to
receiving recommendations from a diverse set of topics.
Table 2. Focus group I: Indicators for the four clusters motivation, discovery, goal and social
learning.
Cluster Indicator
Time of learning/the time spent as learning time
Quality of learning
Motivation
Attitude towards learning
Achieved learning goals
Usage of Recommendation
Discovery
Diversity Indicator
Number of selected goals vs. achieved goals
Goal
Dropout rate
Number of connected peers
Community of practice assignment
Social Learning
Number of shared resources
Number of communication
4.2 Focus Group II
RQ 3: Improvement of teaching/learning with the dashboard for teachers and
learning resource recommender
Round I: Altogether 36 expected achievements were put by the participants on the
blackboard to the 6 initial thematic clusters, that were reorganized in 7 clusters, when
after the contributions ‘Evaluation’ was added. The following clusters organized the
contributions with 3 to 7 achievements each: Motivation (6), Discovery (6), Tracking
(4), Systemic/Organization (7), Evaluation/Feedback & Certificate (6), Learning Goal
(3), and Personalization (4).
Motivation cluster: teachers agreed that the analytical tools will favor the motivation
to learn. The achievements referred to: seeing one owns advances and how one can
overcome objectives; comparison with other students finding out your strengths and
weaknesses; the importance to perform searches of learning content connected with
learning goals as an element of motivation. From the teacher’s point of view, they
mention that the tools could help them to encourage the pleasure of learning as learn-
ers feel motivated seeing their improvements, and students could be more motivated
10
with activities that are measured in the dashboard if it has an impact in the evaluation
without the pressure of an exam. Gamification was also considered in this cluster.
Discovery cluster: as in the first focus group, participants thought that analytics for
everyday learning and specially the recommender will support the discovery of suita-
ble learning resources. They expressed that the recommendations could help to find
activities related with the topic being studied and optimize the time dedicated for
learning, to discover subtopics or new areas of interest and to select ‘extra’ or com-
plementary contents based on the automatic suggestions. They also stated that the
tools could help to know better the interests of the students and use this information to
prepare the classes (connect activities, examples, etc. with the students’ interests).
Tracking cluster: most achievements expressed by participants were related to the
evaluation of students’ work although they preferred to keep some of them in this
cluster (evaluate the competence of ‘learning to learn’, the attitude, the time spent to
achieve goals). One participant also mentioned that with these tools it could be possi-
ble to check if the workload is adequate.
Organization/systemic cluster: participants’ opinion was that analytics tools are po-
tentially useful for the learner to organize oneself and for the teacher to organize work
with students. The arguments they gave for this opinion is that the tools presented
give information that can help students to optimize the time to learn (time spent, con-
sider more and less productive hours…), to focus on the learning points to be im-
proved based on progress shown in the dashboard, and to figure out if the student has
understood the workflow.
Evaluation cluster: (equivalent to Feedback & Certificate): Participants agreed that
analytics and the dashboard will support evaluation of students, and also self-
evaluation helping to increase learner’s autonomy. They insisted again in the contri-
bution to their evaluation without the pressure of the exam. They highlighted that it
would allow to detect if the learner is focused or distracted from the learning goal or
topic, and help to detect causes of diversion.
Learning Goal cluster: In relation to this cluster, the analytics tools and dashboard
considered by the teachers of the focus group would help to go into detail in the learn-
ing goals, to reach a variety of contents, to measure comprehension and progress and
to create interest to learn (not only to pass the exams).
Personalization cluster: participants expected that this kind of tools would improve
personalized learning. On the one hand, it could be possible to personalize learning
based on a better knowledge about own capacities and progress and, on the other
hand, recommendations could be personalized (they talked about the model ‘resources
seen by people with the same interests’). They also mention that it would be interest-
ing to compare oneself with other users with the same interests as you and see their
profiles (even anonymous profiles). From the point of view of the teacher, the tools
would allow more precision in focusing on the content types.
After the clustering, the voting was conducted and the importance of the clusters ac-
cording to these teachers’ view was ranked as follows: Systemic/Organization (1st
place), Personalization (2nd place) and Motivation (3rd place). A detailed voting re-
sults can be found in Table 3.
11
Table 3. Focus group II: Detailed voting results per cluster.
Cluster Gold Silver Bronze
Discovery 1 1
Learning Goal 1
Systemic/Organization 3
Motivation 1 1 1
Tracking 1 1
Evaluation (Certificate) 1 1
Personalization 2
RQ4: Indicators for measuring the improvements
Round II: Similar to the focus group I, we asked the participants which indicators
could be used in the defined clusters. Based on the voting in the first round, we se-
lected the most relevant clusters for this round: organization, personalization and
motivation. The discussion was also opened for other clusters. In this case, we did not
use the blackboard and participants expressed their opinions directly. The contribu-
tions were not too precise to connect with a specific cluster or to define indicators, as
they talked about what they suggested to be measured but not the indicator to do it.
5 Discussion
Our participants of both focus groups were positive about a tool supporting everyday
learning and they supported our claim that analytics can support everyday learning. In
this regard, learners expect that analytics for everyday learning will primary enhance
the motivation to learn, the discovery of learning and the social learning. In addition,
teachers expect advantages in organizing their work, improving self-organization by
students/learners and in delivering personalized contents to learners.
Regarding the motivation, both groups expect that gamification is a major driver
for enhancing the motivation to learn. Especially due to the visualization of different
kinds of indicators, the learners can be motivated to try something new and to en-
hance the score. This is particularly boosted by the social learning component in
which learners can compare their achievements with the achievements of their peers.
Social learning, which has been proven to be very valuable for self-regulated learners,
may not only help them to find peers with the same interest but may also be perceived
as motivation to proceed with their learning through, for example, gamification or
competition approaches [13]. Our findings in this regard are in-line with research on
gamification in learning and supports our claim that analytics for everyday learning
can enhance the motivation to learn.
Regarding the discovery, both groups expect (1) that the discovery of new learning
materials will be easier for the learners and (2) that the learners will find new and
different learning resources. Both are perceived as positive effects. The first effect
12
reduces the cognitive load and time needed for search and the second effect broadens
the scope of learning. The second effect is mainly expected due to the recommender
which should deliver suitable learning resources. Hence, our participants support our
claim that analytics for everyday learning make learning easier and broadens the
scope of learning.
Regarding the organization of learning, teachers expect a reduction of workload
and learners a better organization due to the definition of learning goals. For teachers
the analysis of their classes is expected to provide better overviews about the learning
progress and the learning goals of their students. This saves them time on the one
hand but also enables them to provide more personalized teaching. The learners ap-
preciate the setting of learning goals, as this allows them a better organization of their
informal learning. In contrast to formal learning with its defined structures (i.e. cur-
ricula), providing such scaffolding structures in informal learning is particular rele-
vant [13]. Hence, our interviewees supported our claim that analytics for everyday
learning helps to organize and to systematize everyday learning.
6 Conclusion and Future Work
In our work, we conducted two focus groups exploring the effects of analytics for an
everyday learning recommender and a dashboard. As a result, three main claims are
developed: (1) analytics for everyday learning can enhance the motivation to learn,
(2) analytics for everyday learning make learning easier and broadens the scope of
learning, and (3) analytics for everyday learning gives information that can help to
organize and to systematize everyday learning.
For future work, we plan to use the insights gained in the focus groups to evaluate
and improve our tools, together with the results of other evaluation studies. For exam-
ple, in case of the learning resource recommender, we will focus on providing a good
tradeoff between diverse and topic-specific recommendations. Specifically, by
providing topic-specific recommendations, we aim to support our first claim (i.e.,
enhancing the motivation to learn) and by providing more diverse recommendations,
we aim to support our second claim (i.e., making learning easier by broaden the scope
of learning). Finally, to support our third claim (i.e., helping to organize and systema-
tize learning), we plan to add a filter functionality to our recommender system. This
would allow the user to filter the recommended learning resources by topics (e.g., to
only receive recommendation related to mathematics).
In the second study with teachers, we also wanted to collect some feedback for the
further development plan of the application related to the teacher’s dashboard and
some additional functions for learners, which include the option to share activity with
others, the visualization of aggregated data and comparison with own activity, as well
as the visualization of students’ activity (aggregated and filtered by individuals) by
the teacher. In addition, we also wanted to identify how participants found current
learning indicators, utilities and their presentation in the dashboard, as well as wheth-
er they considered that other additional analytics activity data would be desirable.
13
Acknowledgements
The authors would like to thank the rest of the AFEL consortium. The project “AFEL
– Analytics for Everyday Learning” is funded under the Horizon 2020 of the Europe-
an Commission (project number GA687916). The Know-Center is funded within the
Austrian COMET Program - Competence Centers for Excellent Technologies - under
the auspices of the Austrian Federal Ministry of Transport, Innovation and Technolo-
gy, the Austrian Federal Ministry of Economy, Family and Youth and by the State of
Styria. COMET is managed by the Austrian Research Promotion Agency FFG.
References
1. De Bra, P., Pechenizkiy, M., van der Sluijs, K., Smits, D.. GRAPPLE: Integrating Adap-
tive Learning into Learning Management Systems. In J. Luca and E.R. Weippl, Editors:
Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecom-
munications, 5183–5188, Vienna, Austria, June 2008. AACE (2008).
2. Drachsler, H. Hummel, H. G. K., Koper, R.: Personal recommender systems for learners in
lifelong learning networks: the requirements, techniques and model. Int. Journal on Learn-
ing Technology, vol 3-4, pp. 404-423. (2008).
3. Duval, E.: Attention please!: learning analytics for visualization and recommendation. In:
Proceedings of the 1st international conference on learning analytics and knowledge, pp.
9-17. ACM (2011).
4. Fessl, A., Thalmann, S., Pammer-Schindler, V. Adaptive and Reflective Training Support
for Improving Search Behaviour in Industry 4.0. WM, pp. 125-126. (2017).
5. Fessl, A., Wesiak, G., Pammer-Schindler, V. Transfer of Theoretical Knowledge into
Work Practice: A Reflective Quiz for Stroke Nurses. Knowledge Management in Digital
Change, pp. 291-308. Springer, Cham. (2018).
6. Jaiswal, A. K., Singh, N., Ahuja, N. J. . Learning styles based adaptive intelligent tutoring
systems: Document analysis of articles published between 2001 and 2016. International
Journal of Cognitive Research in Science, Engineering and Education:(IJCRSEE), 5(2),
pp. 83-98, (2017).
7. Ley, T., Cook, J., Dennerlein, S., Kravcik, M., Kunzmann, C., Pata, K., Purma, J., Sandars,
J., Santos, P., Schmidt, A., Al-Smadi, M, Trattner, C. Scaling informal learning at the
workplace: A model and four designs from a large-scale design-based research effort. Brit-
ish Journal of Educational Technology, 45 (6), pp. 1036-1048. (2014).
8. Nabizadeh, A. H., Jorge, A. M., Leal, J. P. . Long Term Goal Oriented Recommender
Systems. In WEBIST, pp. 552-557. (2015).
9. Santos, J. L., Verbert, K., Govaerts, S., Duval, E.: Addressing learner issues with stepup!:
An evaluation. In: Proceedings of the Third International Conference on Learning Analyt-
ics and Knowledge, LAK ’13, pp. 14-22. ACM, New York, USA.(2013).
10. Schäper, S., Thalmann, S. Addressing Challenges for Informal Learning in Networks of
Organizations. In ECIS. 2015.
11. Schett, M.A., Thalmann, S., Maier, R.K. Design of Knowledge Analytics Tools for Work-
place Learning" in Proceedigns of LDWA 2016, Potsdam, Germany, CEUR Vol-1670.
(2016).
14
12. Siemens, G.: What are learning analytics? (2010),
http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics/, last accessed
2018/06/11.
13. Steurer, M., Thalmann, S., Maier, R., Treasure-Jones, T., Bibby, J., Kerr, M.. Assessing
Informal Social Learning at the Workplace-A Revalidation Case from Healthcare. In Ge-
NeMe, pp. 251-266. (2015)
14. Thalmann, S. Adaptation criteria for the personalised delivery of learning materials: A
multi-stage empirical investigation. Australasian Journal of Educational Technology,
30(1). (2014).
15. Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., Duval, E.:
Context-aware recommender systems for learning: a survey and future challenges. IEEE
Transactions on Learning Technologies, 5(4), 318-335 (2012).
16. Winne, P. H. Learning analytics for self-regulated learning. In Handbook of learning ana-
lytics, 241-249 (2017).