Reflective learning is an important activity of knowledgeworkers in order to improve future working-behaviours. The insights gained by reflective learning are based on re-experiencing and re-evaluating past working situations. One time- and cost-effective way to support reflective learning is the employment of applications that collect data about working processes, store the data in user profiles, and visualise it in order to provide timely feedback to the employees. However, a single application can only capture part of the data that might be relevant for reflection and the parallel use of several applications leads to high demands on the user regarding the interpretation of relationships between several single visualizations. A combined visualisation of data captured by different apps should enhance the support for reflection about the working behaviour and experiences. This paper introduces an overlapping user profile application, which combines and aggregates data captured by various applications. The goal of this overlapping application is to provide higher-level reflection possibilities by combining visualisations of different application data in order to better induce and support reflective learning at work. A first proof-of-concept of such an approach indicates that a combined user profile application and especially it's visualisations can be beneficial with regard to reflective learning and can enhance the awareness about the multiple aspects of a user's work life.
Today’s work environments are constantly becoming more complex, globally
integrated, and knowledge-centric. This simultaneously leads to a stronger need
of employees who are motivated and capable to reflect upon their activities and
as a consequence adjust their working practices to new demands. Especially for
knowledge-workers, reflective learning is an important activity to re-experience
past situations during work and to learn from them in order to improve their
future working-behaviour [
In order to support reflective learning at work, within the EU-funded project MIRROR (http://www.mirror-project.eu) several applications have been developed, which aim at motivating and activating users to reflect upon their individual working experiences. After the reflection process itself, knowledge workers should have gained some benefits or insights for themselves and as consequence derive and apply behavioural changes for future working situations. These changes should permanently improve and facilitate the handling of upcoming similar situations or experiences.
The applications developed within the MIRROR project have been applied
within a wide range of working environments (e.g. care homes, hospitals, IT
companies, and emergency situations) and support various sets of professionals (e.g.
knowledge-workers, nurses, physicians and carers as well as emergency workers).
Each of the developed applications collects and gathers different kinds of data
and stores them in their corresponding user profiles. This data encompasses on
the one hand information about the user. On the other hand it consists of
information on users’ work processes, which is captured automatically or inserted
manually during the user’s work. Examples are application switches, application
usage and documents used while working on a PC as well as manually inserted
data such as the current mood status of the user, individual notes, feedback
on different working situations, ratings, scores of serious games or quiz results.
The collected data is stored within the applications themselves and for some
applications additionally in the so-called MIRROR Spaces Framework [
With this paper we want to present a first approach on how to meaningfully combine and visualize data captured by different applications. The goal is to provide a greater variety of reflective learning opportunities in order to facilitate deeper insights on one’s working experiences. We are aware that this approach raises privacy and security issues which need to be carefully considered when employing the app in a real working environment. However, for this first approach privacy and security were only of secondary interest, but will of course be treated in upcoming research settings.
Therefore, we developed the so-called ”MIRROR Integrated User Profile” application (MUP App) which has the task to integrate, summarise, analyse, and visualise data captured by several different applications in order to induce and support reflective learning at work. For a first proof-of-concept, we used two different applications in parallel, namely KnowSelf and the MoodMap App. KnowSelf automatically captures work activities on a PC, whereas the MoodMap App allows knowledge workers to easily state their moods during a working day. We collected, aggregated, and visualised data from a small sample of knowledge workers to to get a first impression of users’ interest and motivation and the app’s usefulness. From this we derived the following three research questions: – RQ1: Are participants interested and willing to use more than one application in parallel with regard to reflective learning? – RQ2: Does the MUP App as overlapping application facilitate reflection about users’ working experiences and contribute to raising awareness of multiple aspects of their work life? – RQ3: Do participants perceive any individual insights or benefits for themselves? 2 2.1
Since the terms user profile and user model are not always used in exactly the
same way, it is essential to clarify our understanding and usage of the term user
profile, which we base on existing theories regarding user models and on our
understanding in MIRROR described by [
In MIRROR we prefer the term user profile (UP). Although the MIRROR user profile (MUP) is based on theory and research of user models, the term user profile better reflects its mission in MIRROR. First, the purpose of the MUP is to guide and support reflection by mirroring user data in the form of activities, experiences or artefacts of work, notes and insights, moods, work practices, and other concrete data sources back to the user. Secondly, we intended these user profiles to be created and maintained by a mixture of automated methods and manual management, where the process of editing or updating the data may also explicitly trigger reflection. 2.2
Although learning analytics is not in focus of our work, several approaches,
methodologies and technologies of this research area are closely linked to
reflective learning. Learning analytics deals with methods for analysing and detecting
patterns within data collected from educational settings or learning
environments about the learner, and leverage those methods to support adaptation,
personalisation, recommendation, and also reflection. Siemens [
The main focus of learning analytics is to support the learner while learning
in an educational setting or learning environment. Although learning analytics
includes also reflective learning approaches (e.g. [
Individual reflection takes normally place in every day’s life and obviously also
during work or work- related situations. Reflection may be triggered by different
reasons for example by conflicts or problems, by unexpected experiences or by
a person acting in a complete different way in comparison with the individual
(external trigger). But also if an individual feels uncomfortable, for something
bothers her or an inner voice is nagging, without being able to make this feeling
external (internal trigger). As reaction, a reflection process may be triggered with
or without the awareness of the person. This reflection should lead to an
individual insight or outcome which may be used to guide or adapt future behaviour.
Within MIRROR we follow the definition of [
Bringing this together, reflection is both a crucial part of learning and a
response to past work experiences. These experiences as well as the behaviours of
the individual engaged serve as starting point for the reflective process. The
desired outcomes of reflection may lead to personal synthesis, integration of
knowledge (internalisation), validation of personal knowledge, a new affective state or
the decision to take on actions for future events. To achieve these results the
characteristics of the individual (learner) have to be taken into account as well
as the intention of the individual self. Individual reflection may occur
spontaneous and unconsciously and in any possible situation especially then when it is
not expected [
In the scope of the MIRROR project a series of applications supporting
individual reflection have been developed and evaluated in different settings [
KnowSelf automatically captures work activities (used applications and
resources together with the exact time of use) on a PC, provides simplistic project
and task recording and presents an overview as well as different visualisations
of the captured data [
The MoodMap App is a web-based application, which allows knowledge
workers to track their mood during a working day or virtual meeting and
recapitulate their work experiences afterwards. The MoodMap App provides an
easyto-understand user interface to state individual mood points by simply clicking
on a bi-dimensional coloured map. Each mood is composed of two dimensions,
namely valence (negative to positive feelings) and arousal (low to high energy)
based on the model proposed by [
Insights from evaluations conducted separately for KnowSelf and the MoodMap
App led to the conclusion that single applications capture only part of the data
that might be relevant for reflection [
In order to efficiently implement a common MIRROR user profile, it is of crucial relevance to use data captured and gathered by several MIRROR applications and not only by a single one. To achieve this, we employed the MIRROR Spaces Framework, an underlying data storage system developed within the MIRROR project to store and exchange data of the applications. For the development of a common user profile based on the MIRROR spaces the following prerequisites have to be considered: assumptions regarding (i) data, (ii) reusability, (iii) sharing, (iv) privacy and security, and (v) accessibility by the user interfaces.
Data stored in the MUP can be divided into three different types, namely personal data about the user, private data, and shared data. Personal data about the user consists of general information about the user (e.g. name or email address) and login information. For the data implicitly captured by the MIRROR applications (e.g. work history in KnowSelf) as well as data explicitly inserted by the user (e.g. mood in the MoodMap) it is essential that the user has full control over her data by deciding for each type of captured data, whether it is private or can be shared.
Reusability is one of the major potential benefits of the MUP. By storing the data according to a predefined data format, applications are able to reuse not only their data but the data captured by other applications and other users as well. Account information can be stored once in the user profile and then be used by all MIRROR applications.
Sharing data is of major relevance for reflection in order to provide possibilities for comparing one’s own data with that of colleagues or a whole team. To account for different levels of sharing, settings (e.g. anonymised, sharing within the same team or department) should be very fine-grained.
As mentioned above privacy and security are a major concern when storing data in the MIRROR Spaces Framework. It has to be ensured that the privacy settings defined by the users via different applications are always met by all applications, aggregations, and visualisations.
Sharing, privacy and security settings along with other data gathered either explicitly or implicitly by applications, should be accessible and modifiable by user-friendly interfaces and visualisations provided by each MIRROR application. This has the advantage, that the user has full control about the data and has the potential to decide on a very fine-grained level, which data she wants to share with whom and which data should be kept private only. 4.2
The MIRROR Integrate User Profile Application (MUP App) serves as a bridge between the MIRROR Spaces Framework and various MIRROR applications. It provides services for data administration as well as for directly supporting reflective learning. The latter is achieved by making users aware of unusual or significant behavioural patterns. The MUP App’s service can be used by other applications to show and promote reflective learning by presenting combined data aggregated by different applications or from different users.
The tasks of MUP App are two-fold, providing (a) access to the data stored in the corresponding user profiles per user within the MIRROR Spaces Framework and (b) a data analysis service, which aggregates data from different applications (on an individual level) and/or from different users (on a collaborative or organisational level). The aggregated data can be used to raise awareness on relationships between data captured from different applications, make comparisons along a timeline or among different users, and finally detect patterns that are relevant for individual or collaborative reflection. Reasons for reflection can encompass the need for problem solving, decision-making, emotion regulation, or detection of significant deviations between the individual user and a team.
In this first phase of the development we pursue a more general approach directed towards basic types of data that are comparable across different applications. We mostly focus on statistical analysis to extract information on for example the number of different applications used by an individual, on providing a chronological overview of the applications used, on presenting the number of entries in various diaries, and on general information (e.g. when, how often or which data) was captured by each application. The data is presented on different types of charts, which can be selected by the user in order to ensure that the chart fits to the available data. In addition, the user may visualise her data in direct comparison with the data to other users (e.g. her team-members).
For the second phase, we will be concentrating on the different types of data captured by various applications, in order to provide analysis on the combined data. For instance, combining the usage of the MoodMap App with data captured by KnowSelf, might show a relationship between moods and specific working tasks. This would lead to new insights that may be the basis for initiating reflective learning. As an example, the left of Fig. 2 shows the hourly application usage history of a single user for both KnowSelf and the MoodMap App. The picture on the right of Fig. 2 visualises combined application specific data, namely the number of hourly switches between tasks or resources captured by KnowSelf and the corresponding mood of a user, depicted as separate lines for arousal (mood.energy) and valence (mood.feel) by the MoodMap App. For this visualization, mood values from the MoodMap App depicted in Fig. 1 are expressed as numbers between 0 and 100. In order to investigate the potential of a common MIRROR user profile as support for reflective learning, we conducted a small combined user study employing the KnowSelf and the MoodMap App in parallel. Although we used only two application for this first evaluation, the MUP App is able to handle all applications that store their data in the common user profile. Based on what we have learned from the separate evaluations, we see this study as first proof-of-concept for the MUP App. The goal was to find out whether a combined analysis of data from both user profiles will i) be accepted by the users, ii) enhance the boost of reflective learning, and iii) provide clearer insights or benefits for the single user. The participating team consisted of 6 knowledge workers (3 women, 3 men), on average aged between 30 and 40, all of them mostly doing computer work. They used the KnowSelf and the MoodMap App in parallel for two weeks during work. Each day, either in the morning or in the evening they were asked to re-evaluate and reflect about their captured data and write down their insights and thoughts directly within one of the two applications. User activities automatically logged by KnowSelf could only be analysed for 5 persons due to technical reasons on one of the PC’s. At the end of the trial the participants were asked to fill in a questionnaire and to take part in a semi-structured interview.
The questionnaire covered information regarding features and functionalities of the applications, usage, and reflective learning. During the interview, combined statistics (see Fig. 2) of the captured data were presented and discussed in order to find out the insights and benefits gained for the individual user. 5.2
The analysis of the log data of both applications is depicted in Fig. 3. Because of the small sample size only descriptive statistics are presented. As measure of central tendency the median is used for the same reasons. Each data point represents the average mood values (in terms of valence and arousal) of one participant in relation to the application usage and working activities (switching frequency and used resources). Whereas there is no trend to be derived from this small sample for the active use of KnowSelf, the number of moods entered per day seems to increase with higher valence and higher arousal values indicated by the participants (i.e. with a more positive mood). Switching frequency was measured in seconds between switching from one resource to another. Fig. 3 (bottom) shows that higher reported valence seems to be connected to longer times between switches (that is a lower switching frequency) and fewer resources used. Interestingly, the arousal level increases with the number of used resources.
Analysing the data collected via questionnaires and interviews, we can give first answers to the research questions:
RQ1: Are participants interested and willing to use more than one application in parallel with regard to reflective learning? Ratings from 6 participants answering the questionnaire (using 5pt. agreement scales) indicate that there is an interest in getting support for time-management (Md (median) = 4) as well as in capturing one’s working activities, own mood, and the team mood (all Md =3.5). Participants found the applications easy to use, liked their visualisations, rated the presentation of information as comprehensible (all Md =4), generally liked using the applications and would recommend them to colleagues (for both items Md =4 for KnowSelf and Md =3.5 for MMA, respectively).
RQ2: Does the MUP App as overlapping application facilitate reflection about users’ working experiences and contribute to raising awareness of multiple aspects of their work life? The interview results revealed that the combination of data has high potential to trigger reflective learning although we have ambiguous statements in which way. One participant reflected mainly on the number of used applications and its relation to how the level of arousal developed over the day. Another participant mentioned that combined data helped her to detect a working pattern, which occurred especially in the morning. After reading emails the application switches and the arousal level increases, thus she knows that she started to work. Similarly, one of the participants observed that her arousal level is very low in the morning and increases during the day. This was a trigger to compare her arousal level to the average level of her colleagues and reflect upon eventual differences between them. An important feature mentioned by more than one participant was the overlapping visualisation of captured data on the timeline chart. Here the data was understood at one glance, which can facilitate reflective learning and enhance awareness of the multi aspects of their work life. Despite of the different approaches to reflect, for all participants the combination of data captured by both applications was important to understand the relationship between their working activities and moods.
RQ3: Do participants perceive any individual insights or benefits for themselves? Besides the findings already described in relation to RQ2, participants reported some additional insights they gained by reflecting on the captured data provided by the MUP. One participant stated that her arousal level fluctuates during the day. By becoming aware of the falling arousal level she decided to take smaller breaks to better recover during the day. Further insights concerned participants’ self-estimations of how they spend their working day. Whereas one participant stated that the captured data confirmed how she estimated the relationship between working activities and mood development, another participant was rather surprised in the first place. Although she was six to seven hours in the office she spent only four hours in front of her computer. Only after comparing this awareness with her dates in her calendar, she could reproduce her day and explain why this happened.
General discussion. In general, this first proof-of concept of the MIRROR Integrated User Profile indicates that such overlapping visualisations can facilitate individual reflective learning and raise overall awareness of users’ work life. All six participants used the combined data to reflect on how their working activities are related to mood changes and could gain some individual insights. Nevertheless there a still some points which need further discussion. While KnowSelf captures automatically the resources and applications used on the PC, the moods need to be inserted manually. Having to repeatedly insert a mood in a web based application can distract from the normal working process. One recommendation to alleviate this distraction was to add five different smileys in the system tray to facilitate the mood capturing. Another point for consideration is the optimal time for reflection. All of the participants perceived the combination of the data captured by the MoodMap App and KnowSelf as useful, because they could check at all times what they were doing during work and how they felt. However, one participant stated that it was not very useful to reflect on how she felt three days ago, but that it was more interesting to become aware of her mood in relation to her work directly while working. For other participants especially the knowledge of how they felt for example three days ago was very important. Especially when the mood could be directly related to the mood note, used applications or used resources. A rather interesting statement from one of the participants was that her working tasks did not influence her mood at all. With respect to the visualisations, the interviews showed that different types of aggregating the data would be useful, so that users could indicate their individual preferences, e.g. to visualise the data along a timeline, to aggregate on an hourly basis, or to offer a summarising view in form a pie chart.
In summary the MUP App provides new visualisations based on data captured by different applications, and therefore offers a multitude of new possibilities for individual interpretations. In our proof of concept, we only combined data of two applications, but also within this small setting we received different approaches on how the participants interpreted the captured data for themselves and what they learned from it. We also mentioned some shortcomings which must be taken into consideration when proceeding with the development of the MUP App. Nevertheless, our findings encourage the assumption that combining data of more than two applications, leads to more meaningful possibilities to interpret the data and to gain more diverse insights for oneself. 6
In this paper we presented the new MIRROR integrated User Profile Application, which aims at supporting reflective learning at work. Based on the results from previous user studies, which evaluated single applications, we derived essential requirements for the development of the MUP App and implemented a first prototype. Results from a first small evaluation regarding the parallel usage of two applications indicate that combining data captured by different applications, analysing and visualising them together can further facilitate reflective learning. Furthermore, it can also enhance awareness of the work life by leading the users to get more diverse insights about themselves. Of course, after this first proof-of-concept, user-studies with larger samples and more applications need to follow. Thus, our future work will focus on the integration of further applications developed within the MIRROR project into the MUP App. The goal is to provide different variants of visualising combined data and more sophisticated ways to provide guidance for reflective learning. For example, Fig. 4 combines KnowSelf data with corresponding geo location data captured by another MIRROR App. Thus it makes you aware of your working activities in relation to your working places (e.g. customer visits or travel activities) and can provide more triggers for reflective learning.
The project “MIRROR - Reflective learning at work” is funded under the FP7 of the European Commission (project number 257617). The Know-Center is funded within the Austrian COMET Program - Competence Centers for Excellent Technologies - under the auspices of the Austrian Federal Ministry of Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.