After a successful evaluation at the IMC in the project MIRROR to enhance reflective learning by coaching and usage of two apps developed in the project, the instructional design was further developed. Below three phases are described that show the development from a research prototype to a commercial blended learning course. The first phase was a summative evaluation. Ten staff members took part in a time management coaching at our company IMC. In the second phase a time management open online-course was created. There the participants were offered to use two apps from the MIRROR project and tele-coaching for a fee. In the third phase a blended learning course was created and evaluated with three human resource managers. It included the two mentioned apps and individual coaching sessions and group sessions. The human resource managers were satisfied with the concept. The course was then included in the internal training catalogue and is currently running.
Reflection on work experiences can lead to new insights and ideas how to handle work
situations better in the future. But the capturing of experiences during work and the
reflection on this data is only a part of a more comprehensive offering needed for a
successful improvement. The other part is the creation of a viable reflection outcome
and the validation of this outcome when it is applied in practice (see [
Another part of the concept is a computer activity tracking app that records the computer usage in a time management scenario. While analysing the recorded data, reflection is triggered and the user is supported in reviewing the personal learning progress. These two apps had the aim to support reflective learning in our different concepts of time management scenarios. But another part was also very important. We wanted to support the participants in using the apps, in reflecting on recorded data and in implementing time management rules. 2
A summative evaluation took place at our company IMC over a period of six weeks
(see [
In the weekly coaching sessions the coach and the coachee reviewed the individual progress, adjusted the time management rules if not appropriate anymore, or, when the particular goal has been achieved and the new behavior has been adopted, decided that no further practice regarding that goal is needed. ARA served well as a support for learning time management by providing a data basis for the coaching sessions. It was used by coach and coachee to set time management goals and to document and monitor the progress in learning new time management techniques. Both benefited from the better preparation for the coaching sessions available with the notes in ARA. Furthermore, the app helped the coachees to focus their goals.
After the evaluation, the coach and several coachees suggested to improve the instructional design of future trainings by forming peer groups to train time management techniques. This would allow participants to benefit from sharing experience data to compare own progress with that of others and learn from each other’s experiences. The study showed that coaching is a good instructional setting to support reflective learning. The coaching approach has increased the reflection in the workplace and coaching serves as a trigger for reflection. People need support in reflective learning. Technical applications as tracking apps or ARA are important, but even here the people often need a guidance of a coach. 3
Based on the insights from the MIRROR project study, IMC has created a free online
course for time management that includes usage of the ARA and KnowSelf (see [
The purpose and use of ARA and KnowSelf is explained in the course with an example scenario. It shows how an office worker uses both applications to improve his time management. In addition, the usage of both applications is explained in several short screencasts.
This MOOC attracted 1000 registered participants in just three months and has now, after one year reached 1600 registered participants.
It turned out there was very little usage of the ARA and KnowSelf (measured by downloads and app registrations). A reason could be that the added value is not visible for learners and their personal use. Another reason could be the complexity of the apps and the related effort at first sight.
In addition, there were only very few requests for tele-coaching. A reason could be that participants of a free MOOC course do not expect to pay for additional services and the cost of coaching is relatively high for a personal budget.
As it is difficult to study MOOC usage in detail and to watch interview individual MOOC participants a new offer was created that is easier to evaluate and optimize. This resulted in a third phase. 4 4.1
Participants of the first study had suggested to form peer groups that work on a common issue. To provide learners with an opportunity to exchange experiences on similar problems we formed a group and a regular meeting. We carried out a time management course with three human resources managers of two companies who had a dual role: participants of the course and at the same time evaluators of our new concept. They should find out if this type of approach is a possible and good program for employees of their company. That should help us to gain further insights into improving our approach.
The concept was a blended learning approach with coaching to enhance reflective
learning and to promote a successful transfer of theory into practice. The course consisted
of three parts: online course, individual coaching sessions and group sessions. Part of
the concept was also ARA and Manic Time. We chose ManicTime because it received
better evaluations in comparison to KnowSelf in the first phase (see [
Online-Course. The time management learning content is only opened for the current participants. There are topic sections that 6ar1e released week after week. The course includes videos, scripts, self-tests, screencasts, apps, exercises and summaries of the group sessions. The participants are able to discuss directly in the portal. Working with the learning content, the participants begin to reflect on their own situation and their personal time management. The idea is that they have already thought about the content before they meet the coach or other participants.
Individual and group sessions. In the weekly one-hour individual coaching sessions the current learning content is discussed, reflected and aligned with the individual situation of the participant together with the coach. They analyze data from the tracking app and discuss the notes in ARA. The coach asks why the recommendation has a certain rating and how it can be improved.
In the weekly one-hour group sessions the participants have the possibility to exchange experiences and to discuss the learning content. They can further discuss exercises in the learning portal as well as the ARA recommendations. Reflection is triggered here by discussions. The participants can propose solutions to each other and motivate each other to follow their goals.
when the user starts to analyze the recorded data. Problems – e.g. spending too much time for unnecessary tasks at work – can be defined. It also supports the user in observing and controlling solution implementation and behavior changes. ARA supports reflective learning by enabling the participants to define problems and solutions. Such solutions are recommendations that the participants have to evaluate for a certain period of time. Step by step they change their behavior by following the recommendations. 4.3
In the last group session the coach wanted the participants to discuss their opinions and experiences. The Coach asked them some questions about how they like the course concept, about the named and reported risks and opportunities at the beginning of the course and about their priorities of the course parts. The statements were reported by the coach.
All in all the human resource managers were very satisfied with the course format. They liked the blended learning approach. The advantage is that a participant learns by watching videos, reading the scripts and doing homework. In this way, he has already collected experiences before having an individual coaching session. So he can discuss problems, ideas and goals concerning the new topic section.
The concept was created with several parts and the intention that all these parts trigger reflection and support reflective learning. We expected that the participants could learn independently while being supported by every part of the concept. After the 12 weeks with the three human resource managers we discussed the results in a group session. They evaluated the concept in their dual role: as participants and then as human resource managers. They provided improvement proposals, said they were convinced of 62 the potential of this instructional approach and asked to provide this as a regular course for the employees.
Risks and opportunities. One of the named risks was work overload (by course tasks adding to the work load of regular work tasks). After the course the group agreed that this became a problem in the Christmas season. This is in line with our findings from the first phase: When participants are very busy, they want to see benefits as soon as possible. But changes in time management practice based on reflective learning need time to unfold and pay off. That is why participants often become unsatisfied during busy periods although this is exactly the time in which the most participants would like to see improvement. Other risks like breach of trust (confidential information leaking out of the group) or frustration (because of reflection leading to realizing problems that cannot be solved) did luckily not materialize.
The hoped-for transfer support of theoretical content into practice took place. The participants confirmed that compared with other conventional trainings this concept really helped them to learn better. The new aspect was that they could combine theory and personal situations and practical experiences. They had time to practice exercises, make experiences and discuss about individual problems and solutions in this course. Behavior changes took place too. This can be seen as the major advantage in effectiveness of supported reflective learning in comparison to conventional trainings. Furthermore, the awareness of personal time management was increased and the participants optimized their work behavior. They even confirmed a reduction of stress. Priorities. Finally, in the group discussion the participants were asked to prioritize the concept parts regarding their personal improvement (see Fig. 1). Here, 1 is the highest and 5 is the lowest priority.
6 5 4 3 2 1 0 Participant 1 Participant 2 Participant 3 1 1 1 It is clearly evident that the individual coaching was the most important for the participants. The reason is that they got suggestions for the practical implementation of the tasks in the individual coaching. They also had a healthy pressure to invest time and 63 energy into their learning as they knew that the coach would observe and ask about personal learning processes. In addition, their personal situation has always been subject and there was space to talk about frustration, but also to find solutions. On the second place are the group sessions. The experience exchange was important for the participants. One participant emphasized, for example, that it helped him to continue with the course content when he has seen how far the other participants are and that they share experiences. A participant complained that it was spoken too much about the course concept in the group sessions. That was because of their second role as concept evaluators. In addition, it was noted that the group sessions should take place less frequently, less than one time per week because there was not enough time between individual and group sessions. In this time the participant cannot make many experiences. Then they often discussed issues with other participants that were already discussed with the coach a short while ago.
The general content of the online course was ranked third. The theory and the exercises contained in the online course provided some basic knowledge. With new methods and time management techniques, the participants were able to improve their time management.
Overall, ManicTime and ARA were regarded as the least important parts. The human resource managers did not see them as essential for their personal progress. Nevertheless, they saw benefits in using the apps. ManicTime can be used to measure work time spent on certain tasks which can for example be used as basis for discussions with the supervisor about too high work load. This is an example of an organizational level problem rather than a personal time management problem that needs to be solved on the organizational level.
ARA was good for documenting experiences and to observe the own learning progress. But as the experiences were discussed in the individual coaching sessions, only the documentation part of reflective learning was supported in this scenario, and ARA had the least benefit in comparison to the other parts of the concept. We created an ARA widget in the course portal to facilitate the use. There the participants were able to directly and quickly document and share experiences, without having to log in separately in ARA. Still the widget was not used by participants as their need for sharing was already covered by the regular group presence sessions.
Proposals for future course runs. The participants were satisfied with the course and saw a high potential in it. As human resource managers, they encouraged us to offer the course with some changes in the internal training catalog. With the support of one of the human resource managers we created an offering for employees.
The participants also suggested a stronger guidance by the coach in the group sessions. In addition, the group sessions should not only function for experience exchanges, but also as training sessions with short exercises. After these exercises the participants could discuss the results.
Since May the instance of the course as an official training offering of the companies group is ongoing. The concept is similar to the course with the human resource managers. There is an online-course with ARA and ManicTime and individual as well as group sessions. While number of group sessions and coaching sessions remain roughly the same, the course duration was extended from six to twelve weeks to reduce the weekly work load of the participants.
At the moment, four employees participate in the course. The participants come from two different companies. The presence sessions take place in one building in Saarbrücken. One participant works in another city far away and therefore he gets telecoaching (and some extra coaching sessions as he cannot participate in the group sessions). He exchanges experiences with the other participants in the course portal, thanks to the ARA widget.
This is a 12-weeks-long course again. But now the individual and group sessions take place alternatively every two weeks. Participants can opt for individual coaching sessions to take place more frequently and for a shorter duration in consultation with the coach. 6
We have learned that the concept works. It supports people in the process of reflective learning. All parts of the concept function as triggers for reflection and together they help participants to implement the reflection outcomes into practice.
We are interested in a continuing development and monitoring of reflective learning based on this concept. We have experienced that awareness and reflection technology together with personal coaching help employees to change work practice. Therefore, we are currently planning more courses and studies based on this concept. We experienced that apps can serve as reflection triggers and can support people in reflective learning. But most often nontechnical parts, e.g. human support in a time management course has the best evaluation results. Our aim is to increase the focus on the applications in future studies and to find out the reason why this is so.