Please refer to these proceedings as c 2014 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. Re-publication of material from this volume requires permission by the copyright owners.
The front-cover was created by Harriett Cornish (The Open University, KMi). Addresses of the editors:
Thomas Daniel Ullmann Knowledge Media Institute The Open University Walton Hall Milton Keynes MK7 6AA Alexander Mikroyannidis Knowledge Media Institute The Open University Walton Hall Milton Keynes MK7 6AA Fridolin Wild Knowledge Media Institute The Open University Milton Keynes MK7 6AA Milos Kravcik RWTH Aachen University Advanced Community Information Systems (ACIS) Ahornstr. 55 52056 Aachen
The #ARTEL14 workshop accepted 4 full papers, 1 short paper, and 4 demo papers. The accepted papers discuss awareness and reflection in diverse settings, such as blue-collar jobs and white-collar jobs, working in small enterprises, or learning at university level. As for the full papers, Maurizio Megliola, Gianluigi Di Vito, Roberto Sanguini, Fridolin Wild, and Paul Lefrere discuss in ”Creating awareness of kinaesthetic learning using the Experience API: current practices, emerging challenges, possible solutions” an interface specification for capturing in particular kinaesthetic learning experiences. The authors also discuss a taxonomy of verbs describing handling and motion. Kinaesthetic skills are in demand for instance in the manufacturing or maintenance sectors. The captured learning experiences can be utilised to generate feedback to the learner.
Fridolin Wild, Peter Scott, Paul Lefrere, Jaakko Karjalainen, Kaj Helin, Ambjorn Naeve, and Erik Isaksson situate their work ”Towards data exchange formats for learning experiences in manufacturing workplaces” in the manufacturing sector. This work models the experiential and reflective learning process with a view on creating data exchange formats, also discussing the necessity to - formally - talk about actions in the real world as a complement to actions performed in virtual environments.
Andreas Janson, Sissy-Josefina Ernst, Katja Lehmann, and Jan Marco Leimeister have as background for their work in ”Creating awareness and reflection in a large-scale IS lecture - the application of a peer assessment in a flipped classroom scenario” university learning in large-scale lectures (much more learners than instructors). They discuss computer-supported peer assessment as possibility to induce reflection on learning content. In contrast to the previous two works, this contributions focusses mostly on cognitive instead of motoric competences.
Angela Fessl, Gudrun Wesiak, and Granit Luzhnica showcase in ”Application overlapping user profiles to foster reflective learning at work” the potential benefits of collecting activity logging data from multiple applications in a single user profile application. This work is set on the background of white-collar knowledge workers, with desktop- and web-based activity logging.
In their short paper, Michael Prilla, Oliver Blunk, Jenny Bimrose, and Alan Brown discuss in ”Reflection as support for career adaptability: A concept for reflective learning in public administration” reflection as learning mechanism to support professional identity change as a means for organisational change in the context of public employment services. The four demo papers are started off by the contribution by Milos Kravcik, Kateryna Neulinger, and Ralf Klamma, in which the authors showcase widget-based personal learning environments for ”Boosting informal workplace learning in small enterprises”. Nils Faltin, Simon Schwantzer, and Margret Jung present the ”Activity recommendation app - software to evaluate the usefulness of improvement recommendations created in a team”.
Min Ji, Christine Michel, Elise Lavoue, and Sebastien George demonstrate the ”DDART: an awareness system to favor reflection during project-based learning”.
Last, but not least, Sven Charleer, Jose Luis Santos, Joris Klerkx, and Erik Duval present the ”LARAe: Learning Analytics Reflection & Awareness environment”.
The official workshop webpage can be found at http://teleurope.eu/artel14 The 4th Workshop on Awareness and Reflection in Technology-Enhanced Learning (ARTEL 2014) is part of a successful series of previous workshops.
3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning (ARTEL13). Workshop homepage: http://teleurope.eu/artel13. Proceedings: http://ceur-ws.org/Vol-1103/. 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning (ARTEL12). Workshop homepage: http://www.teleurope.eu/artel12. Proceedings: http://ceur-ws.org/Vol-931/. 1st European Workshop on Awareness and Reflection in Learning Networks (ARNets11). Workshop homepage: http://teleurope.eu/arnets11. Proceedings: http://ceur-ws.org/Vol-790/ Augmenting the Learning Experience with Collaboratice Reflection (ALECR11). Workshop homepage: http://www.i-maginary.it/ectel2011/index. html 1st Workshop on Awareness and Reflection in Personal Learning Environments (ARPLE11). Workshop homepage: http://teleurope.eu/arple11. Proceedings: http://journal.webscience.org/view/events/The_PLE_ Conference_2011/paper.html#group_Proceedings_of_the_1st_ Workshop_on_Awareness_and_Reflection_in_Personal_Learning_ Environments To stay updated about future events, to share your research, or simple to participate with other researchers, consider joining the group about Awareness and Reflection in TechnologyEnhanced Learning: http://teleurope.eu/artel We especially would like to thank the members of the programme committee for their invaluable work in scoping and promoting the workshop and quality assuring the contributions with their peer reviews.
Alexander Mikroyannidis, Viktoria Pammer,
Michael Prilla, Thomas Ullmann,
Fridolin Wild Organisation committee
Alexander Mikroyannidis, The Open University, United Kingdom Viktoria Pammer, Graz University of Technology, Austria Michael Prilla, University of Bochum, Germany Thomas Ullmann, The Open University, United Kingdom Fridolin Wild, The Open University, United Kingdom Program committee
Alexander Nussbaumer, Graz University of Technology, Austria Carsten Ullrich, DFKI, Germany Denis Gillet, EPFL, Switzerland Ines Di Loreto, Universite´ de Technologie de Troyes, France Jaakko Karjalainen, VTT, Finland John Cook, University of West of England, United Kingdom Joris Klerkx, Katholieke Universiteit Leuven, Belgium Kaj Helin, VTT, Finland Kinshuk, Athabasca University, Canada Marcus Specht, Open University, the Netherlands Monica Divitini, IDI-NTNU, Norway Paul Lefrere, CCA, United Kingdom Peter Kraker, Know-Center, Austria Peter Sloep, Open University, the Netherlands Philippe Dessus, Universite Pierre-Mendes-France, France Riina Vuorikari, European Commission JRC IPTS, Spain Simon Knight, The Open University, United Kingdom Stefan Trausan-Matu, University Politehnica of Bucharest, Romania Stefano Bianchi, SOFTECO, Italy Xavier Ochoa, Escuela Superior Politecnica del Litoral, Ecuador Supporting FP7 projects Supporting FP7 projects http://www.mirror-project.eu http://learning-layers.eu http://www.tellme-ip.eu http://wespot-project.eu
http://employid.eu http://www.boost-project.eu 8
Summary of the contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . Awareness and reflection workshop series . . . . . . . . . . . . . . . . . . . . Organisation committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Program committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Supporting FP7 projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating awareness of kinaesthetic learning using the Experience API: current practices, emerging challenges, possible solutions
Maurizio Megliola, Gianluigi Di Vito, Roberto Sanguini, Fridolin Wild, Paul Lefrere 11 Towards data exchange formats for learning experiences in manufacturing workplaces Fridolin Wild, Peter Scott, Paul Lefrere, Jaakko Karjalainen, Kaj Helin, Ambjorn Naeve, Erik Isaksson Creating awareness and reflection in a large-scale IS lecture - the application of a peer assessment in a flipped classroom scenario
Andreas Janson, Sissy-Josefina Ernst, Katja Lehmann, Jan Marco Leimeister Application overlapping user profiles to foster reflective learning at work
Angela Fessl, Gudrun Wesiak, Granit Luzhnica Reflection as support for career adaptability: A concept for reflective learning in public administration
Michael Prilla, Oliver Blunk, Jenny Bimrose, Alan Brown
Milos Kravcik, Kateryna Neulinger, Ralf Klamma Activity recommendation app - software to evaluate the usefulness of improvement recommendations created in a team
Nils Faltin, Simon Schwantzer, Margret Jung DDART: An awareness system to favor reflection during project-based learning
Min Ji, Christine Michel, Elise Lavoue and Sebastien George LARAe: Learning analytics reflection & awareness environment Sven Charleer, Jose Luis Santos, Joris Klerkx, Erik Duval 3 3 4 6 7 8 23 35 51 65 73 77 81 85
Maurizio Megliola1, Gianluigi Di Vito1, Roberto Sanguini2, Fridolin Wild3, Paul Lefrere3 Abstract. We describe our use of the Experience API in preparing blue-collar workers for three frequently arising work contexts, including, for example, the requirement to perform maintenance tasks exactly as specified, consistently, quickly, and without error. We provide some theoretical underpinning for modifying and updating the API to remain useful in near-future training scenarios, such as having a shorter time allowed for kinaesthetic learning experiences than in traditional apprenticeships or training. We propose ways to involve a wide range of stakeholders in appraising the API and ensuring that any enhancements to it, or add-ons, are useful, feasible and compatible with current TEL practices and tools, such as learning-design modelling languages. 1
Introduction
Apprenticeship today often includes the development of ‘kinaesthetic intelligence’, i.e. the tactile (physical) abilities associated with using the body to create (or ‘do’) something involving highly coordinated and efficient body movements. Prototypical examples of this can be found in the fluid and precise motions of skilled dancers, surgeons, and skilled blue-collar workers. Gardner (2011) links high bodilykinaesthetic intelligence to “control of one's bodily motions, the capacity to handle objects skillfully, a sense of timing, a clear sense of the goal of a physical action, along with the ability to train responses”.
Becoming a skilled kinaesthetic performer through traditional apprenticeship is today largely conceptualised to take years, but this golden age of long-cycle training is quickly disappearing. Professional training environments such as in manufacturing face the challenge that the world of repetition that enabled long cycles of training to be cost-justified is increasingly taken over by short-run or personalised production, 11 using advanced factory machinery that does not require physical dexterity (including teachable robots that can emulate dextrous production-line workers), thereby not only causing shifts in the demand for skills, but at the same time downsizing the economically acceptable time-to-competence.
Achievements in interoperability for technology-enhanced learning (TEL) over the last decade are making it viable to develop TEL-based alternatives to a traditional apprenticeship. For example, TEL is making it possible to author, exchange, and then orchestrate the re-enactment of learning activities across distributed tools, using learning process and learning design modelling languages (Laurillard & Ljubojevic, 2011; Fuente Valentín, Pardo, & Degado Kloos, 2011; Mueller, Zimmermann, & Peters, 2010; Koper & Tattersall, 2005; Wild, Moedritscher, & Sigurdarson, 2008).
What these modelling languages fall short of, however, is the ability to handle hybrid (human-machine) experiences, to teach machines, or to train people at a distance – as required, for example, in workplaces that include robots or software agents, or in workplaces that include distributed participation across companies in a supply chain. In particular, when it comes to bridging between the virtual and the real world, between digital and physical experiences, present means are ill equipped for supporting the capturing, codification, and sharing of hybrid learning experiences.
Moreover, TEL-focused ways of capturing performance and collecting good practice are as of today still resource-intensive, placing a barrier to the spread of innovation and hindering resilience of business.
That barrier can be somewhat reduced through the use of the present version of the Experience API (ADL, 2013), and we claim could be further reduced through possible extensions and complements to the API, the need for which has become apparent in the TELLME project.
The Experience API is is a novel interface specification designed to link sensor networks together to enable the real-time collection and analysis of learning experiences conducted in different contexts, with the aim of providing better interoperability between the different types of participating educational systems and devices. Being precondition to mining and modelling, it forms a centrepiece in the canon of next generation techniques and technologies for capturing, codification, and sharing of hybrid learning experiences.
This paper presents an overview of how the components and concepts of the Experience API are integrated within the TELL-ME project to foster tracking and analysis of learning and training in three different manufacturing environments, namely Aeronautics, Furniture, and Textiles.
We describe within this contribution, how we deploy an open source Learning Recording Store to allow for collecting and reporting learning across the various components of the TELL-ME system. Moreover, starting with the Aeronautics industry, we define a taxonomy of verbs of handling and motion for capturing in particular kinaesthetic learning experiences as required in the helicopter industry, when targeting learning about a defined helicopter model and the connected standard maintenance procedures.
TELLME Learning Locker
The ADL Experience API (short ‘xAPI’)1, formerly known as TinCan API, is an extension of the Activity Streams2 specification, a format for capturing activity on social networks, created by companies like Google, Facebook, Microsoft, IBM etc., that allows for statements of experience to be delivered to and stored securely in a Learning Record Store (LRS). These statements of experience are typically learning experiences, but the API can address statements of any kind of experiences a person immerses in, both on- and offline.
While the core objects of an xAPI statement (Actor, Verb and Object) derive from the core Activity Streams specification, the Experience API has many more defined constructs for tracking information pertinent for the learner (with captured results such as “score”, “success”, “completion”, “attempt”, and “response”), unlike the Activity Streams spec which focuses on the publisher.
In its most basic application, the Experience API allows one system (the activity ‘provider’) to send a message (also known as the ‘statement’) to another system (aka the ‘Learning Record Store’) about something a user has done. Up until now, this process has mostly taken place inside the organisation’s Learning Management System. Anything we wanted to track had to be built as part of the LMS functionality, or needed tailor-made integration. The Experience API now allows sending and receiving data between systems about what someone has done in a more openly defined way (see Figure 1).
This is important, because via the Experience API, any system can send xAPI statements using a standard connection method. This process of sending xAPI statements can happen in systems behind the firewall or openly across the Internet using secure connections. 1 http://www.adlnet.gov/tla/experience-api/ 2 http://activitystrea.ms/
Within TELL-ME, we have implemented an instance of the open-source Learning Locker3 Learning Record Store, a LAMP-based project using MongoDB, PHP, and AngularJS, to begin collecting learning activity statements generated by xAPI compliant learning activities and reporting on such data.
The REST WS interface to the LRS was made available to be used by any component of the TELL-ME architecture to submit and retrieve xAPI statements. An example of such xAPI triple submission (with POST), using curl4, is shown below: curl -X POST --data @example.json -H "Content-Type: application/json" -user 465ea716cebb2476fa0d8eca90c3d4f594e64b51:ccbdb91f75d61b726800313b2aa9f50 f562bad66 -H "x-experience-api-version: 1.0.0" http://demos.polymedia.it/tellme/learninglocker/data/xAPI/statements
The example above is parameterized by a reference to a JSON file named example.json: it contains the actual xAPI statement in form of an actor-verb-object triple: { } "actor":{ "objectType": "Agent", "name": "Gianluigi Di Vito", "mbox":"mailto:gianluigi.divito@piksel.com" }, "verb":{ "id":"http://activitystrea.ms/schema/1.0/watch", "display":{ "en-US":"Watched" } }, "object":{
"id":"http://tellme-ip.eu/media/video/152" }
Once statements are logged, they can be queried again, for example, as needed for constraint validation to check, whether a user actually performed a certain required action step – then requiring additional constraint checker components. 3 http://learninglocker.net/ 4 curl - command line tool for transferring data with URL syntax: http://curl.haxx.se/ 14 5 http://adlnet.gov/expapi/verbs/ 6 http://bit.ly/1zGmAzn
The Learning Locker also allows creating personalized queries by means of its reporting functions, combining data from both learning activities and workplace performance, allowing linking of learning activity to performance.
One important aspect to be considered is the handling of security and the protection of the privacy of learners. In the Experience API, authentication is tied to the user, not the content. The user can be any person or thing that is asserting the statement. The user can be a learner, an instructor, or even a software agent and it can authenticate with OAuth7, a commonly used access delegation mechanism employed by many big names such as Google, Facebook, Salesforce, etc. that eliminates the needs of sharing passwords between applications to exchange data. In this aim, the Learning Locker supports authentication and it is integrated with OAuth 2.0, exposing an API which allows 3rd parties to connect to the API via OAuth 2.0. 7 http://oauth.net/
The <S,P,O> statement vocabulary (Aeronautics industry) Each xAPI statement follows the syntax of providing a subject, predicate, and object. While subjects and objects can vary, predicates (the ‘verbs’) are ideally rather slowly changing and can be defined in advance.
From a cognitive linguistic perspective, there is prior work on defining verbs of motion and handling – and clarifying their relation to the way humans’ cognitively process them. Roy (2005) explains how humans learn words by grounding them in perception and action. The presented theories are tested computationally by implementing them into a series of conversational robots, the latest of which - Ripley - can explain "aspects of context-dependent shifts of word meaning" that other theories fall short of. To construct a vocabulary of verbs that is widely understood, easy to learn, and natural in mapping, the work of Roy provides valuable insights: Roy postulates that "verbs that refer to physical actions are naturally grounded in representations that encode the temporal flow of events" (p. 391). She further details that the grounding of action verbs follows the schema of specifying which force dynamics (out of a limited set) apply and which temporal Allen relations operate (p. 391). Any higher-level composition of action verbs, so Roy (p.392), can be traced back to and expressed in terms of these fundamental temporal and force relations.
This provides an angle for defining and structuring the TELL-ME taxonomy of verbs of handling and motion: it provides a basis for ordering from fundamental to composite actions, also defining their similarities. Moreover, it offers insights on how to map these verbs back to perception: it provides a rationale for how many visual overlay elements are required for an augmented reality instruction to express a certain motion verb primitive. For example, a verb 'pick up' requires the overlay visualisation of a grabbing hand as well as a highlight of the object to be picked up, whereas the motion verb 'move' consists of both 'pick up' and 'put down' actions, thus requiring more visual elements to be specified.
Palmer et al. (2005) further discuss the "criteria used to define the sets of semantic roles" for building verb classes. It provides insights into the argument structure of framesets (and so-called role-sets) of verb classes (Palmer et al., 2005, section 3).
Chatterjee (2001) reviews the cognitive relationship between language and space. He refers to Jackendoff in postulating that the "conceptual structure of verbs decomposes into primitives such as ‘movement’, ‘path’ and ‘location’ (p.57).
From an augmented reality perspective, there is additional prior work of relevance. Robertson and MacIntyre (2009) provide a review of the state of the art in displaying communicative intent in an AR-based system. The proposed taxonomy, however, stays on a level of general applicability across all sorts of augmented reality applications and is lacking the level of handling and motion required for a particular workplace, such as required in learning the maintenance of helicopters. The proposed categories (called ‘style’ strategies) are ‘include’, ‘visible’, ‘find’, ‘label’, ‘recognizable’, ‘focus’, ‘subdue’, ‘visual property’, ‘ghost’, and ‘highlight’ (p.149f). The communicative goals signified by these styling operations for visual overlays are listed as ‘show’, ‘property’, ‘state’, ‘location’, ‘reference’, ‘change’, ‘relative-location’, ‘identify’, ‘action’, ‘move’, and ‘enhancement’ (p.148). Other than the work in cognitive linguistics, here, the focus is clearly defined from a technical and not user angle: helping the user to identify or move an object is on the same level as setting labels.
In the aeronautical field, maintenance operations must be carried out according to official documents issued by the design authority of the aircraft. Such document is called the Maintenance Publication. Usually, it is organised inside the Interactive Electronic Technical Publication (IETP). The AECMA S1000D (European Association of Aerospace Constructors S1000D) is an international standard for development of IETP, utilizing a Common Source Data Base (CSDB). The standard prescribes rules to name, define, and code everything that is necessary to carry out maintenance activities in terms of: • aircraft model; • systems and subsystems of the aircraft; • maintenance tasks; • location of components; • tools; • additional documentations; • miscellaneous.
Every maintenance task is identified by a unique code referring to the Task Category (e.g. ‘servicing’, ‘repairs’, ‘package’), the Maintenance Activity within the category (‘drain’, ‘fill’, ‘remove’, ‘clean’, etc.), and - finally - to the definition which gives the procedure and data necessary to carry out maintenance tasks on a specific system or component. The code allows retrieving both procedures and data necessary to e.g. fill containers with fuel, oil, oxygen, nitrogen, air, water, or other fluids.
For the TELL-ME taxonomy of verbs of handling and motion the following preliminary list was compiled by pilot partners, see Table 1. The initial taxonomy presents the verbs of handling and motion as required in the helicopter industry for a defined helicopter model and the connected standard maintenance procedures. All the actions are handled by the actor ‘certified staff’, which therefore has not been included in the table.
Activity Operation Servicing Examination, tests Examined (visual) and checks
Verb Loaded Unloaded Filled tolerance check) Monitored (condition) Product, system, equipment or component.
Disconnect, remove and disassemble procedures
Disconnected Removed Disassembled Opened for access
Equipment, components, or items.
Equipment, components, or items.
Equipment, components, or items.
Panels or doors (engine bay doors, landing gear doors, etc.).
Repairs and locally Added material make procedures and data
Attached material
Product, equipment, component or item.
Product, equipment, component or item.
Unloaded (download Items. software) Changed (mechanical Structure, surface. strength / structure of material / surface finish of material) Assemble, install and connect procedures Package, handling, Removed from Products, systems equipment or compostorage and trans- (preservation materi- nents, portation al) Products, systems, equipment, or components.
Moved (when in stor- Products, systems, equipment, or comage) ponents.
Material Damaged product, system, equipment or component.
Equipment, components and items.
Equipment, components and items.
Equipment, components and items.
Panels or doors (engine bay doors,
landing gear doors, etc.).
staff to become more efficient, the public demand for higher service quality, role
changes for staff, and, in some cases, alternative service provision (e.g., [
Our work focuses on Public Employment Services (PES) as an example of
challenges faced at public administrations: staff are dealing with more clients in a rapidly
changing labour market and are expected to offer a wider range of services. In many
European countries staff roles are being transformed from offering advice on access
to benefits and available job opportunities towards facilitation and coaching where
staff are expected to support clients in becoming more self-directed and staff are also
expected to understand the labour market better and engage more with employers.
Staff therefore need to be capable of adapting to various and often unforeseeable
changes. Career adaptability [
Related Work
Career Adaptability and Professional Identity Transformation
Career adaptability is the ability to manage successful transitions in employment,
training, education and other contexts. It is key for workers dealing with constantly
changing requirements on the labour market [
Public Employment Services (PES) practitioners deal with career adaptability both
in their personal careers and in the careers of clients they are supporting. Therefore
supporting them in career adaptability not only supports their personal career but also
supports their clients to re-enter the labour market.
Following Boud (1985) [
In addition to most models we differentiate between individual reflection as a
mostly cognitive activity and collaborative reflection, which is done in
communication among peers in a group [15]. The latter has been found to create results that
transcend the capabilities of a group’s members [
It has been shown that reflective learning can be supported by technology (e.g., [15–17]) by providing data or written content on experiences to reflect upon, supporting retrospective analysis or by scaffolding the reflective process, the documenting and sharing of a decision rationale. More specifically, writing down positive or negative experiences and being prompted regularly to think about them has been shown to be supportive for individuals to engage in continuous reflection [18]. However, as most existing work either supports early phases of reflection (e.g., gathering and sharing data) or stems from educational settings, which are often designed in favour of reflective learning, there is still work to be done in the context of reflection at work. 3
Reflection for Professional Identity Transformation: A
Concept
The development of career adaptability relies on four key dimensions: learning to
adapt through challenging work, through updating a substantive knowledge base, by
being self-directed and self-reflexive as well as learning through interactions at work
[
Dealing with challenging work can bring up discrepancies in daily work, which (as described above) trigger reflection [13]. Successfully dealing with these situations can lead to confidence in one’s skills and abilities. Reflecting about work and its challenges comes into play when there are no problem solving patterns available for the challenges met and new solutions are needed [19].
To keep up with knowledge in changing fields of work learning through updating a
substantive knowledge base is required. While workers often use formal learning
offers at work, informal learning can be seen as a key to continuously understand
which knowledge is needed and integrate it into one’s context [
Adapting through self-directed learning and self-reflexiveness is closely related to individual reflection. Tools can help to sustain issues to be reflected upon and to create awareness for them [12, 18]. This combines self-directed and externally triggered reflection, for example by setting up and periodically reviewing career goals in a tool.
Career adaptability by learning through interactions at work can benefit from support for collaborative reflection. Tools can help to create opportunities for reflection even if individuals cannot meet in person [15]. Individuals can support informal learning of their peers by providing their experiences and insights or helping them to reflect about their own learning. Additionally colleagues can reflect to support each other, for example, in coping with emotional work and/or stress and in exchanging best practices in dealing with difficult situations. A team can reflect collaboratively to improve their team performance and organize their learning efforts. 4
Applying the Concept: Reflective Learning Needs in Practice Our work is inspired by field visits, workshops and expert interviews at different European Public Employment Services (PES) agencies, including Germany, Slovenia and the UK. In an early phase of this work we are currently exploring needs and opportunities for reflective learning as well as constraints and potential of implementing it in such workplaces. From this work we describe examples of challenges faced in many European PES and how reflection can be a key process in tackling them.
Supporting Change by Reflection on Training In one of the agencies (referred to as agency A in this paper) staff are supposed to change from providing advice and guidance to clients on benefits and job opportunities to coaching them to become more self-directed and to take responsibility for their own future by proactively looking for ways to develop their skills and possible future career paths. To support this change staff receive a two-day training on coaching methods and related topics and an additional half-day session some time after training to support the application of the methods in practice. Despite this support, staff members reported that they had difficulties in implementing this new way of working, and that they were struggling in reaching good results from coaching their clients.
This situation is an example of challenging work, and it shows how workers struggle with updating their personal knowledge base. Reflecting on their practice of using methods and tools of coaching can help PES practitioners to conduct more successfully the transition to be a coach and thus may make training more sustainable. This may approach mostly benefits from individual reflection of goals stemming from training and involving workers in this reflection continuously (by reminding them to reflect).
Supporting Interaction with External Stakeholders by Reflection In agency B the government requires PES staff to cooperate closely with employers to enhance the conditions of the labour market, including the creation of new jobs, new 68 fields of employment and career opportunities. Staff are motivated to adapt to this strategy, but also told us that this does not come easy and that there is a need for good practices in implementing it. Some reported that talking to colleagues from other subsidiaries had given them insights into how they might improve this work.
Becoming a co-operator with employers can be seen as an example of challenging work, and from the feedback of practitioners we can see which discrepancies it causes. We can also see that there is a desire to engage in exchange with others to reflect on such discrepancies. Collaborative reflection on their work with employers can therefore be seen as a means to make sense of typical challenges in this work, to exchange work practices and to learn from each other. 5
A Prototype for Reflection Support The scenarios above show that support needs for learning about challenges Public Employment Services (PES) practitioners face are diverse, and that support for sustaining experiences, reflecting upon them, sharing them and finding similar experiences need to be close to work tasks. To explore how such support can create impact in PES agencies we created a mobile prototype supporting the reflective learning scenarios describe above. Using mobile devices makes support independent from corporate IT infrastructures (which are usually hard to access away from the office in PES) and enables users to use the tool when and where they want, for example after talking to employers or after work, e.g. while using public transportation on the way home.
In the prototype users can write personal notes about experiences at work (upper part of Fig. 1) and they can enter reflections multiple times about these notes (bottom part of Fig. 1), including an assessment of how they feel about the experience (see the smiley icons in the bottom half of Fig. 1).
The prototype also includes an easy to use sharing feature to enable collaborative reflection. To enhance personal engagement in collaborative reflection, when sharing content with colleagues the system offers users the opportunity to choose from predefined questions (or create a new ques- Fig. 1. The prototype allows users to write tion) to share together with the content. notes and to reflect on them. All notes and This aims at provoking reflection: For ex- reflections can be shared. ample, as user might choose a question such as “Did you ever encounter a similar situation? What did you do?” when sharing the description of an issue. This may personally impact colleagues, who feel personally invited to engage with the user sharing the content and motivated to help her. This may help to establish communities of practice helping each other and it facilitates collaborative reflection by engaging users in conversations about challenging work.
The tool periodically prompts users individually or collaboratively to revisit past issues and reflections. This can be useful to capture changes in perspectives on experiences over time and the resulting insights leading to this change. For example, if a user from agency A experiences she cannot implement a certain aspect of the new coaching process, she may improve over time, also rating this experience more positive after some time (see Fig. 1). It is also possible just to share one of the newer entries of a reflection with another user to enable collaborative reflection on specific aspects of the evolving situation. Using the tool in this way builds up an individual and collective knowledge base on aspects related to career adaptability.
Users control when they are prompted for reflection: they can let the system (contextually) determine when to prompt them or they can set reminders to reflect. This for example can be used to notify a user while she is using the bus on the way home and wants to reflect on situations she had experienced that day. This supports selfreflexiveness as part of career adaptability.
The prototype provides novel features such as sharing personal questions with reflection content and periodically promoting users for individual and collaborative reflection, which are directed towards engaging with challenging work and to support career adaptability. Future work will also aim to integrate its features into existing tools in order to better integrate reflection for career adaptability into daily tasks. 6
Discussion and Outlook We have presented ongoing work in supporting career adaptability in public administration workplaces by reflection support. Our work is in its early stages, and we have created a concept for such support, situated it in needs of learning in PES organizations as typical examples of public administration and showed its feasibility by implementing a prototype. Next steps will include using the prototype with groups of PES practitioners in different agencies and improving the support it provides. In the ARTEL workshop we would like to discuss the concept and how it may be improved. 7
Acknowledgements The work described in this paper is part of the EmployID project funded by the European commission in FP7 (project number 619619). We would like to thank all colleagues in the project for their cooperation and our fruitful discussions. Boosting Informal Workplace Learning
in Small Enterprises
Miloš Kravčík, Kateryna Neulinger, Ralf Klamma Advanced Community Information Systems (ACIS), Informatik 5,
RWTH Aachen University, Germany {kravcik, neulinger, klamma}@dbis.rwth-aachen.de Abstract Where participation of small enterprises in vocational education and training decreases, it risks obsolescence of their knowledge base compared to competitors. Currently we are participating in two projects that aim to address the issue of how to boost take-up of informal learning at the workplace: Learning Layers and BOOST. Previous projects [e.g., ROLE] show the importance of having personalised learning solutions with high relevance, high effectiveness and low barriers to use. Therefore we aim to provide predefined and customizable Personal Learning Environments that support awareness and reflection of users, especially workers in small enterprises.
Keywords: Informal Workplace Learning, Personal Learning Environments.
1 Introduction
Support of informal learning at the workplace is real issue and we attempt to address
it in two projects: Learning Layers [
We distinguish 3 different user roles that have different characteristics and requirements: Manager (e.g. business manager, business advisor or consultant), Trainer (e.g. training manager, learning facilitator) and Employee. Manager specifies BGs for the company, decides which BGs are urgent and which of them are relevant for which employee. Moreover, this role can also assess employees and monitors their learning progress. Trainer describes LIs for selected BGs and the relevancy of LIs for individual employees, recommends LRs for the LIs, and chooses relevant Learning Repositories, where additional LRs can be found. Employee (Fig. 1) gets an overview of BGs and LIs assigned to her, together with the recommended LRs. According to the descriptions of LIs she can search for additional LRs in the predefined Learning Repositories and add them to her portfolio. She can also access the selected LRs in order to learn. Finally, she can monitor her learning progress.
The functional requirements for competence management include: 1. Specification of relevant BGs (high level competences), their priorities and time scales. 2. Assignment of LIs (concrete competences) to each BG, considering also time scales. 3. Assignment of LRs to LIs. 4. Assignment of relevant BGs and LIs to employees. 5. Setting up target LI (proficiency) levels for relevant BGs for each employee, considering time scales. 6. Assessment of the start and current LI (proficiency) levels for the employee. 7. Monitoring the training progress in the company and also of each employee (considering also time scales). The functional requirements for the learning support are still relatively vague, as they will be more domain dependent: 1. Community support – sharing experience, communication, and collaboration. 2. Domain specific support – learning and assessment. 3. Annotation of learning resources assigned to LIs. 4. Considering preferences of individuals. 3
Conclusion and Future Work In the first year the BOOST consortium identified the main requirements and designed a solution. Afterwards we have developed the first version of the technical prototype, which has been evaluated in interviews with 15 stakeholders. Based on their outcomes the technical prototype will be updated and enhanced with additional features, including privacy requirements and personalization. The current version is suitable for companies with open environments, where employees do not mind seeing each other’s competences and learning progress. But in many companies more privacy is demanded, where employee can see just his or her data. Another important feature is assignment of timescales to business and learning goals as well as their monitoring and notifications. The new version will be tested in companies. Acknowledgments. The presented research work is partially funded by the German National Agency BiBB within the Lifelong Learning Programme Leaonardo da Vinci: “Business PerfOrmance imprOvement through individual employee Skills Training” (project no: DE/13/LLP-LdV/TOI/147655) and by the 7th Framework Programme large-scale integrated project “Learning Layers” (grant no: 318209).
the Usefulness of Improvement Recommendations
Nils Faltin, Simon Schwantzer, Margret Jung IMC information multimedia communication AG, Saarbrücken, Germany {nils.faltin,simon.schwantzer,margret.jung}@im-c.de Abstract. The Activity Recommendation App supports employees in individual and collaborative reflection by capturing discussions and solutions for problems that need to be solved. The app enables employees to record personal experiences with the solutions. Based on these experiences the usefulness of a recommendation can be re-evaluated in order to approve, update, or discard the recommendation. The application was successfully evaluated in coaching employees in learning time management techniques.
Keywords: ARA – Activity Recommendation App· soft skills improvement·
recommendation evaluation· solution· experiences· MIRROR Spaces
Framework· time management coaching
1
Reflection on work experiences can lead to new insights and ideas how to handle
work situations better in the future. But the capturing of experiences during work and
the reflection on this data is only half the way for a successful improvement. The
other half is the creation of a viable reflection outcome and the validation of this
outcome when it is applied in practice (see [
Whilst a lot of applications support users to capture data during work in order to
provide it in a subsequent reflection session, the second half of the reflection cycle is
often left unsupported. To also cover this part, the Activity Recommendation App
(ARA) was created in the MIRROR project [
Overview of the Main Functionalities The recommendation is created in an individual or a collaborative reflection session. Major elements of this session are the identification of the concrete issue and a viable solution for this issue. Texts, files, or data from other MIRROR applications can be attached to be used as evidence to back the comprehensibility of a recommendation. The events during the discussion are listed as a kind of minutes. Measurement criteria can be selected to evaluate the usefulness of the recommendation. Before publishing the recommendation, a target person/group is selected and invited to try the new solution.
A concrete scenario could look like this (cf. [
To evaluate a recommendation, the app allows users to view all experiences with an aggregation of the ratings, efforts and benefits captured. All this can then be used to get an overview how well the solution works in practice to be taken as a basis for the decision if the solution should be kept, updated or discarded.
78 In the continuation of the exemplary scenario, the team discusses the recommendation’s weak points (due to the captured experiences) during the regular team meeting and agrees on adapting it in respect to the selected period in time. It is then reevaluated, re-discussed, and finally marked as solved when team agrees about a wellfunctioning final solution. 3
Evaluation & Outlook A summative evaluation of the ARA took place at our company IMC over a period of six weeks. Ten staff members took part in a time management coaching. The approach combined the usage of a computer activity tracking tool and the ARA with a weekly coaching session. In the weekly coaching sessions the coach and the coachee reviewed the individual progress, adjusted the time management rules if not appropriate anymore, or, when the particular goal has been achieved and the new behaviour has been adopted, decided that no further practice regarding that goal is needed. The Activity Recommendation App served well as a support for learning time management by providing a data basis for the coaching sessions. It was used by coach and coachee to set time management goals and to document and monitor the progress in learning new time management techniques. Both benefited from the better preparation for the coaching sessions available with the notes in ARA. Furthermore, the app helped the coachees to focus their goals. Two things were missed concerning ARA: It lacks an interface optimized for smartphones and currently no reminder function is available which motivates the user to capture experiences. These shortcomings can be addressed in future development.
The coach and several coachees also suggested forming peer groups to train time
management techniques. They could then benefit from sharing experience data to
compare own progress with that of others and learn from each other’s experiences.
IMC has started a free online course for time management that includes usage of the
ARA [
1 Université de Lyon, CNRS
2 INSA-Lyon, LIRIS, UMR5205, F-69621, France
{min.ji; christine.michel; elise.lavoue}@liris.cnrs.fr
3 LUNAM Université, Université du Maine, EA 4023, LIUM, 72085 Le Mans, France
sebastien.george@univ-lemans.fr
1
Our research aims to improve learners’ reflection and self-regulation in Project-Based
Learning (PBL). Actually, we observe that the implementation of PBL in engineering
schools, universities or professional training do not benefit from all its capacities,
because it is often action (according to the Kolb’s learning cycle) which is favored to
the detriment of reflection and personal experience [
We focus our work on the design and the development of a dashboard based on
both reporting and activity traces [
Most existing dashboards are designed for the tutors to monitor the learners but
they are rarely designed for the learners to support awareness during their activities.
Furthermore, the indicators are mostly predefined and the users can rarely build their
own indicators [
A reporting tool and a dynamic dashboard We developed a Dynamic Dashboard Based on Activity and Reporting Traces (DDART). We chose to implement this dashboard as a plug-in of the Moodle platform. In our context, project members use the Moodle tools (wiki, forum, chat...) to carry out the project. They are also asked to use a reporting tool to describe and keep traces of the project events.
The reports are composed of semi-structured sentences so that this text information
can be collected and analyzed automatically [
We designed a specific interface to help learners to build their own indicators (see Fig. 1). This interface is composed of three main parts: (1) the “parameters” part (see Fig. 1.a), on the left side, contains the list of all the parameters which are available for creating an indicator, (2) the “calculation” part (see Fig. 1.b and c), in the center, allows learners to place the parameters and view the indicator results and (3) the “visualization modes” part (see Fig. 1.d), on the right side. This user-friendly interface allows learners to create the indicators by dragging and dropping the parameters and the visualization mode. The calculation function is WYSIWYG: the results can be calculated in real-time so that learners can easily adjust the parameters. At last, the presentation of indicators on a dashboard provides awareness to the learners about the way they carry out the project and also about the building of knowledge and skills.
Fig. 1. The interface to assist learners to create personalized indicators
The semi-structured sentences of the reporting tool (reporting traces) and the traces
of use of Moodle (activity traces) are respectively recorded in an XML database
(BaseX) and in a relational database (MySQL). These two kinds of traces are
described according to the same five common entities: Learner, Tool, Activity, Time
and Place. They are merged according to a common time basis and are stored into a
transformed traces base. The transformed traces are used to produce indicators stored
in a dedicated database [
Conclusion In this paper, we have presented the basis of the DDART system. This system can help learners to collect, analyze and visualize their reporting and activity traces in the form of meaningful indicators. By allowing learners to create their own indicators, we aim at making them learn how to regulate their learning activities. The traces collected in the reporting tool allow the construction of advanced indicators that can help learners to build metacognitive skills. For example, the indicators can support the analysis of behavior by comparing the learners’ feeling about their activities (subjective) with the realization mode of the activities (objectively recorded by the system). References Sven Charleer, Jose Luis Santos, Joris Klerkx, and Erik Duval
Dept. of Computer Science, KU Leuven
Leuven, Belgium {Sven.Charleer,JoseLuis.Santos,Joris.Klerkx,Erik.Duval}@cs.kuleuven.be Abstract. Exploring and managing the abundance of data that Learning Analytics generate is a challenge for both teachers and students. This paper introduces a Learning Dashboard that provides an overview, context and content of learner traces to help students with awareness of feedback and progress, and assist teachers with monitoring student effort and outcomes to intervene where needed.
Keywords: learning analytics, learning dashboards, awareness,
information visualization, effort, intervention, inquiry-based learning
1
The purpose of Learning Analytics is understanding and optimizing learning and
the environments in which it occurs [
Learning Dashboards can rely on many different ways of visualizing raw
analytics data e.g. bar, star and bubble charts, interactive histograms, parallel
coordinates etc [
This abundance of data can be abstracted to the essentials [
LARAe: Design & Implementation LARAe visualizes traces gathered from 38 engineering students, teachers and external participants in an open User Interfaces course. Students worked in groups of 3 and reported weekly through blog posts, comments and Twitter. The course generated 419 blog posts, 1580 comments and 538 tweets.
Fig. 1. LARAe: A. Overview, B. Activities, C. Thread view
Every activity is represented by a circle (Figure 1.B) which provides direct access to the related content (e.g. blog post, comment, tweet, retweet). Activities are sorted chronologically, from top left to bottom right. Gradient color values (see Figure 1.A) help recognize the age of an activity. A table (Figure 1.B) structures the activities by student group and type. Every column represents an activity type, every row a student group. The user can sort the data by any activity type. Both activity age and amount help facilitate awareness of (in)active groups. As teaching staff feedback was deemed important by both student and teacher, a second table visualizes activities of teacher activity in a similar way.
Context plays an important role in understanding the activities e.g. a
comment without its surrounding discussion is difficult to assess. We propose a
“focus+context” [
Highlighting related activities helps the user to instantly become aware of the distribution of an activity thread across the class e.g. selecting a blog post will highlight what groups provided most contributions. Simultaneously, the thread view shows the content of each related activity, helping assess the quality of the quantitative data. Visualizing discussion thread size can help students discover interesting threads. Teachers might understand low thread size as an indication for need of intervention. The attribute thread size is indicated by a number in each circle (Figure 1.B).
LARAe is a web application developed using HTML5, JavaScript and D3.js1 running on a Node.js2 web service and MongoDB3 database. It supports both the proprietary API and Tin Can API4. It can easily be extended to support other APIs. The dashboard is designed to run on large displays, desktop computers and tablets. It is available at http://ariadne.cs.kuleuven.be/LARAe/.
The dashboard has also been deployed in an inquiry-based learning setting,
visualizing the learner traces gathered from the weSPOT Inquiry system5 [
References 1 http://d3js.org 2 http://nodejs.org/ 3 http://www.mongodb.org 4 http://tincanapi.com/ 5 http://portal.ou.nl/documents/7822028/f475d712-5467-40ea-968c-5aa00d951400