Living Labs represent a promising approach to bridge the gap between evidence-based educational research and sustained innovation. This position paper presents our initial work related to educational Living Labs. It describes a model of the research and innovation processes that we aim to support. It also presents the preliminary results of a pilot study in which a Living Lab supported a researcher and two teachers to introduce Learning Analytics in their classroom.
During the last few years, we have witnessed the rise of Learning Analytics (LA).
This emerging field promotes the collection, managing, and processing of
educational data to understand and improve learning processes [
One of the barriers that hinder LA adoption by practitioners is the need for
a cultural change towards more evidence-based decision making [
Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
We propose a Living Lab approach in order to overcome these two issues.
Living Labs are user-centred open-innovation ecosystems that integrate research
and innovation processes [
The rest of this position paper further describes our proposal. Section 2 presents the state of the art regarding Living Labs for education. Section 3 describes the research and innovation processes that we intend to support with Living Labs. Then, section 4 briefly describes a preliminary pilot study that is being carried out at Tallinn University. Finally, section 5 summarizes the implications and future work in this research line.
There is still a gap between educational research and innovations in schools. Some
of the research in educational and cognitive science does not find its way into
school practice. At the same time, innovations that are introduced in classrooms
are often not evidence-based, meaning that their effectiveness is not empirically
tested. Moreover, out of the many successful pilots that research on how ICT is
best employed in learning, only a few are able to be sustained [
This gap between research and sustained innovation is not exclusive to
education. In order to overcome it in different domains, Living Labs were proposed
as “open innovation environments in real-life settings in which user-driven
innovation is the co-creation process for new services, products, and societal
infrastructures” [
However, as we know from many successful (and not-so-successful) cases of
adoption of LA [
In order to understand the support that a Living Lab should provide, we firstly
gathered data from local teachers that work at Tallinn University. Teachers that
had experience in LA were firstly interviewed and, later on, we organized a focus
group with a broader group of teachers. After that, we reviewed the literature
related to Design Based Research [17] and informal learning at the workplace
[
The model depicted in Figure 1 defines several phases and arranges them
into two iterative cycles. The first cycle includes phases related to an
educational research process, as defined by the Design Based Research [17]: Problem
Understanding and Shared Vision; Intervention and Evaluation Design;
Enactment; Evaluation and Reflection. The second cycle includes phases
related to scaling up informal learning at workplace, as defined by [
Our Living Lab also approach entails an open set of tools to support the model depicted in Figure 1. Some of these tools will be general-purpose, such as collaborative text editors or forums; others will be purpose-specific, such as collaborative meaning-making tools or data repositories. Additionally, a data infrastructure is needed in order to coordinate all these tools and to share data and learning artifacts among them. This infrastructure should also collect data related to the use of the different applications, which can then be analyzed in order to assess the research and innovation process that occur in the Living Lab.
The support of evidence-based research also adds further requirements. First, some guidance will be needed in the Intervention and evaluation design for researchers and teacher to introduce LA techniques in the classroom. These guidelines should support them to take decisions related to LA, such as how to collect data, how to integrate it or ethical considerations. Data collection and processing applications will be required to enhance the Enactment and Evaluation phases. The data collected during the Enactment phase should also be disseminated to a larger group during the Dissemination phase for them to be able to validate it later on.
We are currently carrying out a preliminary study pilot where a Living Lab supports a researcher and two teachers to introduce and adopt LA techniques into a course. The course is “Research Methods” and is taught at master level in Tallinn University. The researcher has broad experience in LA and in qualitative and quantitative research methods. Regarding the teachers, they are experienced teachers and also familiar with LA.
For this pilot study, we are using GRAASP1 as a technical platform to support the lifecycle depicted in Figure 1. GRAASP is a knowledge management system used both for learning and for supporting communities of practice [20]. It integrates several well-known applications and provides data management and processing functionalities that have already been used for LA [22, 21]. Additionally, Google Drive2 is used as a way to create and share documents between the researcher, the teachers, and the students, and as a data gathering tool (questionnaires sent to the students during the course).
As the researcher and the teachers knew each other beforehand, the engagement phase and the creation of the working group were easy processes. In fact, they discussed the problem face to face to reach a shared vision. Hence, our support with GRAASP focussed on the Intervention and Evaluation Design, Enactment, Evaluation and Reflection phases. As the Intervention and Evaluation Design phase is the only one that has been completed as of this writing, the rest of this section will only focus it.
We embedded an evaluation framework into GRAASP to support the Interv
ention and Evaluation Design phase. The main idea was to propose a
questionnaire that guided the researcher and the teachers during the following
phases. The questionnaire was based on the Hopscotch model [
We did not impose any restriction on the users (the researcher and the teachers) regarding how to use GRAASP or the questionnaire. They decided that the researcher would first go through the questions and the teachers would later on enrich the answers or provide another point of view if needed. The three users interacted with the questionnaire without any remarkable problem. We recorded the interactions between the users and the questionnaire using the think aloud protocol. Additionally, we took notes about any relevant incident when the users were using GRAASP. When the Intervention and Evaluation Design phase finished, we interviewed the users about their experience.
Up to now, we obtained some preliminary findings from the data we collected. Some of the comments invited us to further reflect on our approach to the Living Lab. We imposed how practitioners should behave and which technology they should use. Regarding the technology, future versions of the Living Lab should integrate applications that are already in use by the practitioners, thus enhancing the Living Lab adoption. Regarding our model, its current version considers that the practitioners design the intervention and the evaluation before collecting any data. However, it is sometimes the case that some data is unsystematically collected and analyze later on. There should, hence, be room for a posteriori analyses of previous experiences that can trigger complete loops in the future. 1 http://graasp.eu 2 http://drive.google.com/
Some other comments were related to the questionnaire, which can be grouped into three main aspects. First, additional guidance is needed to answer the questions, especially those where the users are invited to reflect on the data gathering and processing techniques. Sometimes it is not clear for them why some questions are relevant, what are their answers going to be used for or even what are possible answers (e.g., what are possible data gathering techniques?). Second, the questionnaire should be simplified and some questions should be merged (e.g., questions related to the goals or the ones related to similar innovation processes). Third, the users would like to count on evaluation and interventions designs, so that they can find examples in other related research works or innovation processes that aimed for similar goals.
This paper describes our vision towards a Living Labs to fill in the gap between educational research and innovation sustainability. We propose a model to support the collaboration of researchers and teachers for evidence-based educational research and innovation. We also presented an ongoing preliminary pilot study and several preliminary lessons learned.
We envision Living Labs as complex learning ecosystems. From the theoretical point of view, we understand Living Labs as a mix of two worlds: they combine research processes, needed to propose and assess educational innovations; and workplace learning processes, needed for teachers to adopt these innovations. From the technical point of view, Living Labs are open ecosystems where different applications and technologies can coexist. A software infrastructure will also be required to orchestrate the different technologies involved and to collect data from participants’ behavior. From a practical point of view, we can expect cultural constraints from the teachers’ community when adopting innovations.
In addition to the pilot presented in section 4, six additional ones will be carried out during 2017. These pilots will cover the whole lifecycle presented in Figure 1. For these new pilots, we will collaborate with Estonian primary school teachers who are not so familiar with LA to introduce new methodologies in their classrooms. Then, we aim to explore how Living Labs can be exploited to support innovation processes that scale up from the classroom to educational institutions and even to national policy level.
This research has been partially funded by the European Union in the context of CEITER and the Next-Lab (Horizon 2020 Research and Innovation Programme, grant agreements no. 669074 and 731685). 17. P. Reimann. Design-based research. In L. Markauskaite, P. Freebody, and J. Irwin, editors, Methodological choice and design, 9, pages 37–50. Springer Netherlands, 2011. 18. M. J. Rodr´ıguez-Triana, A. Mart´ınez-Mon´es, and S. Villagr´a-Sobrino. Learning Analytics in Small-scale Teacher-led Innovations: Ethical and Data Privacy Issues.
Journal of Learning Analytics, 3(1):43–65, 2016. 19. Y.-S. Tsai and D. Gasevic. Learning analytics in higher education—challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pages 233–242. ACM, 2017. 20. A. Vozniuk, A. Holzer, S. Govaerts, J. Mazuze, and D. Gillet. Graspeo: a social media platform for knowledge management in NGOs. In Proceedings of the 7th International Conference on Information and Communication Technologies and Development, pages 63–66. ACM, 2015. 21. A. Vozniuk, M. J. Rodr´ıguez-Triana, A. Holzer, and D. Gillet. Combining content analytics and activity tracking to identify user interests and enable knowledge discovery. In Proceedings of 6th International Workshop on Personalization Approaches in Learning Environments (PALE), pages 1–7, 2016. 22. A. Vozniuk, M. J. Rodr´ıguez-Triana, A. Holzer, S. Govaerts, D. Sandoz, and D. Gillet. Contextual learning analytics apps to create awareness in blended inquiry learning. In International Conference on Information Technology Based Higher Education and Training (ITHET), pages 1–5. IEEE, 2015.