For the LAK (Learning Analytics and Knowledge) community, it is highly important to pay attention to the development and deployment of learning analytics solutions for blended learning scenarios where students work at diverse digital and physical learning spaces and interact in different modalities. This workshop has been a first attempt in gathering the sub-community of LAK researchers, learning scientists and researchers from other communities, interested in ubiquitous, mobile and/or face-to-face learning analytics. It was clear for all the attendees that a key concern that has not been deeply explored yet is associated with the mechanisms to integrate and coordinate learning analytics to provide continued support to learning across digital and physical spaces. The two main goals of the workshop were to share perspectives and identify a set of guidelines that could be offered to teachers, researchers or designers to create and connect Learning Analytics solutions according to the pedagogical needs and contextual constraints to provide support across digital and physical learning spaces.
Students’ learning commonly occurs in spaces and at moments that go beyond formal education, and this learning is not constrained to a single physical or digital environment [2, 5]. Even in online learning situations, the actual learning does not necessarily occur online. However, most of the learning analytics efforts to date have been focused on understanding only the traces left by students while interacting with specific desktop or web based learning interfaces. Research in the learning sciences (LS), computer-supported collaborative learning (CSCL) and technology-enhanced learning (TEL) has shown the pedagogical benefits of generating learning opportunities for students to experience different types of challenges, "real world" problems, and physical and social interactions with educators or other learners [1, 3]. In short, the design needed for facilitating this variety of learning experiences needs to consider rich, multi-modal and often collaborative tasks that go beyond the intelligent tutoring system, the learning managements system (LMS) or a single educational tool.
Increasing access to emerging communication technologies is increasingly making it possible for students to use of a wide range of devices and educational (and non-educational) software applications to drive their own learning or for their teachers to design novel learning tasks. At the same time, educational providers, including schools and universities, deploy a variety of educational technologies and pedagogical resources in both online and face-to-face settings [4]. These technologies allow learners to get remote access to educational resources from different physical spaces (ubiquitous learning support) or to enrich their learning experiences in the classroom, or in campus, in ways that were not previously possible (face-to-face learning support).
This increasing interest in providing support for students’ learning across physical and digital spaces is of high relevance to the Learning Analytics and Knowledge (LAK) community. The challenge is to find the best approaches that can be applied to automatically capture traces of students’ activity, and understand how learning analytics techniques can be used in this context to exploit these (often) heterogeneous, multi-modal and messy data.
A full-day workshop gathered the sub-community of LAK researchers interested in ubiquitous, mobile and/or face-to-face learning analytics in conjunction with learning scientists and researchers from other communities who have explored the perspective of learning across digital and physical spaces. There were a total of 35 attendees of varied backgrounds who brought a wide range of perspectives to a common problem which was stated during the workshop as follows:
Providing continued support in the classroom, for mobile experiences and using web-based systems has been explored to different extents and each poses its own challenges [4, 6].
Most of the day of the workshop was dedicated to hands-on activities. The first activity consisted in a panel session that served to share approaches and exchange information about how educational data science can be brought to bear to provide continued support across varied spatiotemporal learning situations. The panel session was an outstanding kick start of the workshop because the panelists presented their very different perspectives to the problematic and tried to identify what is the gap in research and development (R&D) that needs to be addressed. These perspectives included the LS/CSCL (Cynthia D’Angelo), TEL (Davinia Hernandez-Leo), Ubiquitous computing (Naif Aljohani) and Learning Analytics (Abelardo Pardo).
The rest of the workshop core activities consisted in the formation of roundtables to: 1) Identify what are the most common learning scenarios and their specific challenges that require a Cross-LAK solution. For this activity, the cases described in the submitted papers were crucial to illustrate the possible scenarios where a more holistic view of learning analytics towards learning is needed. Attendees were requested to propose new use cases, describe their own experiences and document the use cases they wanted to ground upon for the formulation of the guidelines. 2) Formulate a set of guidelines could be given to any practitioner, designer and/or researcher, to implement Cross-LAK solutions. The formulation of any type of guideline is always a very challenging task. However, this activity provided guidance to the attendees and helped them discuss and think out of any particular use case. An overview of the resulting guidelines is presented in the next section.
Outcomes1 The main outcome of the workshop was to contribute with expert guidelines/principles that can help guide future research and development to create learning analytics and monitoring tools that can provide support for each of the four themes of the workshop: Learning analytics across digital spaces, Learning analytics bridging physical (and digital) spaces, Mobile and ubiquitous learning analytics and the Data integration of heterogeneous learning data sources. Attendees were divided in four groups and were scaffolded step by step before engaging in the generation of guidelines. For example, a set of exemplar briefs were generated first with the aim of identifying the key challenges posed by particular learning situations. This allowed the workshop participants to start brainstorming on the key aspects that are important and how learning analytics solutions could be applied. For example, one of the groups generated the quadrant showed in Figure 1 to facilitate the description of the different exemplars that they were discussing, making a clear emphasis on the social and physical dimensions of the learning situations. An overview of this process is depicted in the workshop schedule: https://sites.google.com/site/crosslak2016/schedule. 1 The content of this section is the result of the collaborative work by all the attendees of the workshop. A complete list of attendees can be found here: https://sites.google.com/site/crosslak2016/atendees The Guidelines
- Pick the learning tools and the type of data that the learning tasks require - Decide what will be the scalable feedback strategy beforehand and how the learning analytics tool can support this - Communicate to learners the scope and the ways that any data analytics will be reported to them
- Be as clear as possible about the learning objectives you want to satisfy - Consider the learning tasks that benefit from the physical space where they happen for example: classroom tasks, face-to-face discussions, laboratory sessions or open spaces
- Consider the importance that the students hold ownership of their data - Take into account data privacy and ethical issues
- Promote interoperability: use of metadata, standardised data formats, decoupled architectures - Aim for a White box: the data should be available (ability to be accessed by other services), controllable (by the users) and transparent - Aspects of usability and user experience (UX) can be crucial given the multiplicity of tools used by learners and teachers
- Define the construct you want to investigate (like: collaborative problem solving) and what signifies (what are the observable measures) in your system - There must be an alignment between the tools and algorithms used and the learning goals - A comprehensive learning and activity recording is expected in heterogeneous scenarios
- Describe the strengths and limitations of your data sources with respect to the objectives - Consider if your data is good enough for the data analysis you want to perform - Be aware and explicit about data reliability - Be aware and explicit about the reliability of the proxies we use for gathering data - Algorithm accountability
- Take measures to cater for possible drop data (due to sensors' failure, etc.) - Consider how you are going to manage your data -storage, etc.
- Adjust your visualisation to the needs of the end users - Adjust your visualisation to the needs of the task - Visual integrity: Make sure the visualisation does not yield for false interpretation
Whilst the goals of this workshop were accomplished, there is a growing interest by the community on this area. The overarching idea of taking a more holistic view of learning and learning analytics considering what it occurs in the physical world and across multiple tools is appealing. As suggested by the attendees of the workshop a mailing list and a discussion group was created online (https://groups.google.com/forum/#!forum/crosslak). The objective of this online group is to keep track of any follow up of this workshop so the LAK community can benefit and further contribute to the design space. All the outcomes of the workshop, including the outcomes listed above and the papers submitted, are available through the workshop’s own website.
[1] Delgado Kloos, C., Hernández-Leo, D., and Asensio-Pérez, J.I. 2012. Technology for Learning across Physical and
Virtual Spaces. Journal of Universal Computer Science, 18, 15, 2093-2096. [2] Looi, C. K., Wong, L. H., & Milrad, M. 2015. Guest Editorial: Special Issue on Seamless, Ubiquitous, and Contextual
Learning. IEEE TLT, 1, 2-4. DOI= 10.1109/TLT.2014.2387455 [3] Pérez-Sanagustín, M., Ramírez-González, G., Hernández-Leo, D., Muñoz-Organiero, M., Santos, P., Blat, J., Delgado-Kloos, C. 2012. Discovering the campus together: a mobile and computer-based learning experience.
Journal of Network and Computer Applications, 35, 1. 176-188. DOI=10.1016/j.jnca.2011.02.011 [4] Rogers, Y. 2008. Using external visualizations to extend and integrate learning in mobile and classroom settings. In Visualization: Theory and practice in science education. J.K. Gilbert, M. Reinder and M. Nakhleh Eds. Springer: Netherlands, 89-102. DOI= 10.1007/978-1-4020-5267-5_5 [5] Sharples, M. and Roschelle, J. 2010. Guest editorial: Special section on mobile and ubiquitous technologies for learning. IEEE TLT, 1, 4-6. DOI=10.1109/TCAD.2015.2410671 [6] Wang, M., Shen, R., Novak, D., & Pan., X. 2009. The impact of mobile learning on students’ learning behaviours and performance: Report from a large blended classroom. BJET, 40, 4, 673-695. DOI=10.1111/j.1467-8535.2008.00846.x