Modern cloud-based big data engineering approaches like machine learning and blockchain enable the collection of learner data from numerous sources of di erent modalities (like video feeds, sensor data etc.), allowing multimodal learning analytics (MMLA) and reection on the learning process. In particular, complex psycho-motor skills like dancing or operating a complex machine are pro ting from MMLA. However, instructors, learners, and other institutional stakeholders may have issues with the traceability and the transparency of machine learning processes applied on learning data on the one side, and with privacy, data protection and security on the other side. We propose an approach for the acquisition, storage, processing and presentation of multimodal learning analytics data using machine learning and blockchain as services to reach explainable arti cial intelligence (AI) and certi ed traceability of learning data processing. Moreover, we facilitate end-user involvement into to whole development cycle by extending established open-source software DevOps processes by participative design and community-oriented monitoring of MMLA processes. The MILKIPSY cloud (MPC) architecture is extending existing MMLA approaches and Kubernetes based automation of learning analytics infrastructure deployment from a number of research projects. The MPC will facilitate further research and development in this eld.
Learning complex psychomotor skills involves coordinating physical movements according to a prede ned reference model. The increasing availability of big data solutions presents opportunities in education to converge cloud infrastructures and learning infrastructures. This allows professional communities of practice of instructors, learners and other institutional stakeholders to create better collaborative environments for multimodal learning analytics (MMLA). These Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). cloud-based environments can be used to enhance collaborative knowledge sharing within and between the communities by sticking to established standards, by using established open-source software development processes and by building knowledge repositories. Arti cial Intelligence (AI)-based systems are used to enable e cient analysis of the vast amounts of data that can be collected while performing learning activities. However, the communities will not accept AI-based solutions as the data are not secured and the processing is transparent for all community members by design. To realize this, all community members should be involved in the design of machine learning [8] and other related processes. In the end, it must be explainable to the end users of a system why the AI arrived at the presented results.
This conceptual paper presents the MILKI-PSY cloud architecture, an extension of existing MMLA approaches and the results of a number of European and national research projects for infrastructure building in complex learning domains.
After the related work section, we present our MILKI-PSY cloud. The papers then concludes and gives an outlook on further research. 2
Machine learning as a service (MLaaS) [10] is an umbrella term for various cloudbased platforms that cover most infrastructure issues in training AIs, such as data preprocessing, model training, and model evaluation. This approach is very useful and e ective not only for data scientists, data engineers, and other machine learning professionals, but also for students and researchers who can use it to train machine learning models while bene ting from the scalability of the cloud provider. Blockchain as a Service (BaaS) [11] enables enterprises to use cloud-based solutions to build, host, and use their own blockchain apps, smart contracts, and functions on blockchain infrastructure developed by a vendor. BaaS provides access to a blockchain network of a desired con guration without the need to develop, host and deploy an on-premises blockchain and build in-house expertise on the subject. The distributed ledger of a blockchain can be used to manage the process of issuing, storing, and releasing students' academic certi cates, to store and share competencies and learning outcomes that students have achieved, to assess learning progress [1]. In addition, blockchain technology can be used to enable easier and more secure transfer of credits between learning centers. Explainable AI builds a common communication platform between humans and AI that helps perform learning analytics and improves the usability of AI-powered mentoring processes. To this end, machine-learned features should correlate with human-derived thought constructs and mental models to facilitate understanding of the neural network learning process [5]. In this context, the American Institute of Standards and Technology (NIST1) presented four principles of comprehensible AI [9]: 1 https://www:nist:gov/ 1. Explanation AI should provide evidence, support, or rationale for each output. This principle does not require that evidence be correct or intelligible; it merely states that a system is capable of providing an explanation. 2. Meaningful Systems provide explanations that are understandable to the individual user. A system satis es this principle if the recipient understands the system's explanations and/or they are useful in accomplishing a task. 3. Explanation accuracy The explanation correctly re ects the system's process for producing the output. Taken together, the rst two principles only require that a system produce explanations that are understandable to the target audience, without requiring the explanation to correctly re ect the system's process for producing its output. "Explanation accuracy" requires that a system's explanations should be accurate. 4. Knowledge limits The system operates only under the conditions for which it was designed or when the system achieves su cient con dence in its output. The previous principles implicitly assume that a system operates within its knowledge limits. This principle states that systems identify cases for which they were not designed or approved, or that their responses are not reliable. By identifying and declaring knowledge boundaries, this practice safeguards responses so that judgment is not made when it may be inappropriate.
Due to the lack of end-user engagement concept in DevOps, Koren et al. introduced the extended DevOpsUse approach [3] in our research group, which aims to unify agile practices of developers, operators, and end-users. The inner- CO-DESIGN most circle of the schema (see Fig. 1) re- IDNEEAEDSS& DEV buseaecsrtissc.otThnoetrisrbteaunteidcotanrstdhaeDsecimvonOptporisrbtlauinfteicocenycsolfetoeanSsdoa-- AWARENESS MONIFTEOREeRDlDeBaesvAeeCl&oKpM&oDnTieEtosVrtELOTPEST TEBSETTIANG cietal Software Engineering, an additional aUcStEivirtiiensgi nisthadeddeidetroenrtepphreasseenstofenthde-uDseer- PRACTICE DEOPPLOSY CONTEXT vOps cycle. To improve usability and understanding of complex information systems, such as AI-based cloud solutions, USE the involvement of end-users in the design and creation process is crucial. In particu- Fig. 1: DevOpsUse Cycle [3] lar, this means that users are not only involved in the elicitation of requirements, but are also instrumental for beta testing, providing deployment context, and using the application for their practice. This in turn provides awareness to issues and is a valuable source for ideas and feedback to improve the usability of the designed technology.
Learning analytics [2] is the measurement, collection, analysis, and reporting of data about learners and their contexts for the purpose of understanding and optimizing learning and the environments in which it occurs. Most conventional learning analytics approaches examine learning processes and contexts from a single data source (e.g., the logs of an LMS), which provides only a partial view of learning.
Multimodale Learning Analytics (MMLA) involves complex technical issues
in collecting, merging, and analyzing di erent types of learning data data from
heterogeneous data sources. Di Mitri et al. [4] proposed a Multimodal Learning
Analytics Pipeline (MMLAP), which provides a generic approach to collecting
and analyzing multimodal data to support learning activities in physical and
digital spaces. The pipeline is structured in ve steps, namely: (
MILKI PSY Cloud: An infrastructure for distributed multimodal learning data analysis with a focus on informational self-determination A cloud infrastructure for distributed multimodal learning data analysis should be able to collect data from heterogeneous input streams from a variety of sources, like software solutions and hardware sensors. Additionally to learner data, a data annotation layer is required to apply expert knowledge, which serves as a reference model to compare the learner data to. The learning data then needs to be collected, stored, analyzed and processed to provide insights and understanding into the multimodal data stream. The design of AI elements in the MILKI PSY Cloud (MPC) should not only involve the end-users (DevOpsUse), but also follow the principles of explainable AI. This understanding can then used to provide direct feedback to the learners. Additionally, blockchain approaches can be used to provide secure certi cation of learner progress. Blockchain approaches enable data self-sovereignty by allowing individuals to decide who can access and use their data and personal information. We use OpenID Connect2 for authentication and authorization, which enables modern and secure access to all services. In the educational context, this allows learners to manage their credentials without relying on the educational institution as a trusted intermediary. This becomes important in particular, when learners are changing their institutions.
Figure 2 gives an overview of the proposed infrastructure. The central component here is the MPC [12], marked with a blue frame in the gure. 2 http://results:learning-layers:eu/infrastructure/oidc/ Mul modal sensor data
Reference model 1. Data Collec on 3. Data Annota on 5. Data Exploita on Mul modal sensor kits
Camera feeds
Recorded Movements
Visual Annota ons
Intelligent Tutors
Social Bots
Direct Feedback
Dashboards
Cer ficates Apache Ka ka ...
Distributed Machine Learning
Learning Record Store
LRS 1 ...
LRS n
Blockchain ... 2. Data Storage
Human erronenous ac vity recogni on
ARLEM, xAPI
4. Data Processing Learner
Expert Fig. 2: Architecture of the MILKI PSY Cloud Learner feedback Learner
Legend
Encrypted communica on
Data flow Apache Ka a Ethereum blockchain Distributed Storage AI nodes Docker Container Kubernetes Cluster
The data pipeline, which is based on the Multimodal Learning Analytics
Pipeline by Di Mitri et al. [4], starts with the aggregation and processing of
multimodal sensor data from learners. Thus, the rst step is to collect
multimodal sensor data from learners (
Within the cloud, we rely on a Kubernetes-based solution. This enables a modern and scalable infrastructure for decentralized storage of data. Support for AI-based tools developed by appropriate partners during the course of the project, can thus also be hosted directly by us in the Kubernetes cloud. 3 https://kafka:apache:org/ 4 IEEE 1589-2020 https://standards:ieee:org/standard/1589-2020:html 5 https://kubernetes:io/
The increasing availability of cloud-based big data solutions facilitates the integration of learning infrastructures over institutional and national boundaries. The collaborative exchange of knowledge and the interoperability of the learning approaches improve the spread of multimodal learning analytics, but also raises issues of trust into the infrastructures and the complex services that are hosted on these infrastructures. Learners, instructors and institutional stakeholders need transparency and traceability of learning records and the further processing of learning analytics data. In particular, there is an emphasis on self-sovereignty, which is especially important in the context of storing and processing privacysensitive learner data collected in the course of learning activities. The increased use of AI algorithms means that machine learning and AI-based services are becoming part of the infrastructure (MLaaS). Explainable AI is used here as a communication platform between humans and AI to improve the usability of AIassisted mentoring processes and can be used to provide personalized and direct learning feedback. Blockchain approaches support data protection law compliant and traceable management of learner data, even when learner changes their learning institutions. Thus, to verify the use of complex AI systems, all stakeholders are involved in the design of machine learning and blockchains.
In the project context, the Multimodal Learning Analytics Pipeline will be used to support the development of psychomotor skills with arti cial intelligence. In particular, this will be investigated in two application domains: sports and complex processes in human-robot interaction. The goal of the MPC is to acquire, store, process, and display real-time multimodal data to promote digital learning of psychomotor activities such as a human-robot interaction or while playing sports. In cooperation with project partners, the organizational basis for the creation of a distributed cloud development and learning platform (MPC with special involvement of learners and end users (DevOpsUse) will be created using an agile development process. The use of OpenID creates an open learning environment that enables secure and easy access to the provided services. The multimodal learning data is collected by di erent sensors and camera and learning systems via data brokers and processed in the MPC. To create a reference model, motion pro les are recorded by experts and analyzed in the system. These are then compared together with the raw sensor data by an AI-based Human Erroneous Activity Recognition mechanism to detect di erences between the learner's activities and the targeted learning goal and provide contextual feedback. Cloud-, Fog- and Edge-based machine learning models will be compared with client-side solutions in the future.