Effectively implementing LA at an institutional level is far from trivial, as such a solution needs to be scalable. In this project I aim to build a scalable learning analytics architecture. The main research focus will be on semantic interoperability, for integration of multiple educational data sources, but the architecture will also have components for data analysis and reporting insights to stakeholders. As part of this project we have conducted a systematic review, to understand state-of-the art research and practice regarding multiple data source usage and combination in learning analytics. The initial work with the architecture has also resulted in a first conceptual model. For developing the architecture, semantic web technologies will be employed, to handle aspects such as shared data meaning. Once developed, we will conduct studies that use the architecture to address real-world challenges in higher education.
Learning Analytics (LA) includes collecting, computationally analyzing and reporting
data to stakeholders; to gain insights, and enable decision making and interventions
related to questions about learning and learning environments [
Data integration is related to interoperability. Interoperability involves technical,
semantic, legal and organizational levels. The semantic level is about ensuring that data
format and meaning is preserved and understood. The technical level includes services
for data exchange and data integration. The legal level is about aspects such as enabling
collaboration despite of different organizational policies. Organizational
interoperability includes aligning processes for common organizational goals and addressing user
expectations and requirements [
The number of data sources used in combination for data analysis tend to be limited.
Existing LA projects addressing data integration needs tend to put the emphasis on the
technical level of interoperability [
The main goal of this project is to build a scalable LA architecture for higher education, with research emphasis on semantic interoperability. However, the architecture we will develop also has components for data analysis and reporting of insights to stakeholders. A secondary goal, providing insights into the real-world application of the architecture, will be to address one or more challenges in a higher education institution, through architecture usage. Initially, the challenges will focus on student success.
To achieve the goals, the following overarching research question has been formulated:
How can a scalable learning analytics architecture be built and applied to address challenges in higher education? 3
(Abbreviated) State of the Field
Different studies have used or combined multiple data sources for LA. Data from a
Learning Management System and Student Information System have been combined
to detect students who struggle academically [
Merging data available in different formats is more challenging than combining
similar data. In addition to common operations such as data access, data cleaning, and data
filtering, the data to be combined also needs to be transformed into a common format.
For this purpose the organization JISC has developed an architecture that includes
plugins to transform educational data to xAPI statements, meaning they can be
combined in a common store (learning record warehouse) [
While the approach taken by JISC will enable technical interoperability, there is less
of an emphasis on enabling shared meaning between data. Using semantic technologies,
such as the RDF data model and ontologies, it is possible to add descriptions and
meaning to data coming from various sources, and to combine, support and reuse different
specifications/data models. Ontologies also enable inference (given some stated fact,
we can state new and related facts) [
As an overarching research methodology for this research project, the design science
research framework will be used [
To identify important foundations for the work on the architecture, and to make it clear
where the proposed research fits into the body of knowledge, we have recently
conducted a systematic review. The systematic review follows specific guidelines [
Informed by the results of the systematic literature review, we are developing a LA architecture. Of special importance is that the solution should scale, emphasizing semantic interoperability, to remove barriers for data exchange. 4.3
Once the system architecture has been developed, and we have combined educational data sets, we will use the architecture to address one or more challenges in higher education, through computational data analysis and reporting. Challenge selection will be informed by stakeholder needs for specific insights about learners and their environments. Initially, the focus will be on LA related to student success (e.g. through providing relevant dashboards to the students). 5 5.1
The following search string was formulated to search relevant databases (ACM, IEEE Xplore, SpringerLink, Science Direct, Wiley and AISEL), conference proceedings (Learning Analytics and Knowledge, Learning at Scale and International conference on Educational Data Mining) and journals (Journal of Learning Analytics and Journal of Educational Data Mining) for documents: ("multiple data sources" OR multimodal OR "multi-modal" OR "multiple data sets" OR "multiple datasets") AND ("learning analytics" OR "educational data mining") AND "higher education"
A number of inclusion and exclusion criteria were formulated, to ensure that reviewed papers would fit the research questions and would provide foundations for our development of a LA architecture. One such criteria was that the reviewed papers had to use or combine more than one data source.
The search originally returned 71 papers, but following specified inclusion and
exclusion criteria, we were left with 14 papers for inclusion in the review. This process
was documented using a PRISMA flow diagram [
In our results, we have observed that five of the reviewed papers analyze data in different formats that originate from different sources without a common format, thus these data are not integrated, but analyzed separately. Nine of the studies merge and analyze data from different data sources that are already in the same format. A more extensive list is given in Table 1. With regards to number of data sources used/combined, we found that nine of the fourteen papers use or combine only two data sources.
Having conducted the review, we are now finishing up the resulting paper and plan to send it for review before the end of the 2018. 5.2
The initial work with the architecture has resulted in a first conceptual model.
As seen in Fig. 1, the architecture shall be able to combine a number of data sets with a multitude of formats, supported by semantic technologies. For this purpose we need to develop tools to obtain, clean and transform the relevant data. An ontology will be constructed to enable alignment between concepts in different systems. Such an ontology can build on pre-existing educational data models, but will likely need to be further extended with new concepts to address gaps in the existing specifications. To persistently store and combine educational data from different sources, a RDF database will be used. With this approach we can support data expressed in compliance with both xAPI and Caliper specifications, but also numerous other data models. In this sense the architecture goes beyond the approach taken by organizations such as JISC, and it is more streamlined than data warehousing.
We have just recently obtained educational datasets from a Norwegian higher education institution, thus development of the architecture will soon commence. Studies that use the architecture will follow.