This study presents InDash, a learning analytics web service and web application dashboard to collect, analyze and visualize Moodle log data in the form of interaction categories. The document provides an overview of learning analytics applications and data collection processes in learning analytics, with an emphasis on log-based learning analytics indicators. To showcase the use and application of InDash, we propose an example categorization of indicators, based on different learning cycle theories, and we detail the main components of the system: a web service that exposes Moodle log for data collection, and the web application for data categorization, analysis and visualization.
Learning analytics has gained increasing academic and public interest due to the extended use of
elearning solutions over the last two decades [
In this document, we present an Interactions Dashboard (InDash). The plugin has an open-source license and is available at https://github.com/TIGE-UPM/InDash. From a broader perspective, the main objective of InDash is to provide a platform for exploring Moodle LMS log data and to allow tracking and analysis of student interactions and possible student learning archetypes.
The study details the design and architecture of InDash. To contextualize the research, the following subsection provides an overview of learning analytics applications, common data acquisition techniques, and typically selected indica-tors. InDash is built to accommodate different user-defined categories and classifications of interactions. Section 2 presents an example categorization based on theories of learning cycle models, which is used to illustrate the operation of InDash. Section 3 details the fundamental components of InDash (a Moodle web service and web application) and its technical implementation (data retrieval, processing, and visualization). Section 4 summarizes the main conclusions and limitations of the study, and outlines future research avenues.
The goal of learning analytics is the generation of insights about the educational process for students,
teachers, and other stakeholders [22]. Currently, most of the research on learning analytics is focused
on descriptive analysis (representation of current status) rather than predictive (forecasting future
outcomes) [
Typically, learning analytics applications combine multiple results in a single dashboard, allowing
their users to perform meta-analysis of multiple results, to gain in-depth understanding of a process or
for validation purposes [
1.2.
Before data collection, it is important to consider the sources of educational data for analysis (e.g.,
primary data from direct measurement, data resulting from artifacts, repurposed data, transformed data
[
Most of the time, learning analytics applications are embedded within the LMS or run externally [30], and this has a significant impact on how data are accessed. When embedded in an LMS, the learning analytics application generally takes the form of a plug-in. In that case, accessing the data is usually trivial, as the data source and the plugin are part of the very same system. However, system constraints might complicate the development of these plug-ins because extending the LMS also means respecting technology stacks and workflows to prevent jeopardizing the well-functioning of the whole system.
If, on the other hand, the learning analytics application does not reside within the LMS, it is
considered an external application. There is no restriction on the type of external application, including
spreadsheets [
In general, indicators refer to data that contain relevant information to provide concrete insights. Once input data are collected, instructors and researchers can be more interested in analyzing artifacts than focusing on specific interactions, raising the question of the selection of indicators for particular learning analytics applications.
With respect to LMS log data, previous research has followed one of two different paths: either they
focused on selected interactions present in the log, such as viewed files, completed assessments, or time
spent online [
The main difference between the two approaches is the implicit assumption that a subset of interactions may be deemed relevant enough for the learning analytics application in question but that the categorization of different inter-actions may be more convenient for preliminary exploratory learning analytics because it groups different but related interactions and offers a more global and comprehensive view of the learning process.
1.4.
With the goal of providing a platform for data exploration, a high degree of free inspection possibilities is desirable. Knowing about the trade-offs between a plugin and an external application, a compromise must be found between ease of data access and interactive exploration options for the user. To address the problems mentioned above, we propose the design and implementation of InDash as a tool for the extraction and exploratory analysis of Moodle log data.
The information stored in a log is evolving together with the LMS. Moodle 2.7 has, upon fresh installation, around 150 different possible interaction events, compared to more than 500 in Moodle 3.10. The sheer amount of possibilities makes it inefficient or even impossible to inspect the logs manually and search for any patterns. To reduce the effort required, interactions with similar characteris-tics, such as blog entry added and comment created, which imply the creation of new knowledge by students after having internalized knowledge (be it in the form of original content or a reply to someone’s original content) may be categorized together to form a single indicator.
The indicators used in InDash are inspired by various theories of the learning cycle3 [
More specifically, the categorization includes the following elements: 1. Engagement represents non-meaningful learning interactions that, nonetheless, are indicative of student activity and engagement with the LMS (e.g., every time a student accesses the LMS or a course, the events login and course viewed are triggered, respectively). 2. Content refers to accessing course content that helps in the conceptualization, knowledge and learning stage (e.g., course module viewed, content viewed). 3. Application, or knowledge validation refers to students performing assessment activities, where they have to apply the knowledge acquired on the different topics (e.g., submitting an essay, submission created, or answering a quiz, question answered). 4. Dialogue/sharing, or knowledge creation, reflects any original contribution of students that can be seen publicly and with which other participants can engage and interact (e.g. blog entry added, message sent). 5. Track/review groups activities where students can observe the current status of their learning process and reflect upon the information offered (e.g., badge listing viewed, user competency plan viewed). 6. Learning process management (LPM) includes the actions that users can perform to manage their learning process (e.g. calendar subscription created, learning plan review requested).
This section presents a technical implementation of InDash to extract, categorize, and visualize interactions from the Moodle log system. InDash has two components: a Moodle plugin that exposes additional web services, and an external web application that performs the learning analytics from log data and presents the results to users by means of an interactive dashboard. Figure 2 shows the InDash workflow.
The following subsections describe the interface with the Moodle database, the data categorization mechanism, and the dashboard visualization.
3.1.
By default, external applications interact with Moodle using web services. Each web service
corresponds to a function that can be invoked by calling a Moodle REST API endpoint. As there is no
built-in web service available to retrieve log entries related to a single course, a modified version of the
Moodle Connector plugin [
After authentication and course selection, the data are imported upon the user’s request by the external web application via the web service. For data categorization, the user is then prompted to provide the interaction mapping file that links each interaction to an indicator of the desired categorization system. Table 1 shows an example of the entries in the interaction mapping file. The mapping file is currently defined as an MS Excel spreadsheet, allowing users without programming experience to easily modify existing mappings or create new ones. The web application stores this information and uses it during the analysis process described in the next subsection.
Once all data have been collected and stored, the analysis process begins. The web application provides a visual comparison of the indicators between students and the average of all students within the course (Figure 3). So far, three visualizations have been generated to allow the identification of student learning archetypes of interest.
A box plot provides information on the total number of interactions in each category, including the quartile distribution of users in the course and outliers; when one or two specific users are selected, they are highlighted in the graph (Figure 3a). A radar chart, or spider web diagram (Figure 3b), shows the distribution of interactions in each category for two selected students of interest. Radar charts help users to perceive patterns that involve several dimensions more easily [24]. Furthermore, another box plot (Figure 3c) presents the values of selected students in the context of the entire class. For a more detailed analysis, the user can interactively filter all log entries. Throughout the process, the visualizations are updated live to reflect the entries that match the filters.
In this work, we introduced InDash, a dashboard that facilitates the analysis and visualization of user
interactions in the Moodle LMS. InDash is an updated and enhanced version of the Interactions
plugin used in [
At this stage of development, InDash is an exploratory tool that may help understand the learning process and facilitate educational decision making through data discovery and identification of learning archetypes. However, it does not provide any conclusions or further statistical analysis of the input data. InDash does not incorporate predictive capabilities, such as notifications or the identification of at-risk students. Upon analysis of the data provided by InDash, especially the relationships between categories and academic performance, these functionalities should be incorporated into later versions of the application.
The study also features, to exemplify the capabilities of InDash, the presentation of a new
categorization of interactions that is more suitable for the analysis of recent versions of Moodle than
the ones proposed in [
The design and implementation of InDash are not exempt from limitations. First, the plug-in is course-agnostic; this means that it is currently not possible to compare two different courses. One reason why this happens is that teacher roles are defined at course level, and, therefore, the analysis is offered on a per-course basis. The impact of this design decision should be reconsidered after validation with end users.
Second, and related to the first limitation, some of the events in the proposed categorization (e.g., learning process management) do not occur at the course level, but rather at the site level. While it is relatively easy to incorporate these events for users with Moodle administrator roles or using tailored queries to the database, the results obtained from the analysis of these interactions might introduce different degrees of bias, as they would collect interactions that could be related or not to the activity of students in a specific course.
Third, and final, InDash assigns the same weight to each and every interaction; in other words, all interactions are deemed equal, irrespective of their educational value (e.g., creating and replying to a Moodle forum thread are considered similar dialogue/sharing interactions, submitting an essay and answering a quiz are both similar application interactions). Although this approach is useful for initial exploratory analysis and validation of interaction categories, further research on interaction categorization is required to obtain a fine-grained analysis of the educational process.
[1] Acquila-Natale, E., Iglesias-Pradas, S., Hernandez-Garcıa, A., Chaparro-Pelaez, J.,RodrıguezRuiz, I.: Mwdex: A moodle workshop data extractor. In: Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality. pp. 297–303 (2019) [22] Romero, C., Ventura, S.: Educational data mining and learning analytics: An up-dated survey.
WIREs Data Mining and Knowledge Discovery 10(3), e1355 (2020) [23] Sharef, N.M., Murad, M.A.A., Mansor, E.I., Nasharuddin, N.A., Omar, M.K., Rokhani, F.Z.: Personalized learning based on learning analytics and chatbot. In: 2021 1st Conference on Online Teaching for Mobile Education (OT4ME). pp. 35–41 (2021) [24] Srinivasa, K.G., M., S.G., H., S.: Advanced Visualization. In: Network Data Ana-lytics. A HandsOn Approach for Application Development, chap. 19, pp. 361–383. Springer International Publishing AG Switzerland, Cham, Switzerland (2018) [25] Tsai, Y.S., Gasevic, D.: Learning analytics in higher education — challenges and policies: A review of eight learning analytics policies. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference. p. 233–242. Association for Computing Machinery, New York, NY, USA (2017) [26] Uskov, V.L., Bakken, J.P., Shah, A., Hancher, N., McPartlin, C., Gayke, K.: Inno-vative interlabs system for smart learning analytics in engineering education. In: 2019 IEEE Global Engineering Education Conference (EDUCON). pp. 1363–1369 (2019) [27] Verbert, K., Ochoa, X., De Croon, R., Dourado, R.A., De Laet, T.: Learning an-alytics dashboards: The past, the present and the future. In: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. p. 35–40. Association for Computing Machinery, New York, NY, USA (2020) [28] Vieira, C., Parsons, P., Byrd, V.: Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education 122, 119–135 (2018) [29] Wedman, J., Diggs, L.: Identifying barriers to technology-enhanced learning en-vironments in teacher education. Computers in Human Behavior 17(4), 421–430 (2001) [30] Wise, A., Zhao, Y., Hausknecht, S.: Learning analytics for online discussions: Em-bedded and extracted approaches. Journal of Learning Analytics 1(2), 48–71 (2014)