Enhancing Educational Efficiency: Learning Analytics in the Management of Course Development in Virtual Environments José Abad-Troya1, Irma E. Cadme-Samaniego 2 1 Universidad Técnica Particular de Loja, San Cayetano Alto, 110107, Loja, Ecuador 2 Universidad Técnica Particular de Loja, San Cayetano Alto, 110107, Loja, Ecuador Abstract Learning analytics provide a set of elements for data analysis in the educational field. This analysis is complemented with semantic web tools that allow obtaining a wider benefit, overcoming the interoperability barrier, and the capacity to generate new knowledge. The present work aims to facilitate the management of courses offered in the various online education platforms through learning analytics. The analytics process includes extracting, transforming, and loading data stored in an RDF data repository. It has also required designing and implementing an ontology representing MOOC courses. Finally, the visualization of the obtained indicators will be available in a dashboard with access for end users. Keywords Learning analytics, MOOCs, semantic web, ontologies.1 1. Introduction ICT plays a crucial role in the learning process by improving the quality and effectiveness of education. It provides teachers and students with tools and resources that make learning more engaging, interactive, and learner-centered [1] [2]. The use of ICT tools in the classroom can create a more stimulating and practical learning environment, enabling students to better understand subjects and improve their performance [3]. In addition, ICT helps to plan and evaluate the learning process, identify strengths and weaknesses, and find solutions to improve the educational experience [4]. They also help to capture students' attention and increase their integration with knowledge, fostering interaction and rapport between students and teachers. Overall, ICT is an important tool to support teachers, improve student learning, and adapt education to the demands of the digital age. With the incorporation of new technologies in the educational field, it is necessary to have indicators on the use of resources in virtual learning environments, highlighting the learning analytics to address this problem [5]. Thus, in the educational context, the needs of teachers or tutors also extend to data processing, time investment in obtaining indicators, analysis, and selection of resources, among others. By analyzing the behaviors of online learners, learning analytics can provide a better understanding of which environments and experiences are the most suitable for learning. Semantic web technologies can be used to analyze learner activities in decentralized and heterogeneous learning environments, such as MOOC platforms [6]. In addition, ontologies, a key component of the Semantic Web, offer advantages in structuring data in e-learning systems, but tools are needed to analyze learner behavior [7]. Since learning analytics and the semantic web are closely related, we propose the use of semantic web technologies, which constitute an alternative to interoperability problems and CISETC 2023: International Congress on Education and Technology in Sciences, December 04–06, 2023, Zacatecas, Mexico jmabad1@utpl.edu.ec (J. Abad-Troya); iecadme@utpl.edu.ec (Irma E. Cadme-Samaniego) 0009-0009-9784-6208 (J. Abad-Troya); 0000-0002-5554-0560 (Irma E. Cadme-Samaniego) © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ht tp: // ceur -ws .or g Works hop I SSN1613- 0073 Pr oceedi ngs CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings offer advantages of information enrichment and discovery of new knowledge. In this way, we can provide a solution applicable to different Massive Open Online Course platforms, and support decision-making through a dashboard that visualizes certain important indicators regarding the use of resources. This document presents the work done in five sections. A bibliographic review is presented, which is the theoretical basis for the development of the solution to the described problem. Further on, the methodology used to reach the proposed objective is described. The following section summarizes the development of the proposed solution. Then, the results obtained and their discussion are presented. And, finally, some conclusions of this work have been integrated. 2. Literature review Learning analytics (LA) is a field that focuses on analyzing data generated by learners in educational settings, such as massive open online courses (MOOCs). MOOCs are online courses that target a large audience and have become popular due to their accessibility and flexibility. LA provides researchers with the opportunity to evaluate and monitor various aspects of MOOCs, including institutions, students, teachers, and online learning environments [8]. By analyzing data collected from MOOC participants, LA can provide information on learner behavior, progress, and performance that can be used to improve the design and delivery of educational content [9][10]. For example, analyzing registration data can help identify factors that influence MOOC completion, such as students' interactions with textbooks and problem-solving activities [11]. Overall, integrating learning analytics into MOOCs offers the potential to enhance the learning experience, address challenges such as high dropout rates, and create individualized learning environments [12]. 2.1. MOOCs MOOCs or Massive Open Online Courses refer to new online training experiences that emerged as a response to the specific needs of the digital era, involving a series of contents that are basically characterized by being digital, open, and flexible [13]. Its characteristics are linked precisely to its acronym in English, as it says [14]: • Massive: classes are not directed to a specific target audience. Anyone can enroll and start learning, with the need to have a previous educational level. • Open: geographical barriers are not an impediment for people from any region with Internet access to access the course. The content is open or free, although in some cases there may be some cost. • Online: courses are available online. This facilitates access from anywhere at any time. • Courses: there are many courses available in different disciplines. 2.2. Virtual Learning Environments In general, a Virtual Learning Environment or VLE is associated with formal learning and relationships between teachers, students, and educational entities. There is a growing interest in Internet-supported VLEs, namely between educational institutions, students, and teachers. The concept of learning environment is considered as a dynamic definition, due to the constant evolution of digital technologies, the characteristics, and potentialities, as well as the importance that such environments have in the learning process [15]. 2.3. Semantic Web In [13] it is pointed out that the semantic web is an extension of the current web. Information is assigned a concrete meaning, facilitating cooperation between machines and humans for the execution of tasks. The main objective is to enable machines to understand semantic documents and data, with a defined structure agreed upon by all involved. Figure 1 describes the model of layers that make up the semantic web. From URIs, which serve to identify each resource on the web, XML format for files, the RDF resource description framework and ontologies with their rules with the purpose of incorporating computers in the tasks involved in knowledge management. Figure 1: Semantic Web architecture [16] Learning analytics approaches can enhance the lifelong learning experience by analyzing and understanding the silos of data generated by learners [17]. The application of semantic web technologies in evaluation analysis allows for the analysis of evaluation activities, results, and context, which enables the establishment of inference mechanisms for evaluation analysis [18]. The semantic web enables machines to understand and process data, which has implications for evaluating learning in a knowledge management context [19]. This becomes a powerful tool that not only helps teachers and students, but also allows administrators, managers, and authorities of organizations that offer this type of courses to make decisions. 2.4. Related works In this sense, the Semantic Web offers through the generation of ontologies the opportunity to establish relationships between different concepts that form a knowledge domain. Precisely a visualization of results using semantic enrichment to identify the behavior of students presenting some statistics of a MOOC was proposed in the master thesis work Improvement of visualization in learning analytics using enrichment, for which it proceeds to reuse the ontologies oriented to academic analysis, Activities Ontology and Ontology Context of Learning included in the project "IntelLEO Activities Ontology" [20]. The scope of such work ranges from the process of collecting the data, cleaning, transformation of the structural data into hierarchical data, storage of the formatted data, and its subsequent visualization. The set of technologies used for this purpose contains Open edX, to obtain the data, the R programming language to generate visualizations from the generated data, and SPARQL queries of the ontology, while Shiny Server was used as a web server. Santofimia proposes the use of a learning analytics module, based on the Open edX platform, which is available for use in other online education platforms based on the same architecture. The module follows the Model-View-Controller (MVC) architecture for its construction, something typical when using the Django framework for development, as well as the creation of an API to obtain the data generated in the platform, calculation of statistics to present in the visualizations generated with the Javascript programming language, and the management of periodic tasks to avoid the saturation of the server [21]. In the research previous to the realization of the present project, we found the development of a dashboard within the master's thesis Design and implementation of an academic support dashboard based on data from virtual learning environments, which delves into the implementation of a prototype dashboard using learning analytics to show indicators of the learning process of students. For this purpose, data are collected from the Poliformat platform of the Universitat Politèctica de València, these go through a process of ETL (Extraction, Transform, and Load) using the Pentaho Data Integration tool also called Kettle, after this step and with the transformed data access to Tableau services to generate graphs [22]. The development of this prototype is intended to be a model for future related work in the field of learning analytics and semantic web tools, specifically through the use of an ontology focused on the design of a virtual course. Thus, the contribution reflected here is in the field of Learning Analytics by defining an architecture that integrates the processes of data collection, and transformation of these data to map them in an ontology that facilitates the identification of relationships between concepts, which are finally presented in a dashboard or visualization board for the end user. On the other hand, the contribution of knowledge engineering is found in the creation of an ontology for online courses or MOOCs. 3. Methodology The learning management of MOOC courses is affected due to the unintuitive environment and poor usability in the visualization of information regarding the progress of students in the courses, information that is key for decision-making by managers and teachers. On the other hand, large amounts of information are available in raw data, which is being wasted for the study and analysis of student behavior. 3.1. Proposed solution The development and implementation of a prototype of a dashboard or learning analytics control panel enables the visualization of information corresponding to the progress of students in MOOC courses. To develop this proposal, we have the support of the Open Campus initiative of UTPL. It also seeks to contribute to Goal 4 of the Sustainable Development Goals (SDG) within the framework of quality education. For the development of the prototype, the first step was the systematic analysis of literature related to Learning Analytics and the Semantic Web, the technology used, and related projects. Next, the platform was analyzed, and a study was made of the structure of the courses offered, to know how the participants interact. A data set was provided by the Open Campus administrators, which was analyzed. The analysis of a platform course and the data allowed us to develop an ontology to represent the domain of this type of course. With the ontology, we proceeded to convert the data to RDF triples, which were stored in an RDF repository and were ready to be consumed through the SPARQL language. Finally, a framework was developed that allows the visualization of basic indicators, which can provide a reference for the development of the courses to help make decisions to improve their quality. 4. Solution development 4.1. Architecture The architecture of the application is described in Figure 2, from the extraction of data from the databases for subsequent cleaning and subsequent conversion into RDF through a mapping in the Open Refine tool. Then, the data are stored in the RDF Graph DB repository, from where they are consulted thanks to the tool's API, and finally, with these data, the graphs are generated in the dashboard. Figure 1: Application prototype architecture 4.2. Ontology development The ontology called Ontology for MOOC, shown in Figure 3, contains the main concepts for the analysis of a MOOC course. For the selection of the main classes, the student and his properties were considered, including in the class Person, its academic level in AcademicDegree, its country of residence in Country; likewise, the properties of the course in Course, the final grade of the course in FinalCoursGrade, the data of each module in Module, the textual content of the module lesson in Text, each module has an exam with a grade which is recorded in the class Test, the grade of the tasks that may exist are recorded in Assignment. The resources of each module are first recorded as multimedia resources in MediaResource, and then classified according to their type: videos in Video, images in Picture, documents in Document. Figure 3: Ontology conceptual model for MOOCs Table 1 below describes the classes used to build the ontology: Table 1 Description of Ontology Core Classes for MOOCs Prefix URI Description foaf:Person http://xmlns.com/foaf/spec/#t This class is used to represent erm_Person people vivo:AcademicDegree http://vivoweb.org/ontology/c Student's academic degree ore#AcademicDegree juso:Country http://rdfs.co/juso/#Country Country of residence aiiso:Course http://purl.org/vocab/aiiso/sch Course in which you are ema#Course enrolled :FinalCourseGrade http://www.semanticweb.org/j Final course grade mat/ontology_for_mooc/Finalc oursegrade aiiso:Module http://purl.org/vocab/aiiso/sch Course module ema#Module teach:Assignment http://linkedscience.org/teach Tasks assigned in the course /ns#Assignment dc:Text http://purl.org/dc/terms/#Text Lesson, articles, books, or book sections. edu:Test https://schema.edu.ee/#Test Evaluation tool ebucore:MediaResource http://www.ebu.ch/metadata/ Multimedia resources ontologies/ebucore#MediaRes ource ebucore:Document http://www.ebu.ch/metadata/ A publication which can be an ontologies/ebucore#Document HTML or PDF document. ebucore:Picture http://www.ebu.ch/metadata/ Image-type multimedia ontologies/ebucore#Picture resource vivo:Video http://vivoweb.org/ontology/c Video used as an educational ore#Video multimedia resource 4.3. Visualization The application graphs, generated with the support of semantic web tools such as SPARQL for data query, are displayed on a web page to be consulted by the end user. It has options to select the name of the course, the type of the graph, and the button to generate the graph. The process to generate the graphs includes the use of the JavaScript library Chart.js, which with the data obtained from the server in GraphDB generates the respective graph. However, for the rest of the frontend design, Vue JS was used. Figure 4: Website mockup Figure 4 shows the initial design that was contemplated for the development of the front end, including in this space the informative data of the students and the graphs with indicators that are generated according to the user's choice. 5. Results and discussion The functional prototype made it possible to visualize several indicators of the MOOC course offering process. The data with which we worked made it possible to obtain the following results: Figure 5: Approval Out of a total of 2946 registered students, only 4946 students successfully completed the course. Figure 6: Enrolled by level of education The largest number of registrations in the course corresponds to students with a bachelor's degree education with a total of 887. On the other hand, the group with a primary education is the one with the smallest number of students, with a total of 6. Figure 7: Approval by academic level As suggested by the academic literature, the fact that a student in an online course has a high academic background may be reflected in the successful completion of the course. This is true in the case of students who have previously registered in their profile as having a doctorate degree, obtaining the highest pass rate, contrary to what happens with the group of students who indicate having a primary education, although their presence is small, no student managed to pass the course. The case of students grouped in the Master's level of education is somewhat exceptional since it is the second group with the lowest pass rate in relation to the number of registrations in the MOOC course with such level of education and considering that they are fourth level degrees. Figure 8: Approval by gender As can be seen in Figure 8, gender is not a significant factor in the development of the course. The percentage of approval is similar in both cases, although the number of students identified with the female gender shown in the graph is higher than in the case of the male gender, this is due to the greater number of students registered in the female gender. These basic indicators have provided us with information that can be used by teachers, managers, administrators, and authorities for decision-making. 6. Conclusions This work has allowed us to deliver a Learning Analytics tool supported by Semantic Web technologies, which can be integrated into any MOOC course platform, to show indicators that help the decision-making of different actors in the development process of this type of course. The Semantic Web has provided us with technology that supports the production of tools and instruments that can be leveraged to generate new knowledge for the benefit of education by exploiting data and overcoming the barriers of heterogeneity. An ontology has been developed to represent the domain describing a MOOC course, which can be used and reused to extend, and which has been the model that has allowed us to perform the conversion of our dataset into RDF triples. The RDF data repository has been exploited through a framework that allows the visualization of some basic indicators and is expected to integrate more in the future. Learning analytics is very useful to show indicators that support measures of managers or teachers in the process of decision-making in the management of online course content or MOOCs. The data used in this first pilot shows a relationship between the level of education and the success rate in MOOC courses. Students with higher academic levels, such as bachelor's and doctoral degrees, tend to have higher completion rates compared to those with primary or master's level education. Students with elementary education face significant challenges, as none of them manage to pass the course. This may suggest that additional support strategies are needed for this specific demographic. In addition, the analysis supports the idea that prior academic training can influence the successful completion of online courses. Students with doctoral degrees show the highest pass rate, supporting the existing academic literature. Apparently, gender does not play a significant role in course success, as the pass rate is similar between genders. The disparity in the number of students between genders is attributed to a higher female representation in the records. The group of students with master's level education has a lower-than-expected pass rate, which may indicate specific challenges in this group and may require further exploration to better understand the factors behind this trend. Functional prototyping and the application of learning analytics provide valuable information about student performance in MOOC courses. This type of analysis can be crucial to identify patterns, and areas for improvement and design more effective educational strategies. As future work, the mapping of the OFM ontology with a larger amount of data from educational resources used in MOOC courses is proposed, according to the concepts and properties created, facilitating the discovery of new relationships and/or generating new knowledge. References [1] Miranda, Yeni Erita, Rena ELIA, and Nivetiken, “ICT (Instruction Computer Technology) Based PKN and IPS Learning Designs in Elementary Schools,” JOURNAL OF DIGITAL LEARNING AND DISTANCE EDUCATION, vol. 1, no. 9, 2023, doi: 10.56778/jdlde.v1i9.48. [2] S. Orhani, M. Kolukaj, F. Saramati, and E. Saramati, “ICT as a Didactic Tool to Facilitate the Learning Process in the Subject of Mathematics in Combined Classes,” International Journal of Innovative Research in Multidisciplinary Education, vol. 02, no. 05, May 2023, doi: 10.58806/IJIRME.2023.V2I5N01. [3] S. N. Pratiwi, “Learning Management Based on Information and Communication Technology in Improving Student Learning Outcomes,” Randwick International of Social Science Journal, vol. 4, no. 2, 2023, doi: 10.47175/rissj.v4i2.686. [4] D. De Souza, V. Lima Cangirana, and M. Wendel Santana Coêlho, “THE IMPORTANCE OF ICT IN THE TEACHING AND LEARNING PROCESS,” Revista Gênero e Interdisciplinaridade, vol. 3, no. 05, 2022, doi: 10.51249/gei.v3i05.976. [5] G. Sabulsky, “Analíticas de Aprendizaje para mejorar el aprendizaje y la comunicación a través de entornos virtuales,” Revista Iberoamericana de Educación, vol. 80, no. 1, 2019, doi: 10.35362/rie8013340. [6] K. M. Hover and M. Muhlhauser, “Learning analytics in linked open online courses,” Proceedings - 2014 IEEE International Symposium on Multimedia, ISM 2014, pp. 369–374, Feb. 2015, doi: 10.1109/ISM.2014.51. [7] F. Dalipi, S. Y. Yayilgan, and Z. K. Gjovik, “Enhancing the learner’s performance analysis using SMEUS semantic e-learning system and business intelligence technologies,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015. doi: 10.1007/978-3-319-20609- 7_20. [8] S. Zeng, J. Zhang, M. Gao, K. M. Xu, and J. Zhang, “Using learning analytics to understand collective attention in language MOOCs,” Comput Assist Lang Learn, vol. 35, no. 7, 2022, doi: 10.1080/09588221.2020.1825094. [9] S. Kim and T. Park, “Analyzing Log Data of Students Who Have Achieved Scores Adjacent to the Minimum Passing Grade for a K-MOOC Completion in the Context of Learning Analytics,” Sustainability (Switzerland), vol. 14, no. 18, 2022, doi: 10.3390/su141811136. [10] S. K G and M. Kurni, “Learning Analytics in Massive Open Online Courses,” 2021. doi: 10.1007/978-3-030-70258-8_7. [11] E. İnan and M. Ebner, “Learning Analytics and MOOCs,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12205 LNCS, pp. 241–254, 2020, doi: 10.1007/978-3-030-50513- 4_18/COVER. [12] H. Khalil, M. Ebner, and P. Leitner, “Using learning analytics to improve the educational design of MOOCs,” International Journal of Education and Learning, vol. 4, no. 2, 2022, doi: 10.31763/ijele.v4i2.641. [13] T. Berners-Lee, J. Hendler, and O. Lassila, “The Semantic Web,” Sci Am, vol. 284, no. 5, pp. 28–37, 2001, doi: 10.1007/978-3-642-29923-0. [14] M. Altamirano Di Luca and N. González Benítez, “Integración MOOC y web semántica en el proceso de Enseñanza - Aprendizaje del siglo XXI,” 3C TIC: Cuadernos de desarrollo aplicados a las TIC, vol. 9, no. 2, 2020, doi: 10.17993/3ctic.2020.92.41-57. [15] P. Alves, L. Miranda, and C. Morais, “The Influence of Virtual Learning Environments in Students’ Performance,” Universal Journal of Educational Research, vol. 5, no. 3, 2017, doi: 10.13189/ujer.2017.050325. [16] T. Berners-Lee, “Semantic Web - XML2000.” Accessed: Jan. 11, 2015. [Online]. Available: http://www.w3.org/2000/Talks/1206-xml2k-tbl/Overview.html [17] K. Halimi and H. Seridi-Bouchelaghem, “Social Semantic Web-Based Analytics Approach to Support Lifelong Learning,” International Journal of Educational and Pedagogical Sciences, vol. 13, no. 5, 2019. [18] A. Nouira, L. Cheniti-Belcadhi, and R. Braham, “A semantic web based architecture for assessment analytics,” Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI, vol. 2017-November, pp. 1190–1197, Jul. 2018, doi: 10.1109/ICTAI.2017.00181. [19] C. A. Presti and M. A. Nicolosi, “Semantic Web and evaluation of learning,” Journal of E- Learning and Knowledge Society, vol. 8, no. 2. 2012. [20] “Mejoramiento de visualización en learning analytics utilizando enriquecimiento semántico.” Accessed: Nov. 21, 2023. [Online]. Available: https://repositorio.uniandes.edu.co/entities/publication/3327d4f8-8ea0-46fc-9a39- 1c531783037f [21] J. Santofimia Ruiz, “Diseño de arquitectura y visualizaciones para un módulo de analítica del aprendizaje en Open edX,” Escuela Politécnica Superior UC3M -Departamento de Ingeniería Telemática., 2015. [22] V. A. Haro Valle, “Diseño e implementación de un dashboard de soporte académico basado en datos de entornos virtuales de aprendizaje,” Nov. 2018, Accessed: Nov. 22, 2023. [Online]. Available: https://riunet.upv.es:443/handle/10251/111761