Learning analytics has the potential to contribute to the improvement of quality in education. Still, there is scarce evidence of its use in practice, especially in school contexts. The QUALAS (Quality Assurance for Learning Analytics in Schools) Erasmus+ project aims to enhance the capacity of secondary schools to utilize learning analytics within quality assurance. To do so, the project aims to identify practices related to quality assurance and learning analytics in schools (QUALAS) in each participating country (Belgium, Ireland, Italy, and Spain). We describe here the overall goals of the project, its workplan, and some initial findings regarding the existing practices related to QUALAS in the Spanish context.
This paper presents QUALAS and the initial outcomes of the work carried out by the Spanish partners. It aims to provide preliminary evidence on how learning analytics is being used—or could be used—for quality assurance in Spain, with a particular focus on Castilla y León, where the project is currently being implemented.
The rest of this paper is structured as follows. The next section introduces the project, its goals, and the workplan. Section 3 describes the current policies and practices related to QUALAS in the Spanish context at a national and regional level. Section 4 describes the first impressions obtained by analyzing documents provided by the participating schools in the case studies. Finally, the last section provides conclusions and an overview of future work.
QUALAS is an Erasmus+ KA cooperation project devoted to creating partnerships for school education. It is led by Vrije Universiteit Brussel (Belgium) with the participation of Dublin City University (Ireland), Universidad de Valladolid (Spain) and Instituto nazionale per la valutazione del sistema educativo di instruzione e di formazione (Italy). It spans from October 2023 to September 2026. Its objectives and work plan are described in the rest of this section.
The QUALAS project aims to support school leaders, middle managers, and especially those interested in playing a role in quality assurance and data use by ofering tools, a training module, and good practices to use data in their learning management systems (LMS). This objective will be realized by developing three concrete aims: 1. Identifying practices that can stimulate the dialogue in secondary schools for using learning analytics in the quality assurance framework. 2. Developing a training module for school leaders, middle managers, and teaching staf interested in learning analytics data and/or quality assurance staf in secondary education. This training module will address the following topics: What are learning analytics? How can learning analytics be integrated into processes of quality assurance? What are the ethical issues that should be considered when developing plans for making use of learning analytics? 3. Developing supporting tools that can facilitate school practices, like checklists for organizational readiness (data/system, ethical and competence level), and prompting boards that will stimulate conversations in secondary education school teams on how learning analytics can contribute to quality development.
QUALAS workplan consists of three implementation workpackages (WP2-4), together with two workpackages (WP) devoted to project management (WP1) and quality assurance (WP5). The implementation workpackages are the following:
This workpackage aims to identify the core concepts from the two fields involved in the project—quality assurance and learning analytics. It focuses on analyzing the diferent narratives within each field and bridging them together. This WP includes a literature review of quality assurance and learning analytics in each jurisdiction, primarily based on grey literature such as regulations and policy documents that outline how quality assurance is addressed in each educational context studied in the project.
One of the expected outcomes of this WP is identifying current practices (or the absence thereof) in each participating jurisdiction and developing a conceptual framework that explores the afordances and constraints of using learning analytics for quality assurance in secondary schools.
Due to the innovative nature of the project and the fact that the application of learning analytics and quality assurance in secondary education is a highly unexplored area, this WP aims to inspect the issue in depth by analyzing real cases in secondary schools. The case studies are based on the collection of documents and interviews with relevant roles related to quality assurance and learning analytics in the schools.
The cases follow a case study protocol organized by the recommendations by [
This WP comprises several activities. The first involved establishing a set of criteria for selecting cases to capture the full spectrum of maturity in using learning analytics for quality assurance in schools, ranging from those that do not use them at all to (ideally) those that have fully integrated a culture of quality assurance and data analysis. These criteria also guide the selection of interviewees, including key profiles such as school principals, ICT coordinators, and quality assurance oficers. Other roles, such as novice and experienced teachers, are also considered to ensure diversity. In selecting these profiles, the project will consider diferences between jurisdictions and, where necessary, incorporate additional jurisdiction-specific criteria. The second activity developed a case study instrument to guide data collection in each selected case. The instrument provides a guide for questions that will be addressed in interviewing the respondents and provides guidelines on gathering physical artefacts from the case. The last activity in this WP consists of analyzing the collected data, based on a set of pre-specified categories. The analysis is done first at a local level, producing a local report, and then a cross-case analysis in a case study report. This report will feed into the activities in WP4.
Based on the indicators framework (WP2) and the case studies (WP3), this workpackage is responsible for proposing resources for capacity building of practitioners for using learning analytics for quality assurance purposes. The WP4 main output will be a set of conceptual tools and training modules. These modules will be based on schools’ needs in each jurisdiction to fit their specific contexts and needs in using learning analytics for quality assurance. The information from the previous workpakages will enable us to take a human-centred approach in developing the tools and training module.
To safeguard the quality of developed materials and anticipate the needs of our target population, these materials will be proposed following a co-creative approach. The co-creative approach will be preceded by mapping out the expectations of our practitioner target population before the start of the development stage. This development will be conducted in several iterations.
At the time of writing this document, WP2 has carried out a rapid narrative report of a systematic
literature review following the guidelines in [
The rest of this paper reports initial results obtained by the members of the Spanish partner in their efort to understand how learning analytics and quality assurance are currently being implemented in their jurisdiction and the afordances and challenges for that implementation to be efective.
The first activity carried out as part of WP2 was to identify the reported current practices and policies related to quality assurance and learning analytics in each jurisdiction. This section presents the procedure followed and the findings obtained by the Spanish partner, who is part of a university in Castilla y León. Due to the decentralized nature of education in Spain, the regional policies are relevant and must be reviewed to complement European and national policies.
The review presented in this section is based on the purposeful sampling of information related to
policies on quality assurance and LA available at institutional sites at the European [
Additionally, we carried out several queries using Google Scholar to identify concrete cases in which
LA has been used in the context of secondary education in Spain. Still, we found almost no evidence
reported in the literature, except for Donate-Beby et al.[
Finally, we discussed with several experts who have helped us understand the current state of afairs regarding quality assurance and usage of digital technologies: namely, an inspector in charge of the quality assurance evaluation, a responsible for quality in a secondary school specialized in vocational training, and two teachers in charge of ICT in their high schools.
We summarise the main outcomes of this review in the following two sections. The first one provides an overview of the structure of Secondary Education in Spain and Castilla y León, which is provided to help understand the context of the schools selected for the study.
The Organic Law on Education 3/2020 (LOMLOE) defines the secondary educational system in Spain. Spain has a decentralized educational system where educational competencies are shared between the General State Administration (Ministry of Education and Vocational Training) and the authorities of the autonomous communities (Departments for Education). Likewise, the regional education authorities develop the State regulations and have executive and administrative competencies to manage the education system in their own territory. On the other hand, schools have pedagogical, organizational, and managerial autonomy over their resources.
Basic secondary education may take two forms: 1) Compulsory Secondary Education (ESO) is provided in secondary schools between the ages of 12-16. At the end of this stage, students receive their first oficial qualification. This Compulsory Secondary Education Graduate diploma gives them access to upper secondary education or the labour market. 2) Basic-level training cycles are ofered in secondary education high schools between the ages of 15 and 17. Students who complete these courses receive a diploma as a Graduate of Compulsory Secondary Education and the qualification of Basic Technician in the corresponding specialization.
Upper secondary education is also mainly provided in high secondary education. It lasts two academic years, and is generally attended between the ages of 16 and 18. It ofers students two possibilities: • Bachillerato (general option). Upon completing high school, students receive the Bachiller’s diploma. • Intermediate vocational training (vocational option). The latter is also ofered in high schools, integrated vocational training centers, and national reference centers. Once these studies are completed, students receive the title of Vocational Training Technician. Those graduates who wish to do so can continue their studies with intermediate vocational training specialization courses in those professional families where this option exists.
In Castilla y León, secondary education is provided by two kinds of schools: high schools or secondary education institutes (IES, Institutos de Educación Secundaria) that ofer ESO, Bachillerato, and training cycles at all levels; and integrated vocational training centers (CIFP, Centros Integrados de Formación Profesional) which are fully devoted to vocational training courses. 3.2.1. Policy context: Quality assurance in Spain Quality assurance in Spain is carried out at diferent levels and processes. The overall schema can be found in English on the Eurydice portal. According to this source, several responsible bodies oversee quality assurance: • State Educational Inspectorate (SEI): Its scope is not quality assessment but ensuring compliance with the rules of the educational system. Duties include verifying academic and professional qualifications, ensuring compliance with national curriculum regulations, and overseeing government-funded subsidies and scholarships. • National Institute for Educational Evaluation (INEE): This body, under the Ministry of Education, Vocational Training and Sports (MEFD), evaluates the Spanish education system. It coordinates annual reports on quality indicators at the national and regional levels. • Educational Inspection Bodies of the Autonomous Communities: These bodies inspect all aspects of the education system and serve as a bridge between the administration (both national and regional) and educational institutions, playing a key role in external school evaluations. • Bodies for the Evaluation of the Education System in the Autonomous Communities: Some regions have established dedicated bodies for educational evaluation, while others integrate these responsibilities into existing educational authorities.
The most recent definition of quality assurance in Castilla y León is established by ORDEN
EDU/701/2023 [
At the school level, leadership teams oversee internal evaluation, while the inspectorate board
manages external evaluations and supervision.
3.2.2. Technology Integration in Educational Centers
The Regional Government of Castilla y León has launched a strategic plan to integrate Information
and Communication Technologies (ICT) in educational centers. This plan outlines how these centers
can use technology to improve the quality of teaching and learning. As part of this initiative, “CoDiCe
TIC” has been developed—a certification that evaluates digital competence in ICT integration. The
certification process complies with European Union guidelines, and the SELFIE tool is required as
evidence for assessment. CoDiCe TIC categorizes schools into five levels, with level 5 representing the
highest standard.
3.2.3. Learning Analytics in Spanish schools
The integration of LA in Spanish schools is not widespread, and there is no formal regulation for its
use in decision-making at the school level. A recent systematic literature review [
• High number of studies on game-based learning analytics.
Quality assurance mechanisms are somewhat linked to Learning Analytics through indicators used by the National Institute for Educational Evaluation (INEE). In Castilla y León, a regional database provides insights into the school system (2014-2022). Schools can also use the SELFIE self-assessment tool, though it is not mandatory.
At the national level, capacity building in Spain is led by INTEF (National Institute of Educational Technologies and Teachers’ Training). INTEF provides training initiatives on Learning Analytics, including online courses to improve student learning and promote personalized education. At the regional level, capacity building is organized by Centers for Teacher Training (CFIEs). These CFIEs can be specialised in diferent types of training: language learning, technology-enhanced learning, etc. One of these centers, of special interest for QUALAS, is devoted to training school leaders.
In conclusion, this review of existing policies related to quality assurance shows that INEE mainly accomplishes external evaluation at the national level. At the regional level, internal evaluation processes are defined in the context of “quality experiences”, which may take many shapes and are not compulsory for schools. The review also showed that implementing LA in Spanish schools is still scarce, with microlevel experiences oriented to pedagogical innovation (like the reported ones related to gamification). There is no evidence of an institutional approach to combining QA and LA. Also, we found no reports of current QA practices based on LA being accomplished or even fostered in our educational context.
Due to the innovative nature of the project and the fact that current practices at the school level may not be reported in the literature, it is necessary to go deeper into the field and see how all these concepts are being implemented, or at least, what are the perspectives of school management and teachers about the possibilities that LA may pose to enhance quality of learning in secondary schools. This is the main expected outcome from the interviews in the case studies. The following section reports the first impressions obtained from the first phase of data collection.
Following the case study protocol defined for WP3, the partners conducted the selection of schools and collected relevant data. VUB obtained ethical approval as the coordinating institution.
Given the aim of identifying a schools’ diversity of practices regarding the use of learning analytics for quality assurance, selection of cases was based on the following set of criteria: i) the focus of the selection should be schools with experience or interest in carrying out quality assurance for learning analytics; and ii) the selection of schools should also include schools from a variety of settings which are theoretically relevant to the research question: high/medium/low diversity composition of the student body; urban/rural; big/small; and, if possible, big/small achievement gaps.
With these criteria in mind, the Spanish partner met with the Head of the Teacher Training and Innovation Service at the regional government of Castilla y León. This person can grant access and permission to work with the schools. Together with this regional government representative, we identified six schools in our region that were trying to meet the specified criteria. Five of them agreed to participate in the case studies, which enabled us to meet the target number of schools defined in the protocol.
The characteristics of the five participant schools are summarised in Table 1. We selected two types of schools: three secondary education institutes (IES) and two integrated vocational training schools (CIFP) (see section 3.2 for a description of these two types of schools). The selection was based on schools’ interest in participating in pedagogical innovation projects and their certified level of ICT integration. Top-ranked CoDiCe schools were selected, assuming that LA practices would be more feasible or better understood in these institutions.
We aimed to include schools with diverse profiles and locations: two in a small town (with fewer than 65,000 inhabitants) and three in larger cities situated in diferent areas (suburban and town center). Additionally, CIFP (vocational training schools) were selected to diversify the sample further, as well as because of their long-standing involvement in quality assurance certifications. This allowed us to compare how a culture of quality assurance within the schools might influence their perspective on using LA to support these QA processes.
Only public schools were considered, meaning the student population in all selected schools is generally diverse. However, due to this choice, students from upper-class backgrounds were not represented in our study.
School School 1 School 2 School 3 School 4 School 5
Students Teachers 600 students, 70% from the nearby wider district, 20% minorities / 79 teachers 650 students. A very diverse population, urban and rural areas / 81 teachers 387 students. 32 intern students from rural areas.
Very diverse population. / 85 teachers / Interviews 1 teacher (f) (coordinator of BITS program), 1 training coordinator (f), 2 novice teachers (f, m), 1 experienced teacher (m) 1 principal (m), 1 ICT coordinator (m), 1 quality coordinator (f), 1 studies sponsor (f), 1 experienced teacher (m), 2 novice teachers (f) 1 principal (f), 1 ICT coordinator (m), 1 head of department (m), 1 novice teacher (m), 1 experienced teacher (f) 1 principal (f), 1 coordinator of bilingual program (f), 1 vice-principal (f), 1 ICT coordinator (m) Urban / Small 881 students from Town. urban area / Middle class / 105 teachers Urban with students from rural areas.
750 students from urban and rural areas / 72 teachers 1 principal (m), 1 ICT coordinator (m), 1 quality coordinator (m), 1 head of department (m); 1 novice teacher (m); 1 experienced teacher (m) Type
Location Secondary Educa- Urban tion School BITS1 Center. Level 5 of ICT integration Vocational Training School. ISO 9001 quality certification. Level 5 of ICT integration Secondary Education School. Level 5 of ICT Integration Secondary Education School.
British Council.
Level 4 or ICT Integration.
Vocational Training School. ISO 9001 certification.
Level 5 of ICT integration
Urban. Suburban area Urban in a small town
Main characteristics of the schools and the participants in the case studies. 1 BITS: Bilingual, Inclusive, TechnoThe case study protocol identified the following goals for the collection of data: • The collection of documents relevant to the study’s goals, such as the school’s mission statement, the policies and procedures, and, if available, data from LA collected in the schools. • The collection of views of diferent actors concerning exactly how they use (or perceive) QUALAS. • The study of QUALAS—by asking more thorough and process-related questions (which qualitative approaches allow)—confronts these accounts to arrive at a more comprehensive image of a school’s practice.
These goals were operationalised in a data collection plan comprising two main phases. The first phase (preparation) was devoted to collecting information about the school, using the analysis of documents and a telephone call with the contact person at that school. The outcome of this phase is a document with a general description of each school, regarding aspects such as demographics, size, location, special features, a descriptive account of the school’s pedagogical mission, the school policies and procedures in place for quality assurance, digital learning and teaching, and which digital learning management systems and learning analytics (if any) and school evaluations. The telephone call complemented the data obtained from the documents and identified the persons who would participate in the interviews. Table 1 reports the main data related to schools’ demographics and the profiles of the persons selected to participate in the interviews.
The second phase consisted of the interviews themselves. They were organized around four central axes: evaluation and quality assurance; learning analytics (pre-conceptions); school culture and characteristics; and learning analytics and quality assurance in the schools. The interview protocol was flexible to enable project partners to make reasonable adaptations. For example, in the many cases where the participants could not identify current practices related to LA in their schools, questions were re-shaped to ask them about the conditions that would lead them to use data (or not) to support evaluation and quality assurance.
This section reports the first impressions obtained after analyzing the documents and the interview with the contact person in each school. Besides the demographic data reported in Table 1, the analysis of documents focused on the reported procedures for quality assurance, the use of digital learning management systems, the internal procedure of scholar evaluation, and any evidence of learning analytics procedures.
Regarding the reported procedures for quality assurance, schools S2 and S5 (the CIFPs) report having a well-established procedure, thanks to their long-standing experience with obtaining the ISO 9001 certification. The other three schools are less homogeneous in their reported experiences. Under this topic, S1 lists a series of meetings at diferent levels, although it mentions a continuous improvement procedure consisting of three phases: diagnosis, evaluation and constant innovation and improvement; S3 reports its participation in specific ‘quality experiences’ (see section 3.2.1) most of them devoted to environmental education, but it does not refer to quality assurance procedures; S4 did not provide information about this topic.
All the schools are actively promoting the use of these tools. Various digital platforms are mentioned in the documents, such as the Microsoft 365 suite (with special emphasis on MS Teams), Moodle, and Stilus, which the schools have started to use very recently. Stilus is a new tool developed by the regional government devoted to classroom management, evaluation, and family communication. Overall, the documents do not report global criteria for using these platforms in a systematic or mandatory manner. In other words, it is up to each teacher to decide whether and how to use the platforms and how to study and make educational use of the data generated. However, a more widespread use of MS Teams has been detected as a tool for online teaching and virtual complement. This is mainly due to its large-scale implementation in the educational system of Castilla y León, where the Junta mandates the use of MS Teams for communication purposes. As a result, the use of Moodle as the primary tool has decreased.
The schools report using a variety of other specific tools for evaluation, such as Blinklearning, Quizziz, Kahoot, and digital portfolios for continuous and formative assessment.
The documents analyzed provided little evidence of using data to support evaluation or learning. The only exception was the mention of the Electude tool (1), used in a vocational training course to collect statistics on students’ learning sequences in that module. In general, most practices related to data use for internal evaluation were based on the manual analysis of student results, supported by traditional timesheets, as reported by schools S2 and S4.
The analysis reveals that most schools do not systematically establish quality assurance processes, except for two with ISO 9001 implementation. Schools are actively participating in various innovation programs and widely using MS Teams along with a new management tool called Stilus. This unified technological support ofers the potential for an integrated solution for learning analytics. However, data is either not easily accessible or very limited. Special procedures should be implemented to gain access to these data traces, particularly between the administration and the tool provider (in the case of Ofice 365). We found very few examples of actual data used to support evaluation, and even fewer examples of using these traces to provide information to teachers and/or students.
In summary, even though the selection of schools aimed to maximize opportunities to meet schools that could report experiences using LA for QA, these practices are almost nonexistent. This analysis helped shape the interviews, which were adapted to the fact that participants may not be familiar with any uses of LA. As a result, the questions focused on participants’ perceptions of the potential benefits and challenges of LA for QA.
These initial impressions, derived from the analysis of available documents, provide an incomplete and probably not entirely accurate picture of the state of QA and LA in the schools. Therefore, these impressions must be complemented with data obtained from the interviews and will contribute to the qualitative analysis of the data.
Learning analytics brings the possibility to use data in eficient ways to improve education. However, the potential, especially in schools, is still not realized. This is due to the lack of a culture related to school quality assurance, platform limitations, and appropriate capacities among leaders and teachers.
QUALAS aims to contribute to capacity building regarding quality assurance and learning analytics. Due to its human-centered approach, the project has started by getting knowledge about current practices, challenges, and opportunities in schools across four countries.
The results reported in this paper refer to the analysis of existing policies related to quality
assurance and learning analytics in one of the participating jurisdictions (Castilla y León, Spain), which
complements the findings presented in [
The development of instruments for WP4 will take into account this analysis once it is completed. Although the analysis is not yet complete, one of the most outstanding results is that there is scarce awareness in the schools about the potential of LA for supporting internal evaluation and other processes related to quality in education. Another fact is that, even with a common platform, there are dificulties in getting the data from those platforms if the schools depend on external services (and also on the tool provider) to provide the data.
All these factors and the other findings obtained by analyzing the cases in the four participating countries will be the basis for co-creating training modules to address the gaps identified in the project.
This research is partially funded by the Erasmus+ programme of the European Union under project 20231-BE02-KA220-SCH-000159845 (QUALAS), and by MICIU/AEI/10.13039/501100011033 under grants PID2020-112584RB-C32 and PID2023-146692OB-C32.
The author(s) have employed ChatGPT for text formatting (tables, BibTeX entries) in LATEX. After using this tool, the authors edited the content as needed and take full responsibility for the publication’s content.