Before the Covid-19 pandemic, the use of Learning Management Systems (LMS) had not been mandatory nor regulated at the primary level of education in the Republic of Serbia. After the Government decision on suspending face-to-face classes, the Ministry of Education obligated schools to implement a combined (hybrid) teaching model based on digital technologies. The aim of this paper is to analyse the quality of frequently used LMS for implementing an effective hybrid teaching model in primary schools in the Republic of Serbia, with regard to the current educational policies. The quality analysis was conducted based on the ELQ criteria. The research results indicate that the sampled LMS meet the set criteria, and although mandatory, the set criteria are not sufficient pre-condition for implementing the quality hybrid teaching in primary education.
Learning Management Systems (LMS) such as: Moodle, Canva, Talent or Google Classroom are applied in different scope and quality, depending on the type and level of the educational system that uses them, i.e. their goals and characteristics, as well as the technical and personnel capabilities of the institutions.
In the educational system of the Republic of Serbia, at the level of primary school education, the use of these systems had been neither mandatory nor regulated until the Covid-19 pandemic started. Due to the outbreak of the pandemic and the declaration of a state of emergency and the Decision of the Government of Republic of Serbia on the suspension of classes [1] in higher education institutions, secondary and primary schools, classes in the form of video lectures lasting 30 minutes were organised and publicly broadcasted via the Radio Television of Serbia (RTS). Alongside video classes, the accompanying activities for the schools, teachers and students were all organised by the Ministry of Education, Science and Technological Development (MoESTD). In this way, the continuity in the education of students and the work of educational institutions was preserved, at the achievable level and in accordance with the circumstances of the health crisis in which the entire society and the educational system were at that time. Via television channels and multimedia internet platforms, video lectures with certain interactive elements were broadcast for primary and secondary school students. At the same time, the schools organised follow-up educational work for their students in line with the provided instructions of MoESTD, in accordance with the available digital devices and tools, as well as the level of digital competences of the teachers at the given moment. In cooperation with the Office for IT and eGovernment of the Republic of Serbia, an internet location was established (rasporednastave.gov.rs), where the schedules of television video lectures were available; a link to the national platform for online learning built on the Moodle LMS, called My school (mojaskola.gov.rs) which, in addition to televised video lectures from RTS, also offered the possibility for interactive work through tests for self-assessment of students’ knowledge, as well as recommendations for establishing online communication between teachers and students, i.e. software solutions and instructions. Furthermore, MoESTD issued a Manual [2] which instructed the necessary adjustments to the school work organisation should include, i.e. the application of a certain learning model in primary schools in order of health preservation and accordingly, promotion of the LMS application within the existing teaching organisation. New organisational approaches to the implementation of distance learning in the period of crises were designed in accordance with the knowledge about the current level of digital maturity of institutions, the level of digital competences of teachers, opportunities and while taking into the consideration the short deadlines. For periods when schools had to move to a total distance learning model, organisational guidance was also provided, which took into account the resources and level of digital maturity of the school and student capabilities [2].
The aim of this work is to detect which most frequently used LMS are in primary schools during Covid-19 pandemic, and then to analyse the quality of most frequently used LMS for implementing an effective hybrid teaching model in primary schools in the Republic of Serbia, in regard to the current educational policies. The analysis of the quality of the applied LMS as well as software solutions and tools in the national school system during the pandemic within the hybrid model of primary school teaching in the Republic of Serbia is planned based on first seven ELQ (e-learning quality) criteria.
The question of the quality of the implemented hybrid teaching in primary education rests on the quality criteria of the LMS, but also opened the questions of the digital competencies that are part of the complex professional competencies of teachers and the importance of digital maturity in schools.
It should be emphasised that there are different definitions of LMS. According to its basic
characteristics, it is specialised software used for planning and implementing the teaching process in an
online environment and involves enabling the interaction of all members, as well as ensuring the
conditions for evaluating student achievements. Also, within the LMS, the planning of the teaching
process includes, among other things, the creation of digital teaching content, the addition of
educational resources, the design of various learning activities (assignments, workshops, forums) that
foresee providing feedback, encouraging peer learning and providing transparent formative and
summative assessments of student achievements [
Google Classroom was created in 2014 as part of the Google G Suite for Education package, which required registration of the institution. In 2017, Google made Google Classroom available to all users with a Google Account, which can be used to create online courses. The platform relies on Google tools, primarily on Google Drive where documents are stored, but also on other Google applications such as Google Documents, Sheets, Slides, questionnaires and various upgrades. This free platform allows teachers to build an online environment, where they can share materials, links, communicate with students, set assignments and evaluate their activity. A free Google G Suite for Education package was available for schools, which allowed the schools to manage and decide which Google services students could use, and it was supposed to provide a safer environment in terms of protecting the privacy and security which is important in a school environment. Google Classroom is a type of user-friendly platform for both teachers and students. All participants, both teachers and students, use it intuitively, which certainly affected the percentage of use of this platform.
During the pandemic period, the Microsoft Corporation provided schools with free use of a platform adapted for education - Office 365 for Education. The platform included Office Online (Word, PowerPoint, Excel and OneNote), one terabyte of OneDrive storage, Yammer, SharePoint, as well as other useful services such as Teams, which enables conducting a video conference which resembles a classroom. Although not many primary schools had been using the Microsoft platform before, it was widely spread at the University level - the services of this platform were used by over one hundred and ten teachers who are in the academic institution and students, and it had a set of different services that enabled teachers to collaborate effectively with students.
On the website for school support established by MoESTD, there were instructions for using the Microsoft Teams application, with which teachers could share the content of their screen with students through a group video call, as well as record the entire course of the lesson and make it available to all students from the corresponding study group. The platform could be accessed by millions of teachers in the world’s leading universities, colleges and schools, it was available free of charge for computers and mobile devices; for those who had no other access to the Internet, free access to the platform was provided through the nationwide telecommunications operators. Also, video instructions have been prepared for teachers on how to use the video conference tool, namely: how to form a class and subjects; how to schedule and hold a video conference class; how to distribute teaching materials to students; how to prepare an assignment and how to answer the assignment; how to review the returned assignments. Video instructions with guidelines were also made for the students.
Edmodo is a global educational network that was another available free software option, created
in 2008, with the aim of connecting and cooperating teachers and students, sharing teaching content,
sharing messages within groups, connecting teachers, sharing pictures and videos, recording activities
in the calendar, forming a personal library with important web addresses, monitoring student work,
publishing student work in one place, creating quizzes and surveys. Also, students’ answers are
processed statistically and automatically, which ensures immediate feedback. Teachers, students and
parents have the option of opening a user account on this platform, through an administrator or email
address. The research results on the use of the Edmodo platform during the Covid-19 pandemic show
that it is effective and can be successfully used in a hybrid teaching model [
The Serbian Edmodo sub domain is accessed on the website srb.edmodo.com. This system has a user-friendly interface which is suitable for primary school students and is simple to use. On the national platform for online learning, there were no video instructions for its use, as was the case for the My Classroom TeslaEDU system based on the Moodle platform, The Institute for Improvement of Education of the Republic of Serbia prepared instructions for its use.
Moodle is a free online distance LMS with a professional and stable work environment for lectures and incorporates many pedagogical tools. The structure of Moodle supports many types of data, and is flexible for changing the appearance and functionality of an individual page or course. For this reason, it was used at many faculties in the Republic of Serbia. In this environment, students can interact both with their peers and the teacher, participate in polemics on the forum, use the interactive chat room, search and collect data, send their homework, take part in quizzes, use the wiki module and more. Also, it is available as the translated version in Serbian language. It is important to note that recently, trainings for school Moodle administrators from all primary and secondary schools in the Republic of Serbia have been conducted for the implementation of the Moodle LMS (90% of primary schools delegated participants); two iterations of trainings Online and hybrid teaching in a digitally proficient institution in which over 2,000 teachers, professional associates, headmasters and pedagogical advisors involved in the pre-university education participated; the online Instrument for Self-Assessment of Digital Competences of Teachers in Serbia Digikomp2 was published, which was developed according to the Framework of Digital Competences - Teacher for the Digital Age 2019, the revision of which is planned in 2023; the Digital School program was started as a pilot project as well as the Academy of Digital Schools.
My TeslaEDU classroom - represents a unified package of functional modules based on the
Moodle LMS, localised in the Serbian language and provided by the MoESTD in cooperation with the
Office for IT and eGovernment and one tech company. For the purpose of helping out the users of this
system, the website rasporednastave.gov.rs contained instructions for: accessing the web and mobile
application, for teachers and students respectively; attending the course; reviewing and completing
tasks; taking tests; for students communication (from the web browser, as well as from the mobile
application). Video instructions were provided for teachers on how to: assign courses and add class
groups to a course; create and grade assignments; create tests; communicate and monitor student
attendance. The Institute for Education Quality and Evaluation prepared the guidelines for teachers that
should help the users with the above-mentioned system [
Professional instructions for the organisation and implementation of educational work in primary school in the school year of 2020/2021, represented an obligation and starting point for organising, planning and programming the entire educational work in schools in difficult conditions, in accordance with the law and regulations governing the plan and program of teaching and learning, as well as work models that should be applied in school year 2020/2021, depending on the epidemiological situation. The instructions and hybrid models for organising the teaching process were developed taking into consideration the potential of distance learning, as well as the video lectures that were available through the RTS, as well as the possibilities provided by the hybrid teaching model.
In January 2021, the Institute for Education Quality and Evaluation, as an expert institution of
the Ministry of Education, issued the document Online and hybrid learning - long-term aspirations and
short-term guidelines [
The guidelines given for organising hybrid classes were precise, with clear and detailed instructions for planning and organising classes and intended for both the school and the teachers. Compared to the other two models, the hybrid model was most often applied during the pandemic, combining the advantages of the other two enlisted models. Except in the circumstances of the state of emergency, it is also applicable in various different situations.
The most significant feature of hybrid teaching is that face-to-face teaching is enhanced with online learning. There are several models of hybrid teaching, depending on the different approaches and practices of teachers. For example, the flipped classroom is one type of blended learning that can be applied in hybrid teaching, in which students are asked to study independently with selected online materials that will later be used in face-to-face teaching for a specific activity.
The hybrid teaching model enables the effective integration of technology and the school
curriculum. However, similar to some other approaches in learning and teaching, it is necessary that the
hybrid model also follows the pedagogical principles in order to be effectively implemented in teaching
practice. This implies that the teachers are educated to implement methodologies which ensure
achieving the teaching goals [
In our work ELQ model was used in the quality assessment of the LMS that were regularly
used in the Republic of Serbia. The quality assessment model was qualitatively adapted in order to
assess the quality of used LMS as well as software solutions and tools in the national school system
during the pandemic within the hybrid model of primary school teaching. It consists of ten quality
criteria: learning material and content; structure of the virtual environment; communication,
cooperation and interactivity; student assessment; flexibility and adaptability of the system; teacher and
student support; competences and experience of teachers; vision and leadership of the institution;
content resource allocation and holistic and process aspect. The criteria are not listed in the order of
importance; however, fundamental differences exist among them which imply the aspects are ordered
from the smallest elements in the learning process to the more organisational, systemic perspective [
Timely and efficient planning of these elements is of particular importance in online teaching.
Communication in an online environment depends on the available infrastructure, the level of
digital competences of teachers and students, as well as the teaching goals. Communication
itself in an online environment can be synchronous or asynchronous. Online learning
collaboration can be extended to open online communities, or secured and accessible only to
students in a specific department within a LMS. Synchronous communication can be organised
through communication channels and video conference tools between teachers and students.
Synchronous communication is planned on a weekly basis within the guidelines for teachers,
while certain interactive activities can be conducted among students only, without the
synchronous presence of the teacher. When it comes to primary school, precise guidelines for
teachers are given for weekly planning and also how to conduct synchronous and
asynchronous communication with students. Criteria that can be used for implementation and
quality control are: a clear strategy for communication, cooperation and interaction in
accordance with pedagogical needs, available technology and human resources;
implementation and evaluation; improvement of the first two criteria.
3.4. When it comes to assessing students and their achievement, there are significant differences
between assessing students in face-to-face classes and in an online environment, later which
has additional challenges related to student identification, proper formative and summative
assessment, swift feedback etc. However, the way of assessing student achievement in the
online environment can be diverse: by posting students’ text works, through oral synchronous
communication with the help of video-conferencing tools, through individual or group
discussions, by posting digital materials by students, by giving tests... In the hybrid
organisation of teaching, the assessment of student achievements is primarily conducted
during face-to-face teaching, to avoid the shortcomings of online testing, but it is also possible
to use testing in LMS, which provides various opportunities and advantages. Assessment of
students in the online environment is a complex area, and more information about the
formative and summative assessment of students can be found in the available publication of
the Institute for Education Quality and Evaluation [
The qualifications and experience of teachers and associates for the use of LMS include a continuous system of support and professional development, increasing awareness of the use of new technologies, knowledge of how students learn with the help of different media and communication channels, a critical approach to new technologies and improving knowledge as well as the abilities for continuous improvement of the environment for online learning, and above all improving knowledge and abilities for the pedagogically driven use of new technologies. The criteria that could be used for measuring the implementation and quality control are: the strategy for the development of teacher and student competencies, as well as the implementation of the strategy and the evaluation and improvement of the strategy and its implementation.
When it comes to the remaining aspects of quality - the vision and leadership of the institution, the allocation of content sources and the holistic and process aspect - they are not discussed in this paper since they represent a more organisational, systemic perspective of the online learning quality, which can be a separate topic for analysis, within the analysis of educational policy and its application in the Republic of Serbia.
For the purposes of this paper, the analysis of the quality of the applied LMS in the hybrid model of teaching in the Republic of Serbia was performed based on the above-mentioned seven ELQ aspects of the quality of online learning.
Data was obtained from official Informational Database of the MoESTD[
For the implementation of online teaching, which is fully conducted via the Internet, the MoESTD instructed in the Manual [2] that primary schools should use only one LMS per educational cycle, respecting the age of the students, as well as the guidelines for choosing the system: that access to content and activities should be possible at any time, depending on the program and pace of the user’s work (24 hours a day, 7 days a week); that the system should be a closed virtual educational environment meaning it cannot be accessed by an unauthorized persons; that it should be accessible via Internet through various types of digital devices (computer, laptop, tablet, mobile phone...) regardless of the pre-installed operating system (Windows/Android/iOS/MacOS...); that it should be restricted to the Serbian language and/or the languages of ethnic minorities; it should enable the creation of digital content containing text, images, sound, video and links to verified web locations to which the teacher directs students; to enable uploading and downloading of basic files (text document, spreadsheet, multimedia presentation…); that it has built-in functionalities which enable communication between teacher-student and student-other students; to enable the integration of conference tools (sharing video, audio, screen) with the possibility of recording online lectures/content and videos that can be made available to all students on demand (VoD-Video on Demand); to enable teachers to create quizzes/texts or questionnaires to assess students’ knowledge; to have the ability to store teaching materials and student works and to have the ability to integrate other digital online resources [2].
As a form of support, MoESTD has provided teachers and students with instructions for using the following systems and tools on the web location (rasporednastave.gov.rs): My TeslaEDU Classroom (based on Moodle LMS); Microsoft Teams, a video-conferencing tool within the Office 365 for Education LMS; Zoom application and Viber Community application. At that moment, these instructions were useful for teachers, many of whom were encountering the organisation of distance learning on a systemic level for the first time. The instructions were intended to help and support all participants in the teaching process – the teachers, as well as students, and their parents. However, other tools and platforms were also used, which were user-friendly and free, and thus are widespread and well-known, such as Google Classroom and Edmodo, which will be presented in the part of the paper on the solutions used at the level of the Republic of Serbia.
Based on the data obtained from MoESTD [
From the Table 1, we can see that the majority of primary schools used Google Classroom (78.9%), Microsoft Teams was used in a significantly smaller percentage (9.9%), while the Moodle platform was used to a considerably smaller extent (0.7%) which indicates the level of digital maturity in basic education in the Republic of Serbia.
Assessing the above-mentioned systems indicated that all of them meet the mentioned quality
criteria, but that does not speak for the quality of the implemented hybrid teaching. During the analysis
of the MoESTD data [
Based on the data obtained from MoESTD [
The question of the quality of the implemented hybrid teaching in basic education certainly rests on the quality criteria of the LMS, but also on the digital competencies that are part of the complex professional competencies of teachers. When it comes to the initial education of teachers on the topic of obtaining digital competences in the function of creating a stimulative and safe hybrid teaching environment, the results of earlier research indicate that at Teacher Education Faculties that educate future teachers, more attention is paid in comparison to non-teaching faculties that have teacher profiles. Furthermore, faculties also need constant social support and cooperation, as well as a well-coordinated system of professional development of teachers from their initial education during studies to continuous professional development during work [10]. The Education Development Strategy 2030 speaks in favour of strengthening the entire system, which indicates the strengthening of the institution capacity of the pre-university education, which can result in raising the quality of hybrid teaching environments that also include the use of LMS [11].
We can conclude that all four analysed samples of LMS meet the quality criteria according to the ELQ model, which include: building content and setting up learning materials; the structure of the virtual environment; multidirectional communication, collaboration and interactivity; student assessment; flexibility and adaptability of the system; teacher and student support; putting into practice the teachers’ existing experiences; implementation of the institution’s vision and leadership; allocation of content sources and a holistic approach.
Fulfilment of the quality criteria of the LMS according to the ELQ model are necessary, but there are no sufficient conditions for building a safe and stimulating hybrid constructivist teaching environment. The digital competences of teachers, students and associates play a key role, as do other factors (number of students in the class, age of students, nature of the subject, technical prerequisites, preparation of parents etc.) that we must not ignore and that need to be thoroughly and in-depth investigated. Pandemic living and working conditions have intensified the use of hybrid teaching models in basic education in the Republic of Serbia, which has also influenced the educational policy, which increasingly recognizes the importance of digital maturity in schools.
[1] Odluka o obustavi izvođenja nastave u visokoškolskim ustanovama, srednjim i osnovnim školama
i redovnog rada ustanova predškolskog vaspitanja i obrazovanja, Službeni glasnik RS br. 30/20,
Vlada Republike Srbije, Beograd, 2020.
[2] Stručno uputstvo za organizovanje i ostvarivanje nastave neposrednim putem i putem učenja na
daljinu za srednje škole u školskoj 2020/2021 godini, Službeni glasnik RS br. 61/20-03, Ministarsvo
prosvete, nauke i tehnološkog razvoja Republike Srbije, Beograd, 2020.
[