=Paper=
{{Paper
|id=Vol-3085/paper19
|storemode=property
|title=Agile methodology in higher education quality assurance system for SDGs 4, 8 and 9 achievement: national experience
|pdfUrl=https://ceur-ws.org/Vol-3085/paper19.pdf
|volume=Vol-3085
|authors=Artem E. Artyukhov,Iurii Iu. Volk,Tetiana A. Vasylieva
|dblpUrl=https://dblp.org/rec/conf/cte/ArtyukhovVV21
}}
==Agile methodology in higher education quality assurance system for SDGs 4, 8 and 9 achievement: national experience==
https://ceur-ws.org/Vol-3085/paper19.pdf
Agile methodology in higher education quality
assurance system for SDGs 4, 8 and 9 achievement:
national experience
Artem E. Artyukhov2 , Iurii Iu. Volk1 and Tetiana A. Vasylieva1
1
Sumy State University, 2 Rymskogo-Korsakova Str., Sumy, 40007, Ukraine
Abstract
This paper is focused on considering agile methodology as an instrument to use in education quality
assurance. We propose the Scrum method applicable for education quality assurance based on adapted
Manifesto for Agile Education Quality Assurance and twelve principles behind it. The Scrum procedure
is described and roles are distributed for two real-life cases of external and internal educational program
quality evaluation. We illustrate that proposed Scrum procedure perfectly fits existing practices and
can be used to enhance both external and internal quality assurance processes in higher education.
We consider achievement of SDG 4 targets through proposed methodology as the necessary step to
take in achieving SDGs 8 and 9. It is concluded that stakeholders feedback about their satisfaction by
economic and innovative factors should be included in each sprint review procedure in proposed Scrum
methodology. We discuss SDG 4 achieving within multilayered DIKW+DM hierarchy as a framework
for education quality assurance that allow to join information processing, knowledge acquisition and
corresponding decision-making algorithm.
Keywords
education quality, agile methodologies, scrum, sustainability
1. Introduction
Application of agile methodologies in education is a widely discussed topic nowadays. Due
to the constant changing labor market demands, higher education institutions are forced to
adapt approaches aimed at flexible learning. This matter is especially at hand due to worldwide
orientation at life-long education: agility is of crucial importance for keeping the same pace as
constantly changing requirements of society. Recent interest in application of agile methodolo-
gies to form innovative pedagogical tools is logical reaction of educational institutions aimed to
address these challenges.
When building a model for generating knowledge and testing the “quality of education"
system, one should also pay attention to the trends dictated by the world community. These
CTE 2021: 9th Workshop on Cloud Technologies in Education, December 17, 2021, Kryvyi Rih, Ukraine
" a.artyukhov@pohnp.sumdu.edu.ua (A. E. Artyukhov); y.volk@mss.sumdu.edu.ua (I. Iu. Volk);
tavasilyeva@fem.sumdu.edu.ua (T. A. Vasylieva)
~ https://personal.sumdu.edu.ua/artyukhov/en/ (A. E. Artyukhov);
https://pom.sumdu.edu.ua/en/department/personal-pages/252-iurii-i-volk (I. Iu. Volk);
https://biem.sumdu.edu.ua/en/director-biem (T. A. Vasylieva)
0000-0003-1112-6891 (A. E. Artyukhov); 0000-0002-0262-762X (I. Iu. Volk); 0000-0001-9534-1251 (T. A. Vasylieva)
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
81
trends have been outlined as Sustainable Development Goals in United Na- tions Millennium
Development Goals and 2030 Agenda for Sustainable Development. Given that there is a
separate Sustainable Development Goal (SDG) 4 dedicated to quality education, it is important
to establish a link in the proposed approach SDG 4 and other SDGs. This idea is the basis
of the presented material. Considering that the result of achieving SDG 4 is, among other
things, the growth of economic indicators among stakeholders, it should be noted that there is
a specific connection between SDG 4, SDG 8 and SDG 9. The role of university education is to
ensure economic growth and create an innovative infrastructure in connection with the main
stakeholders, which determines the need to consider mentioned relationship between SDGs.
However, labor market’s main interest is not the education process itself, but its outcomes
in form of trained graduate possessing skill-set suitable for current position or able to quickly
adapt to position’s specifics. Despite the small contribution in form of direct pedagogical
innovations, employers are highly motivated to participate in education quality assurance
activities. Therefore, the need arises to form educational quality assurance system that accounts
interests of all stakeholders and allow them to collaborate in efficient and flexible framework.
This paper attempts to provide Ukrainian higher education institutions (HEIs) with such quality
assurance methodology developed upon agile philosophy and principles.
2. Literature review
Agile methodologies in education have their constant attention in forms of direct application to
learning process [1, 2, 3]. As stated in [4], “By using Agile methodologies to design, structure
and steer courses as a whole, or punctual activities and projects, instructors are offering a
valuable framework and environment for students to develop valuable competencies that can
serve to increase their performance in their work life and their development as responsible
citizens living in a sustainable way”. However, as we defined in the introduction, the aim of this
article to propose not a learning tool, but education quality assurance method based on agile
philosophy.
To determine the trending direction of research, a bibliometric analysis was carried out using
data from the Scopus scientometric database (https://www.scopus.com/) using the VOSViewer
(https://www.vosviewer.com/) and SciVal (https://www.scival.com/home) tools. The main task
of bibliometric analysis is to determine the relationship between agile methodology and various
aspects of the educational sphere. Bibliometric analysis was carried out using query “agile
quality of education”.
Analysis of the relationship between different keywords by specified query (figure 1) high-
lights the areas of application of agile in education. These keywords actually create a set of
indicators for quality assurance in education. Individual clusters within the keyword map
are of particular interest. The “scrum” cluster (figure 2) can be used as the basis to create an
algorithm for assessing educational program quality. The “decision making” cluster (figure 3) is
a prototype for testing the education quality system based on the DIKW model [5] and on the
agile approach at the decision-making stage which in its turn is based on data, information and
knowledge received.
The relevance of the proposed topic in terms of practical implementation is confirmed by
82
Figure 1: Keywords corresponding to the query “agile quality of education”.
the results of bibliometric analysis shown in figure 4. Topic clusters show the main points
of influence of the quality of education including economic factors: learning environment,
educational innovation, information quality, business model innovation , sustainable business
etc.
These data allow to create a testing model with specific outcomes, which can be used to
determine the degree of performance of the system “quality of education”.
As for the analysis of various indicators impact on the system “quality of education”, we refer
to specific literary sources, systematizing them in following way:
– marketing and knowledge management as a basis for modeling the system “quality of
education” [6, 7, 8, 9];
– main stakeholders the influence of the on the educational environment [10, 11, 12];
– ensuring the quality of education in general and personnel training quality [13, 14, 15, 16];
– socio-economic factors of the education influence [17, 18];
– quality of education and sustainable development [19, 20]].
Thus, the bibliometric analysis in general and the analysis of specific literary sources allow
us to define a niche where it is possible to use the agile approach for ensuring the quality of
education in relation to socio-economic indicators
83
Figure 2: Cluster “scrum”.
3. Modelling of university quality education system utilizing
agile methodology
Before we introduce direct agile applications in university quality assurance system, let us briefly
discuss Agile Manifesto in terms of education quality. It can be formed as legacy of Manifesto
for Agile Software Development [21] with corresponding alignment of twelve principles behind
it [22]. Therefore, the Manifesto for Agile Education Quality Assurance can be adapted as
follows:
• Individuals and interactions are valued over processes and tools (same as in software
development);
• Education quality is valued over comprehensive regulatory framework;
• Stakeholder collaboration is valued over requirements discussion;
• Responding to change is valued over following a plan (same as in software development).
Introduced manifesto is based on following twelve principles:
1. Highest priority of education quality assurance is to satisfy all stakeholders through
continuous improvement of learning quality.
84
Figure 3: Cluster “decision making”.
2. Welcome changes in education quality criteria, even at formed systems.
3. Deliver evaluation results frequently, both internally and externally.
4. Frequent collaboration between all stakeholders: students, university management and
employers.
5. Build the system around motivated individuals giving them the environment and support
they need, and trust them to get the job done.
6. Face-to-face conversation is the most effective method of communication within quality
assurance system.
7. Student level of knowledge is the primary measure of education quality.
8. Sustainable development is maintained throughout the whole system functioning.
9. Continuous attention to technical excellence and good design enhances agility
10. Simplicity – the art of maximizing the amount of work not done – is essential
11. Self-organization of the team fosters the best practices for education quality assurance.
12. At regular intervals, the team reflects on how to become more effective, then tunes and
adjusts its behavior accordingly.
Note that some principles outlined for software development at [22] are perfectly applicable
for education quality assurance and hence they are incorporated in unchanged form.
Hereby we propose few insights on how to build education quality assurance system based
on agile principles and applicable for any Ukrainian higher education institution.
85
Figure 4: Top 25% topics by prominence defined by query “agile quality of education”.
3.1. Scrum method for education quality assurance
Scrum is the agile methodology widely used in project management and systems development.
First published in [23] it has since gained significant development. Nowadays scrum is one of
the most preferable agile methodologies with benefits in transparency, risk management and
flexibility.
In this paper we propose scrum method for education quality assurance. This method allows
organizing education quality assurance procedure in comprehensive way. It can be scaled
from overall university performance evaluation (including both educational and scientific
components) down to single educational program assessment on the department level. Figure 1
represents the diagram of proposed scrum method.
The process is comprised of the following steps:
1. Forming of Educational Quality Assurance Backlog. It is closely coupled with internal
or external quality criteria (e.g. National Agency for Education Quality Assurance of
Ukraine (NAQA) educational program quality criteria).
2. Forming Evaluation Sprint Backlog. Basically, Scrum team selects tasks (activities) from
86
Activities backlog formed on previous step to include in the next sprint (e.g. check
academic integrity environment in the university).
3. Performing Evaluation Sprint. Along with direct fulfillment of sprint backlog tasks, team
evaluates total Activities backlog completion taking new tasks for new sprint and/or
including tasks failed on previous sprint (e.g. re-evaluate unclear points explained by
university management).
4. Each spring gets reviewed and based on this retrospective team implements sprint’s
deliverables (e.g. recommendations for educational programs improvement).
Figure 5: Scrum method for education quality assurance.
• Stakeholders provide their vision of education quality since they are highly motivated
in high quality education due to direct economical connection to the graduates landscape.
They also provide initial self-assessment report which is used as the base for activities
backlog forming.
• Evaluating Body such as NAQA mostly contribute to forming of structured quality
assurance backlog providing universal evaluation criteria and independent experts staff
for external quality assurance. However, internal quality assurance department might
also act as evaluating body in case of scaling Scrum method to internal quality assurance
procedure.
• Experts Team perform the evaluation activities during each sprint in transparent and
sharable way. Most of their evaluation activities should be backed by deliverables, e.g.
meeting reports and grades. Scrum Master supervises their work.
87
• Scrum Master maintains the control over the sprint duration and ensures that tasks
from sprint evaluation backlog are fulfilled. Regular Stand-up meetings during the spring
help to keep all the stakeholders and team informed about each other’s activities.
To illustrate the viability of proposed scrum method let us provide examples for external
and internal quality of educational program assurance procedures. Based on existing practices
(NAQA Evaluation procedure and internal assessment of educational program) we define
Educational Quality Assurance Backlog, divide it by sprints and distribute roles according to
positions of engaged staff. Figure 6 should be considered as key to be used with scrum method
given in figure 5.
Figure 6: Scrum backlogs and roles for external and internal quality assurance procedures.
As figure 6 demonstrates, such distribution of roles and formation of Education Quality
Assurance Backlog fully fits into scrum method described by figure 5. We should note that
figure 6 was constructed based on existing evaluation procedures: external evaluation of
educational programs by NAQA and internal evaluation of educational programs within the
university.
Another important note to take here is that after reviewing the Sprint activities the evaluation
outcomes (grades, recommendations to improve some parts of the program, staff evaluation
reports) are than used as deliverables for university. Sprint review procedure is also closely
coupled with stakeholders feedback collection about their satisfaction by economic outcomes
(in terms of graduates competencies) that are provided by educational program and level
88
of innovation that industry gets from implementing educational program. Thus, proposed
methodology connects clear SDG 4 tasks behind the quality assurance system and not so
obvious in this context SDGs 8 and 9.
As can be seen from figure 6, external accreditation of educational program is included in
Sprint 4 of internal evaluation procedure. This may seem counterintuitive on the first glance,
but is backed by logic of educational program development, which is much more complex
project than its accreditation.
3.2. University quality education system aimed at SDG achievement
SDG 4 is a combination of ten targets addressing problems of education quality and equality on
different levels of education. The first two targets (4.1, 4.2) are intended to ensure equal access
of all children to early childhood development and primary education with effective learning
outcomes. This is followed by targets 4.3 and 4.4 stating the need to ensure equal access for all
women and men to technical education and availability of quality employment for all graduates.
Target 4.5 addresses the problems of educational access for vulnerable groups including people
with disabilities, indigenous peoples and children in vulnerable situations. Target 4.6 is aimed
to reduce both youth and adult illiteracy and innumeracy. Target 4.7 raises the need to focus
educational content on the knowledge and skills promoting sustainable development, human
rights, gender equality and culture of peace along with global citizenship. Additional targets
4A, 4B and 4C lay the timeframe and suggest means to fulfill targets 4.1–4.7 [24].
However, the quantitative measurements of the stated targets fulfillment is complicated
despite the indicators formulated by SDG 4. According to [25], “Education quality, equality,
inclusion, gender equality may be unmeasurable with current indictors, but if metrics are useful
to enhance human rights agendas and develop strategies to tackle considerable injustices, then
research and critical discussion is needed concerning what indicators might help develop policy,
practices and accountability to realise the vision of SDG4. A frequent riposte to the complexity
of ideas of quality, equality and inclusion in education is that they are actually unmeasurable.”
Therefore, combining efforts aimed to achievement of the stated targets leads to the idea of
the development of quality education system focused on SDG4 principles and aimed at fulfilling
targets 4.1–4.7.
Figure 8 illustrates multidirectional pipeline focused on achieving SDGs 4, 8 and 9 through the
university engagement. Each of the six stages is carried out in the presented sequence when the
“initial data” for following stages are the results of the previous. Each of the stages is associated
with the fulfilling of different SDG targets. In addition to SDG 4 and SDG 7, which connection
with university activities was covered in [19], the diagram shows other SDGs that can be
achieved during the implementation of the algorithm, in particular SDGs 8 and 9 discussed
above. One can see that a cycle of university activities is organized between the main elements
(nodes). Each stakeholder has the ability to influence the nature of the model implementation
at a certain stage. In this case, stakeholders are initiators of error search and proposing changes
on the stage testing phase (algorithm modernization, conditions for performing a particular
stage, conditions for moving to the next stage, etc.) This fact is demonstrated in the description
of the model testing algorithm. The model assumes that at the beginning of the algorithm
implementation (when developing the technical task for each stage — the regulatory framework
89
of the university), the stages can be carried out in parallel. However, the effective implementation
of the algorithm can be continued only when the result is achieved at the “Quality education
system” stage.
Figure 7: From quality education to innovations and economic growth.
One of the ideas of achieving SDG 4 on level of university is to use DIKW+DM hierarchy. Its
aim is the modification of known DIKW information hierarchy to get multilayered framework
for education quality assurance that allow to join information processing, knowledge acquisition
and corresponding decision-making algorithm.
The agile approach is not recommended for all layers of the DIKW hierarchy. At the stage of
collecting data, analyzing them, systematizing and obtaining an array of information, the data
is not corrected due to their constancy. Only at the stage of knowledge generation as a tool for
subsequent decision-making, it becomes necessary to assess certain factors degree of influence
and revise the system’s functioning model. At the same time, the procedure of internal quality
control by the university and external quality control by independent educational agencies and
external stakeholders is implemented. The report on the monitoring of the quality system is a
guiding document for further revision of system structure, content, interrelation of elements
and the degree of their mutual influence. This part of the model is tested and refined through
an agile approach.
90
Figure 8: DIKW+DM Model incorporating agile approach on the system testing phase.
4. Conclusions
Therefore, in this paper we proposed Manifesto for Agile Education Quality Assurance and
twelve principles behind it based on Manifesto for Agile Software Development. We used
these principles to propose scrum method for education quality assurance. We defined the
overall scrum procedure and distributed roles participating in planning, spring execution and
implementation phases. It was illustrated that Scrum method perfectly fits existing procedure
and staff roles and can be adopted with minimal adjustments by national HEIs.
A consistent approach to the optimal implementation method of quality assurance main
tasks at the university allows solving other problems simultaneously. Achieving SDG 4 targets
all continues the quality assurance journey beyond the educational process. The creation of
91
innovative infrastructures and the achievement of economic growth (thus fostering SDGs 8
and 9 achievement) are important outcomes of an effective model for ensuring the quality of
education. Based on feedback from stakeholders about the degree of their “economic” and
“innovative” satisfaction, subtasks are formed within the framework of internal and external
assurance sprints of education quality assurance procedure.
Acknowledgments
This research was funded by the grant from the Ministry of Education and Science of Ukraine
“Reforming the lifelong learning system in Ukraine for the prevention of the labor emigration: a
coopetition model of institutional partnership” (reg. n. 0120U102001), “Convergence of economic
and educational transformations in the digital society: modeling the impact on regional and
national security” (reg. n. 0121U109553).
References
[1] M. J. D’Souza, P. Rodrigues, Extreme pedagogy: An agile teaching-learning methodology
for engineering education, Indian Journal of Science and Technology 8 (2015) 828–833.
doi:10.17485/ijst/2015/v8i9/53274.
[2] P. Salza, P. Musmarra, F. Ferrucci, Agile methodologies in education: A review, in:
Agile and Lean Concepts for Teaching and Learning, Springer Singapore, 2018, pp. 25–45.
doi:10.1007/978-981-13-2751-3_2.
[3] D. Parsons, K. MacCallum, Agile education, lean learning, in: Agile and Lean Con-
cepts for Teaching and Learning, Springer Singapore, 2018, pp. 3–23. doi:10.1007/
978-981-13-2751-3_1.
[4] A. López-Alcarria, A. Olivares-Vicente, F. Poza-Vilches, A systematic review of the use of
agile methodologies in education to foster sustainability competencies, Sustainability 11
(2019). URL: https://www.mdpi.com/2071-1050/11/10/2915. doi:10.3390/su11102915.
[5] O. Dammann, Data, information, evidence, and knowledge: A proposal for health
informatics and data science, Online Journal of Public Health Informatics 10 (2019).
URL: https://journals.uic.edu/ojs/index.php/ojphi/article/view/9631. doi:10.5210/ojphi.
v10i3.9631.
[6] B. A. Brimah, W. D. Olanipekun, A. G. Bamidele, M. Ibrahim, Knowledge management and
its effects on financial performance: Evidence from Dangote Flour Mills, Ilorin, Financial
Markets, Institutions and Risks 4 (2020) 34–42. doi:10.21272/fmir.4(2).34-42.2020.
[7] H. Yarovenko, Y. Bilan, S. Lyeonov, G. Mentel, Methodology for assessing the risk associated
with information and knowledge loss management, Journal of Business Economics and
Management 22 (2021) 369–387. doi:10.3846/jbem.2021.13925.
[8] K. Onopriienko, V. Onopriienko, Y. Petrushenko, I. Onopriienko, Environmental education
for youth and adults: a bibliometric analysis of research, E3S Web of Conferences 234
(2021) 00002. doi:10.1051/e3sconf/202123400002.
[9] S. Kvitka, G. Starushenko, V. Koval, H. Deforzh, O. Prokopenko, Marketing of ukrainian
higher educational institutions representation based on modeling of webometrics rank-
92
ing, Marketing and Management of Innovations (2019) 60–72. doi:10.21272/mmi.2019.
3-05.
[10] F. Vidic, Entrepreneurial orientation and knowledge creation and their impact on com-
pany performance, SocioEconomic Challenges 2 (2018) 37–48. doi:10.21272/sec.3(2)
.37-48.2018.
[11] I. Kobushko, O. Jula, M. Kolesnyk, Improvement of the mechanism of innovative develop-
ment of small and medium-sized enterprises, SocioEconomic Challenges 1 (2017) 60–67.
doi:10.21272/sec.2017.1-07.
[12] L. Sineviciene, O. Shkarupa, L. Sysoyeva, Socio-economic and political channels for pro-
moting innovation as a basis for increasing the economic security of the state: Comparison
of ukraine and the countries of the european union, SocioEconomic Challenges 2 (2018)
81–93. doi:10.21272/sec.2(2).81-93.2018.
[13] O. Pavlenko, V. Martynets, O. Dreval, D. Smolennikov, Analysis of influence of the quality
of specialist training on social and economic development, Quality - Access to Success 21
(2020) 81–86.
[14] V. Levchenko, A. Boyko, T. Savchenko, V. Bozhenko, Y. Humenna, R. Pilin, State regulation
of the economic security by applying the innovative approach to its assessment, Marketing
and Management of Innovations (2019) 364–372. doi:10.21272/mmi.2019.4-28.
[15] C. G. Cosmulese, V. Grosu, E. Hlaciuc, A. Zhavoronok, The influences of the digital
revolution on the educational system of the EU countries, Marketing and Management of
Innovations (2019). doi:10.21272/mmi.2019.3-18.
[16] I. S. and, Towards the assurance of transparency and quality of higher education in
Ukraine: National qualification framework, Business Ethics and Leadership 2 (2018).
doi:10.21272/bel.2(1).96-105.2018.
[17] S. Lyeonov, O. Liuta, Actual problems of finance teaching in ukraine in the post-crisis
period, in: The Financial Crisis, Springer International Publishing, 2016, pp. 145–152.
doi:10.1007/978-3-319-20588-5_8.
[18] I. Didenko, J. Paucz-Olszewska, S. Lyeonov, A. Ostrowska-Dankiewicz, Z. Ciekanowski,
Social safety and behavioral aspects of populations financial inclusion: A multicountry
analysis, Journal of International Studies 13 (2020) 347–359. doi:10.14254/2071-8330.
2020/13-2/23.
[19] A. Artyukhov, I. Volk, T. Vasylieva, S. Lyeonov, The role of the university in achieving
SDGs 4 and 7: a Ukrainian case, E3S Web of Conferences 250 (2021) 04006. doi:10.1051/
e3sconf/202125004006.
[20] A. Vorontsova, T. Vasylieva, Y. Bilan, G. Ostasz, T. Mayboroda, The influence of state
regulation of education for achieving the sustainable development goals: case study of
Central and Eastern European countries, Administratie si Management Public 34 (2020)
6–26. doi:10.24818/amp/2020.34-01.
[21] Manifesto for Agile Software Development, 2001. URL: https://agilemanifesto.org/.
[22] Principles behind the Agile Manifesto, 2001. URL: https://agilemanifesto.org/principles.
html.
[23] K. Schwaber, SCRUM development process, in: Business Object Design and Implementa-
tion, Springer London, 1997, pp. 117–134. doi:10.1007/978-1-4471-0947-1_11.
[24] United Nations, 4 Ensure inclusive and equitable quality education and promote lifelong
93
learning opportunities for all, 2022. URL: https://sdgs.un.org/goals/goal4.
[25] E. Unterhalter, The many meanings of quality education: Politics of targets and indicators
in SDG4, Global Policy 10 (2019) 39–51. doi:10.1111/1758-5899.12591.
94