Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 Monitoring the effectiveness of the STEAM-oriented environment in general secondary education institutions: approaches to defining criteria Oksana V. Ovcharuk, Nataliia V. Soroko Institute for Digitalisation of Education of the NAES of Ukraine, 9 M. Berlynskoho Str., Kyiv, 04060, Ukraine Abstract The article is devoted to the problems and advantages of monitoring the implementation of STEAM-oriented educational environment in institutions of general secondary education; identifying criteria and indicators for its implementation; analysis of the state of development of STEAM-oriented educational environments in 2024 in Ukraine based on defined criteria. The approaches to monitoring the STEAM-oriented educational environment development for general secondary education are highlighted. The study showed that there is a significant interest of teachers, students, and other participants in the educational process in the implementation and development of a STEAM-oriented educational environment in general education institutions. The study identified gaps in the assessment of the effectiveness of the use of STEM education in Ukrainian schools through a series of surveys and studies, especially in the absence of a single approach to continuous monitoring. Therefore, the authors proposed criteria based on both international approaches and the peculiarities of the Ukrainian education system. The following problems were identified in the Ukrainian educational institutions: funding of STEAM projects; provision of general education institutions with modern laboratories, tools, means of virtual and augmented reality, and special classrooms; maintaining the motivation of teachers and students to participate in STEAM- activities. The article analyses the results of studies by the Ministry of Education and Science of Ukraine as well as own studies carried out in 2022-2023 years. These studies have raised the issue of the effectiveness of the STEAM-oriented educational environment based on the point of view of Ukrainian teachers. The data revealed in the studies enhanced the identification of criteria that can be applied to the assessment of the influence of modern technologies on the educational process and the identification of the key factors contributing to the successful integration of STEM education in general secondary education institutions. The article aims to present the approaches and criteria for monitoring STEAM-oriented educational environments and analyse the challenges faced by Ukrainian secondary educational institutions nowadays. Keywords STEAM-oriented environment, quality monitoring, secondary education, innovative teaching 1. Introduction In the modern educational landscape, STEAM (Science, Technology, Engineering, Arts, and Mathematics) education has emerged as a crucial framework for preparing students for the requirements of the 21st century. Integrating these disciplines into school curricula equips students with a broad skill set, fostering creativity, critical thinking, and problem-solving abilities. STEAM education is important for the education and training of students at school. This education is important for raising well-rounded innovative individuals who can contribute to the rapidly developing world. After all, to obtain a future profession, young people today need quality education, and the ability to adapt to rapid changes in science, technology, engineering, art and mathematics. Therefore, monitoring the development of STEAM education is a key process for determining the quality of education. The development of STEAM education in general secondary education institutions in Ukraine is conducted by Ukrainian legislation, including the laws “On Education” [1], “On Full General Secondary 3L-Person 2024: IX International Workshop on Professional Retraining and Life-Long Learning using ICT: Person-oriented Approach, co-located with the 19th International Conference on ICT in Education, Research, and Industrial Applications (ICTERI 2024) September 23, 2024, Lviv, Ukraine " oks.ovch@hotmail.com (O. V. Ovcharuk); nvsoroko@gmail.com (N. V. Soroko) ~ https://iitlt.gov.ua/structure/departments/komparaktiv/detail.php?ID=63 (O. V. Ovcharuk); https://iitlt.gov.ua/structure/departments/technology/detail.php?ID=64 (N. V. Soroko)  0000-0001-7634-7922 (O. V. Ovcharuk); 0000-0002-9189-6564 (N. V. Soroko) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 78 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 Education” [2], and “On Scientific and Scientific and Technical Activities” [3]. It also aligns with the State standards of primary and basic secondary education. The Government approved the Concept of implementation of state policy in the field of general secondary education reform “New Ukrainian School” for the period until 2029. Implementation of the Concept will create conditions for reforming general secondary education, taking into account the experience of the leading countries of the world. New content standards of general secondary education will be developed, which will be based on a competency-based and person-oriented approach to education as well as the Concept of the Development of Science and Mathematics Education (STEM education). Furthermore, the Ministry of Education and Science of Ukraine has developed and presented a Plan of Measures for the implementation of the Concept for the Development of Science and Mathematics Education (STEM education) until 2027. All these resolutions and regulatory documents serve as the foundation for implementing this educational direction. By implementing the main tasks, the development of STEM education in Ukrainian educational institutions is ensured at the following levels: elementary – stimulation of curiosity and support of interest in learning and the search for knowledge, motivation for independent research, creation of simple devices, constructions, scientific and technical creativity; basic – forming a stable interest in natural and mathematical subjects, mastering technological literacy and problem-solving skills, involvement in research, invention, project activities, which will make it possible to increase the share of those who seek to choose scientific, technical, engineering professions; professional – in- depth mastery of the system of knowledge and skills of STEM education using methods of scientific research, implementation of innovative projects; higher/professional – formation of specialists in various scientific and technical, engineering professions on the basis of institutions of higher education, as well as improving the professional skills of pedagogical workers in the implementation of new teaching methods, relevant courses and implementation of innovative projects. For the development of STEM education, it is important to create an appropriate environment and constant support and evaluation of the effectiveness of functioning [4, 5, 6, 7]. These activities are important for both students and educational institutions to understand the problems and gaps that educators face and need to be addressed. In connection with the large-scale aggression of the Russian Federation against Ukraine in 2022, certain problems have deepened and intensified in general secondary education institutions. In particular, this applies to the digital environment and support, including the STEM environment. The research problem of this study is to find out the problems and gaps accompanying STEAM education in Ukrainian schools based on existing surveys, and also, guided by international approaches, to identify criteria for evaluating the effectiveness of STEAM education in Ukrainian educational institutions. Also, the tasks of the study are to analyse the views of Ukrainian teachers and foreign researchers on the issue of STEAM education, which will help to formulate a vision of monitoring the effectiveness of the STEM environment in Ukrainian schools in the future. 2. Literature review It should be noted that the issue of evaluating the effectiveness of the educational environment is part of the processes related to monitoring the quality of education in general. According to the Decree of the Ministry of Education and Science of Ukraine No. 54 adopted on January 16, 2020, education quality monitoring involves implementing consistent and systematic measures to identify and track trends in educational quality across the country, specific regions, and educational institutions. It includes assessing how well actual educational outcomes align with the stated objectives and identifying the extent, direction, and causes of any deviations from these goals [8]. That is why it is important to identify among scientific studies those that carry out the monitoring of the quality of education, the effectiveness of the STEM education environment and the teachers’ perceptions of the state of such environment. The problem of monitoring the quality of education was studied by Kondrashova [9], Hrynevych et al. [10], Batechko [11], Lukina [12], Denysiuk and Tytarenko [13], Gong and Wei [14], Pérez Torres 79 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 et al. [15]. Lukina [12] proposes to define the concept of ‘monitoring of education” according to the model of the quality of education, which covers the following structural elements. Internal: the quality of educational processes, the quality of professional training and qualifications of teaching staff, the quality of students as individuals, the quality of educational technologies, the quality of resource support (teaching methodical, informational, material-technical), the quality of management technologies and resources in educational institutions and the quality (effectiveness) of education management, the quality of objectives and quality norms in the educational system of education, the quality of organisational structures in the educational system, the professional quality of management personnel in education, the quality of the education monitoring system (the quality of evaluation mechanisms and procedures), the quality of education financing mechanisms. The external quality of education in the gymnasium as a level of training of graduates (future specialists who must implement the tasks set forth by society) [12]. Denysiuk and Tytarenko [13] suggest focusing on specific aspects when evaluating the educational environment. These aspects include the formation of IT thinking among the participants of the educa- tional process, creating an accessible, comfortable, and safe educational environment, and organizing a system of psychological support. The researchers emphasise that the monitoring study is based on several conceptual tasks, such as obtaining information about the state of pedagogical activity in general secondary education institutions within the context of education reform and tracking trends in the organization of educational activities. They also propose using standardized online questionnaires at five stages of the monitoring study to ensure the process’s repeatability and the structure of the results. Kalogiannakis and Papadakis [16] examine opportunities for STEM learning early in life, which is important because developing STEM skills can foster student interest in the future. In particular, Papadakis [17] evaluates the game-development approach to teach introductory programming concepts in secondary education. Papadakis et al. [18] analysing the STEM education experience in young chil- dren’s education has found that it helps children get the appropriate tools crucial to any successful study. They concluded that teachers’ perceptions, attitudes, and technological competencies are considered the primary determinants of technology adoption in curriculum and pedagogy. The important gap that needs to be addressed is the teachers’ lack of relevant knowledge or the lack of relevant provisions for their effective integration of innovative technologies into school curricula [19]. Also, the issues of the teachers’ attitudes toward the implementation of digital technologies and their information literacy are revealed in the works of Ukrainian scholars. Lukianova and Ovcharuk [20] pay attention to the analysis of the information literacy of teachers and vulnerable categories of the population as well as the measures for informational support for elderly people. The authors present the results of the Ukrainian teachers’ self-assessment of the level of digital competence and state that this competence needs to be improved. Soroko [21] focuses on the teachers’ digital competence development as one of the important factors for the establishment and support of the STEAM-oriented learning environment in schools. Gong and Wei [14] emphasise the importance of analysing Innovative Teaching Quality. They focus on identifying exceptional teaching talents through various evaluations and implementing the role model concept to highlight the best teaching practices and effective classroom management conducive to teaching and learning. They introduce three key components for evaluation: multilevel evalua- tion, humanity-related investigation and discussion, and developmental random lecture attendance. Multilevel evaluation involves evaluating various activities of participants in the educational process, such as classroom teaching, students’ learning experiences, and practical education. Humanity-related investigation and discussion utilise questionnaire surveys or discussions to address issues and short- comings in various aspects, such as curriculum development and updating, teachers’ teaching methods, and students’ learning styles. Developmental random lecture attendance includes occasional visits by teachers and management to other teachers’ lectures, providing real-time feedback, exchange of ideas, and technology skill improvement for curriculum development. Having reviewed the works of researchers on monitoring the quality of education, and the impact of STEM education on schooling and the learning environment, this study investigated the state, gaps and problems that exist in Ukrainian secondary education institutions, and based on this, there were 80 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 criteria identified that can be used to determine the effectiveness of the STEM education environment. 3. Research methods To achieve the goal of our research, a review of Ukrainian and foreign research was carried out to identify common features regarding STEM education in educational institutions. The analysis of surveys carried out by the Ministry of Education and Science of Ukraine, as well as other Ukrainian research institutions, made it possible to reveal the attitude of teachers to the use of STEAM laboratories and to find out the extent of the spread of STEAM education in Ukraine. Such an analysis made it possible to outline the criteria for evaluating the effectiveness of the STEM environment in secondary education institutions, which can be used. Own research carried out by the authors in 2022-2023, namely, a survey of teachers, made it possible to identify several problems and gaps both in the technical support of schools and allowed to identify gaps in the readiness of teachers to use digital technologies for the implementation of STEM education. For these surveys, we used quantitative methods of calculating the obtained data. The generalization of the received problems, which were noted by the teachers, made it possible to single out problems common to many institutions in the implementation of STEAM education, which require further solutions at various levels of educational policy. 4. Research results Monitoring a STEAM-oriented educational environment is a complex process that must cover dif- ferent procedures and take into account the specifics of the educational and social environment. A STEAM-oriented educational environment incorporates computer, mobile, and cloud-based technolo- gies, along with practical, interdisciplinary, and project-based learning methods in the study of natural and mathematical disciplines. It aims to encourage students’ creative thinking through the integration of various art forms in the learning process. This approach also enhances the digital and professional competencies of teachers, helping students to acquire essential 21st-century skills. It motivates students to explore scientific problems and effectively integrate knowledge across various subjects at national and international levels. It should be noted that despite the many obstacles faced by teachers in Ukraine due to martial law, teachers actively participate in scientific and practical activities related to the development of STEM education, which is part of STEAM. This experience is reflected in the articles in the anthologies of several conferences devoted to this topic. Thus, the collection of materials “STEM school – 2022” contains 11 developments of practising teachers on various issues, in particular, virtual STEM laboratories for research and experiments in natural and mathematical subjects; modelling and research of physical phenomena using STEM education; involvement of parents in the implementation of the concept of development of science and mathematics education (STEM education); methodological cases of gender-sensitive STEM lessons, etc. After the full-scale invasion of Ukraine, the existing STEM centres began to work with IDP children who do not have the technical ability to get free access to education, as well as to support teachers in the organization of the educational process. This important process supported local communities and contributed to the continuity of the educational process. For example, as early as September 2022, the Educational Initiatives Foundation opened the educational hub “Skills of the Future” based on the STEM centre in Cherkasy with the support of the EU4Dialogue project of the European Union and the United Nations Development Program (UNDP). By the beginning of summer 2022, more than 3.5 thousand teachers from Donetsk, Luhansk, Kherson and other regions were able to improve their qualifications remotely based on such STEM centres. Continuing the scientific search, the authors also turn to sources that reveal the importance of having STEM laboratories in Ukrainian schools. The monitoring study, which was carried out as part of the determination of the readiness of pedagogical workers for the implementation of the New Ukrainian School by the Institute of Educational Analytics, confirmed that the directors of general 81 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 secondary education institutions in the majority (94.5%) indicate the absence of STEM laboratories in the educational institution. Only 277 institutions out of 5,440 (5.1%) participating in the study have such laboratories [22]. To coordinate the work of STEM laboratories in the Institute of Modernization of the Education Content of the Ministry of Education and Science of Ukraine, a separate unit is functioning, the task of which is the implementation of the general strategy for the development and advocacy of STEM education and the organization and conduct of educational events aimed at the popularization of STEM education, career guidance work among students. The term of implementation of STEM programs in Ukrainian educational institutions can be: short-term (from 02 to 24 hours); course (for summer schools, courses, etc.) (from 24 to 80 hours); medium-term (annual) (from 80 to 120 hours); long-term, continuous additional education (from 300 to 600 hours). It should be noted that, despite state support for STEM education, Ukrainian schools face unresolved issues. Many schools lack STEM laboratories due to insufficient funds, as noted by 85.4% of institution directors. Other barriers mentioned include a shortage of qualified specialists (6.8%), methodical support (2.7%), and adequate premises (0.5%) for schools. According to the directors, these factors hinder the establishment of STEM laboratories. Additionally, some respondents indicated that they believe it is unnecessary for their students (2.8%) or that they have no desire (0.2%) to address this issue [13]. Out of the 277 general secondary education institutions that have STEM laboratories, approximately one-third (31.8%) have used or are currently using state funding, while 32.9% have received assistance from their community or city. For the organisation and functioning of the laboratory activities, method- ological support is provided by institutes of postgraduate pedagogical education (15.5%) and methodical offices (12.3%). Postgraduate pedagogical education institutions usually receive support from multiple sources, such as international donors and non-governmental organizations, or both postgraduate edu- cation institutions and students’ parents. In cases where no specific assistance was mentioned, school principals noted that teachers utilised available equipment, and local entrepreneurs provided support. Almost 47.2% of surveyed principals in educational institutions with STEM laboratories report a general lack of necessary equipment, while 31.8% say they have the required equipment [13]. A survey of teachers conducted by the authors of this study indicated that among the main problems pointed out by respondents in 2020 and 2023 are the following: lack of high-quality Internet – 64.7%, insufficient material and technical support for students – 56.8%, fan outages electricity – 49.1%; low level of self-organization and motivation of students – 41.3%; lack of support from parents – 30%; lack of time due to increased workload on the teacher – 25.9%; insufficient level of material and technical support of educational institutions – 23.1%; psychological difficulties during distance learning – 14.5%; difficulties with distance learning of elementary school students – 13.9%; decrease in the quality of providing educational services – 6.3%; insufficient level of digital competence of the teacher – 5.1%; the institution was destroyed or damaged during the war – 4.4% [20]. All these data indicate the existence of a number of problems with the functioning of the STEM environment in Ukrainian schools among which, for example, non-systematic use of STEM laboratories, insufficient equipment, lack of access, insufficient training of teachers and low level of digital competence and information literacy of teachers, etc. Among the monitoring tools of the STEAM-oriented environment, the results of international comparative studies of the quality of education can be used, For example, the International Comparative Study of the Quality of Natural and Mathematical Education (Trends in International Mathematics and Science Study, TIMSS), the International Study of the Quality of Education (PISA), etc. In particular, the Programme for International Student Assessment (PISA) was initiated over 23 years ago by the Organisation for Economic Cooperation and Development (OECD) [23]. This study is one of the important sources of information about secondary education in the world. More than 80 countries participate in the study. Ukraine participated in the last two cycles of PISA – 2018 and 2022. Due to the full-scale aggression of Russia against Ukraine and related security risks in the 2022 cycle, 15-year-olds from only 18 regions of Ukraine were able to participate in the study [23]. The results of the study demonstrated a significant decline in academic performance and quality of education, as can be seen from the chart, especially in the average performance in mathematics, reading and science in Ukraine and OECD countries between the 2018 and 2022 PISA cycles [23]. 82 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 It should be noted that the elaborated volume of the presented research allowed us to single out the criteria that are used to evaluate the effectiveness of the educational environment. In particular, Laniado et al. [24] propose a model to evaluate the effectiveness of environmental projects linking it to the academic success of students. Academic achievement is the amount of educational material that a student studies in a certain period. This can be any way in which the student has achieved short-term or long-term academic goals in the learning environment. Yang et al. [25] propose to assess the student’s educational achievements, testing and evaluation are conducted. We agree with García-Senín et al. [26] who propose to using learning analytics to support STEAM students’ academic achievement and self-regulated learning. El Mhouti et al. [27] suggest creating a tool to assess the quality of digital learning resources. They discuss the criteria for evaluating these resources, specific to four dimensions: academic quality, pedagogical quality, didactic quality, and technical quality. These criteria aim to guide the design of easy-to-use evaluation instruments. The same position is taken by the Ukrainian State Center for the Quality of Education, which annually surveys students and teachers regarding the learning environment and readiness to use ICT. Based on the mentioned studies, it is worth highlighting such a criterion as the resources of the educational environment. Indeed, an important criterion is the resources involved and available in the educational environment. Resources can be divided into software products, facilities and tools, as well as educational (teaching) and human (teachers, management staff, administrators). Another important criterion should be the availability of innovative training and educational methods. Teacher training is critical because teachers need to be provided with regular learning opportunities to stay on top of innovation. STEM teacher training workshops should be aimed at creating an opportunity for secondary STEAM teachers to explore Science, Technology, Art, Engineering and Math (STEAM) as well as Computer Science training through technology-rich demonstrations, classroom presentations, hands-on activities and online connections. The next criterion is the degree of government attention to the problem of organisation and function- ing of the STEAM environment. This criterion is related to educational policy and the needs of education and society in STEAM competencies of students.It should be noted that the issue of the implementation of STEM education is declared in Ukrainian regulatory documents, partly also in schools where STEM laboratories are established. For example, the Institute for the Modernization of the Content of Education has a specialized department of STEM education, which has been experimenting in this area for many years. However, as a result of crisis situations in the state, in connection with the state of war, the wide distribution of STEM laboratories did not take place, and all this requires further support from the state. After considering the aforementioned points, we can outline the following criteria for monitoring a STEAM-oriented educational environment: academic achievement (success in academics), resources and educational environment for the implementation of STEAM education, innovative teaching and learning methods, and government attention to the issue (table 1). Analyzing the criterion “academic achievements (academic success)” for teachers, the authors of the article used information obtained during conferences, seminars, training, webinars, received applications and web resources of conferences, seminars, and training. The main interest of the participants is in the study of new digital tools and means for online education of students, in particular, the use of AI, augmented reality, the creation of virtual laboratories and their effective use in lessons and the STEAM environment. Today, teachers pay great attention to the safety and security of students and themselves in the environment and the opportunity to learn safe tools for learning. 5. Conclusions The issue of implementation and development of a STEAM-oriented environment is important today for Ukrainian education. The conducted research showed that today obtaining a quality education is a challenge and a problem since a full-scale invasion is ongoing in the country. STEAM education technologies make it possible to raise the educational process to another level and ensure accessibility and continuity of education, creating opportunities to acquire the necessary skills for citizens of the 83 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 Table 1 The criteria and indicators for monitoring the development of STEAM education in correlation with the require- ments of teachers and students. Criteria Indicators for teacher Indicators for student Academic The percentage of teachers who completed courses and Level of knowledge of STEAM sub- achievements/ received certificates of successful completion of STEAM jects; Results of standardized tests academic suc- education; The percentage of teachers who participate and exams. Grades for labora- cess in scientific and practical events (conferences, seminars, tory work and practical tasks. Per- training, etc.) that have thematic areas related to STEAM centage of students receiving high education (for example, modernisation of the content marks in scientific Olympiads and of education in secondary and of higher education in competitions. conditions of digital transformation, features of STEAM implementation in the educational process of secondary and higher education; digital education, etc.); Analysis of articles, theses and reports by topic regarding STEAM education. Resources and Number of websites dedicated to STEAM education; The Level of satisfaction of students educational number of methodological recommendations for the im- with material and technical sup- environment plementation of STEAM projects; The level of classroom port. Level of providing students for the im- equipment for conducting laboratory work, remote teach- with equipment and educational plementation ing; Level of satisfaction of teachers with material and materials of STEAM technical support. Number and condition of equipment education (computers, laboratories, 3D printers, VR/AR equipment). Innovative Frequency of using interactive platforms for conducting The percentage of students in- teaching and lessons (eg, Kahoot!, Quizlet, Nearpod); Percentage of volved in working in teams for learning meth- lessons conducted using interactive whiteboards or other project implementation ods interactive learning tools; The number of projects that in- clude an interdisciplinary approach and involve different subjects; the percentage of projects and courses using new technologies such as of AI, AR/VR etc. State attention Availability of orders and other documents regarding Number of Olympiads and STEAM to the problem the implementation of STEM education in educational education competitions organised institutions; The level of funding for projects related to for students; Number of partici- STEM education pants in Olympiads and competi- tions; Analysis of students’ satis- faction with participation in events 21st century. Turning to the issues related to the support of the STEAM environment in secondary education institutions, it was found that today there are issues that should attract the attention of educators, in particular, the insufficient level of information literacy and digital skills. This is also evidenced by research conducted by both Ukrainian and foreign researchers [19, 20]. In addition, the need for continuous professional development of teachers who implement STEAM education and its elements is evidenced by research and surveys conducted by Papadakis et al. [18], Kalogiannakis and Papadakis [16]. As a result of the analysis and approaches used in other countries, the STEAM environment is a remarkable achievement, and its effectiveness needs monitoring. Based on the study conducted and the experience of supporting such an environment, criteria for the effectiveness of the STEAM environment were proposed, which can serve as a starting point for the evaluation system of the educational environment and STEAM education. The main emphasis was placed on the criteria by which the presence and needs of such categories as teachers and students should be considered, ensuring the functioning of the environment. We consider it expedient to pay attention to such urgent problems of the lack of STEM laboratories and the environment, which single out teachers, namely: lack of funds, lack of appropriate specialists, lack of familiarity with equipment, and lack and insufficiency of methodical support for teachers. These and other circumstances not only reduce the motivation of teachers to develop STEM and STEAM education and create the necessary environment at school but also cancel the opportunity for students to develop, 84 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 and acquire the necessary skills and competencies for life and future work in a digital, information society. The authors analysed the reports of state institutions and identified four important criteria that can be used to monitor the effectiveness of a STEAM-oriented environment, namely: academic achievement (success in academics), resources and educational environment for the implementation of STEAM education, innovative teaching and learning methods, and state (government) attention to the issue. These criteria and related indicators of the effectiveness of the STEAM environment take into account the interests of teachers and students. It should be noted that our research is an attempt to systematize teachers’ attitudes toward such components of the environment as its software, content, and compliance with the interests of students. Therefore, the best practices of a STEAM-oriented educational environment, the role of such an environment in overcoming educational losses, and a method of evaluating the effectiveness of a STEAM-oriented educational environment taking into account the specified criteria can be perspectives for further scientific research. Based on the presented study results several recommendations can be made to evaluate the STEAM- oriented environment in secondary education institutions: state institutions and organisations should develop plans to support the STEAM environment in schools to support distance learning of students and education in general, especially in conditions of full-scale aggression by the Russian Federation; teacher training programs can be guided by developed criteria for teaching teachers to use STEAM tools in the classroom; educational administrations can conduct surveys of teachers and students, taking into account the proposed criteria, expanding them with other indicators to determine the state of use and effectiveness of STEAM environments and STEAM education in their regions. The problems common to many institutions in the implementation of STEM education, which require further solutions at various levels of educational policy, should include insufficient funding and, as a result, insufficient equipment of STEM laboratories, unpreparedness of personnel and low level of information literacy of teachers, lack of systematic monitoring of the effectiveness of STEAM -oriented environment. In the future, we see the expansion of tools for monitoring the quality of education due to the addition of developed criteria to the study of the state of use of new teaching aids. Studying the effectiveness of the STEAM environment for continuous education, starting from elementary grades, should become an important area of both research and practical experiments, as well as the exchange of experience from advanced educational practices of STEAM education. References [1] Verkhovna Rada of Ukraine, On education, 2017. URL: https://zakon.rada.gov.ua/laws/show/ 2145-19/page?lang=en. [2] Verkhovna Rada of Ukraine, On general secondary education, 2020. URL: https://zakon.rada.gov. ua/laws/show/651-14?lang=en. [3] Verkhovna Rada of Ukraine, On scientific and scientific-technical activities, 2015. URL: https: //zakon.rada.gov.ua/laws/show/848-19?lang=en. [4] S. O. Semerikov, M. M. Mintii, I. S. Mintii, Review of the course “Development of Virtual and Augmented Reality Software” for STEM teachers: implementation results and improvement potentials, in: S. H. Lytvynova, S. O. Semerikov (Eds.), Proceedings of the 4th International Workshop on Augmented Reality in Education (AREdu 2021), Kryvyi Rih, Ukraine, May 11, 2021, volume 2898 of CEUR Workshop Proceedings, CEUR-WS.org, 2021, pp. 159–177. URL: https: //ceur-ws.org/Vol-2898/paper09.pdf. [5] R. P. Kukharchuk, T. A. Vakaliuk, O. V. Zaika, A. V. Riabko, M. G. Medvediev, Implementation of STEM learning technology in the process of calibrating an NTC thermistor and developing an electronic thermometer based on it, in: S. Papadakis (Ed.), Joint Proceedings of the 10th Illia O. Teplytskyi Workshop on Computer Simulation in Education, and Workshop on Cloud- 85 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 based Smart Technologies for Open Education (CoSinEi and CSTOE 2022) co-located with ACNS Conference on Cloud and Immersive Technologies in Education (CITEd 2022), Kyiv, Ukraine, December 22, 2022, volume 3358 of CEUR Workshop Proceedings, CEUR-WS.org, 2022, pp. 39–52. URL: https://ceur-ws.org/Vol-3358/paper25.pdf. [6] M. M. Mintii, N. M. Sharmanova, A. O. Mankuta, O. S. Palchevska, S. O. Semerikov, Selection of pedagogical conditions for training STEM teachers to use augmented reality technologies in their work, Journal of Physics: Conference Series 2611 (2023) 012022. doi:10.1088/1742-6596/ 2611/1/012022. [7] M. M. Mintii, STEM education and personnel training: Systematic review, Journal of Physics: Conference Series 2611 (2023) 012025. doi:10.1088/1742-6596/2611/1/012025. [8] Ministry of Education and Science of Ukraine, About the order to conduct education quality monitoring, 2020. URL: https://zakon.rada.gov.ua/laws/show/z0154-20#Text. [9] L. V. Kondrashova, Monitoring of educational processes in higher school, in: 2005 15th International Crimean Conference Microwave & Telecommunication Technology, volume 1, 2005, pp. 129–131 Vol. 1. doi:10.1109/CRMICO.2005.1564838. [10] L. Hrynevych, O. Linnik, J. Herczyński, The new Ukrainian school reform: Achievements, devel- opments and challenges, European Journal of Education 58 (2023) 542–560. doi:10.1111/ejed. 12583. [11] N. Batechko, Quality management in higher education sphere: Methodological aspect, Osvitolohiya (2017) 156–162. doi:10.28925/2226-3012.2017.6.156162. [12] T. Lukina, Change of the model of state final attestation of gymnasium graduates: Organiza- tional and methodological aspects, Ukrainian Educational Journal (2020) 42–50. doi:10.32405/ 2411-1317-2020-2-42-50. [13] O. Denysiuk, N. Tytarenko, Indicators of the New Ukrainian School as a tool for measuring education reform implementation by general education teachers, Educational Analytics of Ukraine (2023) 91–103. URL: https://science.iea.gov.ua/en/2023_4en_91-103/. [14] L. Gong, C. Y. Wei, Promoting Innovative Teaching Quality Monitoring (TQM) System: A Case Study at University of Sanya, Hainan, China, in: Proceedings of the 10th International Conference on Business, Accounting, Finance and Economics (BAFE 2022), Atlantis Press, 2023, pp. 401–413. doi:10.2991/978-2-494069-99-2_30. [15] M. Pérez Torres, D. Couso Lagarón, C. Marquez Bargalló, Evaluation of STEAM Project-Based Learning (STEAM PBL) Instructional Designs from the STEM Practices Perspective, Education Sciences 14 (2024) 53. doi:10.3390/educsci14010053. [16] M. Kalogiannakis, S. Papadakis, The Use of Developmentally Mobile Applications for Preparing Pre-Service Teachers to Promote STEM Activities in Preschool Classrooms, in: S. Papadakis, M. Kalogiannakis (Eds.), Mobile Learning Applications in Early Childhood Education, 2020, p. 82–100. doi:10.4018/978-1-7998-1486-3.ch005. [17] S. Papadakis, Evaluating a game-development approach to teach introductory programming concepts in Secondary Education, International Journal of Technology Enhanced Learning 1 (2020) 1. doi:10.1504/ijtel.2020.10024287. [18] S. Papadakis, J. Vaiopoulou, E. Sifaki, D. Stamovlasis, M. Kalogiannakis, K. Vassilakis, Factors That Hinder in-Service Teachers from Incorporating Educational Robotics into Their Daily or Future Teaching Practice, in: Proceedings of the 13th International Conference on Computer Supported Education - Volume 2: CSEDU, SciTePress, 2021, pp. 55–63. doi:10.5220/0010413900550063. [19] E. Tzagkaraki, S. Papadakis, M. Kalogiannakis, Exploring the Use of Educational Robotics in Primary School and Its Possible Place in the Curricula, in: M. Malvezzi, D. Alimisis, M. Moro (Eds.), Education in & with Robotics to Foster 21st-Century Skills, volume 982 of Studies in Computational Intelligence, Springer International Publishing, Cham, 2021, pp. 216–229. doi:10. 1007/978-3-030-77022-8_19. [20] L. Lukianova, O. Ovcharuk, Information Literacy and Digital Inclusion: Challenges of the Modern Information Educational Environment in Ukraine, in: Ł. Tomczyk, F. D. Guillén-Gámez, J. Ruiz- Palmero, A. Habibi (Eds.), From Digital Divide to Digital Inclusion: Challenges, Perspectives and 86 Oksana V. Ovcharuk et al. CEUR Workshop Proceedings 78–87 Trends in the Development of Digital Competences, Springer Nature Singapore, Singapore, 2023, pp. 541–565. doi:10.1007/978-981-99-7645-4_25. [21] N. Soroko, Methodology for Teachers’ Digital Competence Developing through the Use of the STEAM-oriented Learning Environment, in: O. Sokolov, G. Zholtkevych, V. Yakovyna, Y. Tarasich, V. Kharchenko, V. Kobets, O. Burov, S. Semerikov, H. Kravtsov (Eds.), Proceedings of the 16th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer. Volume II: Workshops, Kharkiv, Ukraine, October 06- 10, 2020, volume 2732 of CEUR Workshop Proceedings, CEUR-WS.org, 2020, pp. 1260–1271. URL: https://ceur-ws.org/Vol-2732/20201260.pdf. [22] Information and analytical materials “Study on the implementation of the NUS reform in pilot educational institutions (basic school)”, 2023. URL: https://iea.gov.ua/wp-content/uploads/2023/ 10/pilot_nus_report_2023.pdf. [23] Ukrainian Center for Evaluation of the Quality of Education, PISA 2022, 2024. URL: https:// testportal.gov.ua/tag/pisa-2022/. [24] E. Laniado, S. Vaghi, E. Amodeo, A. Capriolo, R. Mascolo, A model to evaluate the effectiveness of environmental projects, in: The Sustainable City VIII, SC13, WIT Press, 2013, pp. 755–765. doi:10.2495/sc130642. [25] J. Yang, K. Qin, Y. Wang, Effect of the Duration of Preschool Attendance on Academic Achieve- ments—Evidence from PISA 2018, Child Indicators Research 17 (2023) 1–22. doi:10.1007/ s12187-023-10095-1. [26] S. García-Senín, M. Arguedas, T. Daradoumis, Using learning analytics to support STEAM students’ academic achievement and self-regulated learning, Research on Education and Media 14 (2022) 36–45. doi:10.2478/rem-2022-0005. [27] A. El Mhouti, A. Nasseh, M. Erradi, How to evaluate the quality of digital learning resources?, International Journal of Computer Science Research and Application 03 (2013) 27–36. URL: https: //www.researchgate.net/publication/260392089. 87