The digital transformation of general education is seen as one of the stages of the education renewal process in a rapidly evolving digital environment (EREDE). Practice-oriented models of this process are used to monitor its development and help schools to innovate. To assess the parameters of such models, quantitative estimates are used that help to fix the ongoing changes, but do not give a meaningful assessment of the current state of transformation processes. This prevents the development of automated tools to inform schools about the maturity of the ongoing processes, helping to identify the causes of success and failure, to outline the steps to update their work. An approach is proposed for automating the transformation of quantitative estimates that characterize the maturity of transformation processes into qualitative ones. The experience of using it to construct a methodology for assessing the use of innovative digital-supported (ICT-supported) methods of educational work is discussed. The methodology combines the use of the results of surveys of participants in the educational process and the interpretation of innovative changes provided by prescriptive analysts. The proposed approach can be used for meaningful interpretation of questionnaire data on the use of ICT in educational institutions, collected during monitoring surveys, and for preparing targeted recommendations for school workers on this basis. 1 Supported by the Russian Science Foundation, grant No. 19-29-14167.
Thinking about what digital transformation means, it can be considered as the stage of
educational renewal in the evolving digital environment (EREDE), which has been
continuing for several decades. This process has been initiated by the transition from
analog ("paper") to digital processing of all types of information. This transition is
accompanied by ongoing changes in the economy, culture, public life, science, and
technology. The transition to the use of digital technologies promises to solve the
eternal problem of the typical school, providing everyone with equal access to
highquality education, improving educational performance, and ensuring the development
of each student. Without this, it is impossible to ensure sustainable socio-economic
development in a rapidly changing world, and cope with global threats to humanity
[
Within the EREDE, there is a systemic change in the educational process's key
components: goals, content, methods, and forms of teaching and learning, which
should ensure learning outcomes for each student [
There is a substantial need for automated tools for changes monitoring and targeted
support of schools. However, it is still hindered by insufficient methods and data
infrastructure. Usually, statistical reporting data from schools is used for
parameterization, as well as the results of surveys (questionnaires) of school leaders, teachers,
students, and their parents. These models can be used as a monitoring tool to obtain
integrated assessments of education renewal processes at various levels, and / or to
provide schools with recommendations to support them in the implementation of
appropriate changes. An analysis of these models [
A methodology is required that allows us to move from a numerical representation of changes to meaningful (qualitative) indicators of digital transformation processes. Such a solution can be treated as a tool for data collection, and processing simplification, and school targeted recommendations preparation. Focusing on the maturity of the changes being made, it is possible to identify the key successes and failures associated with changes, and outline the steps taken.
The paper describes an approach that allows automation of numerical (quantitative) representation of changes, into maturity indicators of the school digital transformation. The methodology is illustrated by an example of innovative, digitally supported (ICT-supported), teaching and learning methods assessment. The methodology combines the use of the survey results (quantitative tools) and the interpretation of innovative changes that prescriptive tools provide.
The first part of the article shows why the digital transformation of education can be interpreted as the final stage in the renewal of education in the evolving digital environment. In the second part, a solution is considered that allows to move from quantitative (numerical) assessments, to meaningful (qualitative) indicators of changes in digital transformation processes. The third part discusses the use of this approach, to build a methodology for assessing the use of innovative technologyenhanced learning methods of educational work.
Digital transformation is often unpacked as changes in the educational
environment. The learning environment includes both visible and the invisible parts: people,
technology, physical classroom and its virtual layout, objects within the classroom,
books, notes, websites, software, school buildings, and “the social and cultural
environment” [4, p.10]. However, the learning environment is directly related to the
instructions and methods of teaching and learning. The beliefs and experience of
teachers and students influence the way how they interact with the learning environment
[
• basic computer literacy development, and equipping of schools with computers i.e. computerization of education (1st stage);
• use of ICT across various school disciplines - named, early informatization of education (2nd stage);
• integration of ICT into teaching and learning - named, mature informatization of education (3rd stage).
Transformations observed in innovative schools were shown to be reasonable
enough to introduce a 4th stage, which is called the digital transformation of
education [
The past decades have shown that the development of digital capabilities occurs
unevenly in schools. Schools are moving along this path at a different pace, which
complicates the periodization, when the renewal of education unfolds at a variety of
educational organizations (municipality, region, country). For practical purposes, it
can be considered that the transition from one stage to another has begun, if at least
10% of schools (innovators according to Rogers) have reached the next stage [
At each stage, digital technologies are being updated, the educational environment is developing, and key tasks are changing: (1) equipping schools with ICT to ensure computer literacy; (2) the use of computers in the study of certain school subjects; (3) introduction of computers and the Internet into the educational process; (4) the transition to personalized learning.
● Computerization of education is the first stage of EREDE. The main focus of it is on equipping schools with computers. Today, only a few schools in Russia are still at this stage of EREDE.
● Early informatization is the second stage of EREDE, during which, schools solve the problem of using computers in the study of certain subjects. The focus here is on the use of digital teaching materials (interactive multimedia content for teaching). Today, many Russian schools are at this stage.
● Mature informatization of education is the third stage of EREDE. The distinctive feature of this stage is the dissemination of innovative teaching methods and forms of teaching, which are supported by digital technologies (broadband access to all participants of the educational process, distribution of cloud services, etc.). Today, many Russian schools, especially in large cities, are entering this stage.
● The digital transformation of education is the upcoming fourth stage in
school development, which is now being entered by individual educational
organizations [
At the stage of digital transformation, all participants in the educational process are moving to the daily use of digital technologies, just as they used to use "paper" technologies previously. Therefore, we can say that this stage completes the renewal of the general education system in the developing digital environment. Of course, this does not mean that the development of the education system will stop. It will simply take different forms.
Today, Russian schools are situated at different stages of EREDE. It must be taken
into account before any digital intervention is carried out. Monitoring of this process
should include indicators that will describe and help to prepare recommendations for
schools at different stages of EREDE.
3 From a quantitative assessment of digital transformation
toward qualitative indicators
In the early stages of ICT in education, computers affected the change in educational
work (mainly, improvement and modification levels, according to SAMR). At the
stage of digital transformation, they transform it significantly. To assess these
changes, classic survey methods of data collection are used (questionnaires, interviews,
etc.). Results are usually converted into quantitative (numerical) representation using
inferential statistics. It allows us to record the ad-hoc new ways of educational work,
but does not help to provide a current assessment of the transformation processes. An
illustrative example is the recently developed service, SELFIE [
The combined methodology is based on the K-model clustering approach [
The main practical aim of the combined methodology is to correspond descriptive data on the use of innovative ICT-supported teaching methods, presented as a set of responses to the survey, and qualitative judgments that are set in the level models of the use of ICT in school.
There are four main steps in the proposed hybrid method: 1. To conduct a survey in order to collect the data for further analysis. This methodology can use survey Likert data collected from at least two cohorts of educational process participants, wherein a survey contains an item on innovative ICTsupported teaching. It is assumed that a five-point Likert scale is used, and each group contains at least five respondents for each school.
2. To determine characteristics. For each school, a group of characteristics is calculated that reflects that a school is working consistently and constantly (according to teachers and school leaders) on the use of innovative ICT-supported teaching. The obtained characteristics are a set of variables: (1) – how coordinated are the efforts of school leaders to use innovative ICT-supported teaching at the school level;
– how consistent are the efforts of school leaders to use innovative ICT-supported teaching at the school level;
– how coordinated are the efforts of teachers to use innovative ICTsupported teaching at the school level;
– how consistent are the efforts of teachers to use innovative ICTsupported teaching at the school level.
To determine the value of consistency variables, the proportions of positive answers (“agree” and “absolutely agree”) are calculated in each group of school respondents. If at least 60% of respondents say that a school regularly works on the use of ICT-supported innovative teaching practices, the value of both variables is fixed as “high”. Otherwise, it is fixed as "low".
When determining the value of the coordination variables , we calculate a
standardized characteristic - a measure of variability. This assessment is widely used in
research of decision-making processes, and the corresponding computational methods
are constantly being improved [
3. To cluster schools. We use the k-mode clustering algorithm. When choosing the number of clusters, we use qualitative descriptions of the resulting clusters after calculating the optimal number of clusters (using the elbow method). Here, we evaluate clusters in terms of significant differences in the use of innovative ICT-supported teaching at the school level. If necessary, the number of clusters can be changed.
4. To correspond the cluster structure against maturity levels. Here, we extend
the resulting cluster structure to the maturity scale of the innovation process by expert
evaluation. We use the six-level scale of the innovation process maturity model
adopted in the RISC community [
The scope and scale of ICT-supported innovative ways of teaching in individual schools differ markedly [20; 21]. It is usually understood as the presence of any innovative ICT-supported methods of educational work in pedagogical practice, regardless of its scale and specification. The determining factor of such a change in teachers work is the development of change management in the school, which supports the emergence and testing of new methods of work by teachers.
The combined technique was developed and tested when processing the results of SELFIE piloting in Russian schools. SELFIE (Self-reflection tool for digitally capable schools) is an online service that allows schools to assess the extent to which digital systems are used in the educational process [10; 18].
The survey takes place on the online platform. The survey involves three groups of respondents:
- school leaders. These include all employees performing any managerial functions (directors and deputies, heads of departments, etc.)
- teachers, which include all members of the teaching staff who do not have managerial functionality; - students.
As a result of the survey, the school receives an automatically generated report with the results of an anonymous survey of leaders, teachers and students, which the teaching staff can use to improve their work.
In this study, the teachers' (N=685) and school leaders' (N=206) questionnaires
were drawn from the SELFIE tool dataset in primary, secondary, lower secondary
schools. SELFIE tool is developed by the European Commission Joint Research
Centre(Self-reflection tool for digitally capable schools (SELFIE) - European
Commission, no date). In 2017 it was piloted in more than 600 schools from 14 European
countries. Based on DigCompOrg [
Schools for the study were chosen from IITE and UNESCO Associated Schools Project Network (UNESCO ASPnet) - "Learning for the Future" within the scope of the SELFIE (Self-reflection tool for digitally capable schools) project pilot where Russia took part in October 2017. We used two statements (Table 1) to which survey participants gave answers on the Likert scale, consisting of 5 positions
School leaders statement It is a part of our digital strategy, that Teachers use digital technologies innovatively to try out new ways of teaching
As a result (coordination of efforts of leaders in using innovative ICTsupported new ways of teaching) it was found that in 14 schools (60%) the coordination of efforts is low, and in 9 schools – is high.
The assessment of showed that in 21 schools (91%) the coherence in the teachers' answers was low. The fact that school leaders are more likely to show consistency in their responses can be interpreted as a fact that school leaders are more likely to feel that their schools are regularly working on new ways of teaching, although in fact, not all schools teachers use innovative ICT-supported ways of teaching on a regular basis.
The assessment of showed that in 9 schools (40%) it is low, and in 14 schools, it is high. An assessment of the showed that the consistency of efforts of teachers in 11 schools (48%) is low. This data suggests that leaders are more likely to agree that they are making more efforts to use innovative, CTsupported learning styles than teachers.
As a result of clustering, schools were distributed into 9 clusters (Table 2).
learning. This can be expressed by plans development, regulations and supportive measures. Some teachers also use innovative ICTsupported teaching. However, while only few of teachers in the school use innovative ICT-supported teaching, teachers mostly use traditional teaching preferably.
ICT-supported teaching and learning. This can be expressed by the development of plans, policies, regulations for its implementation. School teachers use innovative ICT-supported teaching methods regularly.
begun to do something toward the use of innovative ICTsupported teaching and learning, but it goes neither systematically nor consistently. Most teachers of the school use traditional ways of teaching.
innovative ICT-supported teaching and learning, but the planning and organization of the use of innovative ICT-supported teaching is not yet systematic. There are individual teachers who consistently use innovative ICT-supported teaching methods.
the use of innovative ICT-supported teaching. Most teachers of the school use traditional ways of teaching.
Clusters were corresponded over the innovation process maturity scale, which was
used in the RISC community [
Results in the table 3 suggest cluster distribution over the innovative process maturity scale.
There is already a “core” of active teachers and / or leaders who use innovative ICT-supported teaching methods. Their influence is growing. The school already has sustainable forms of innovative ICT-supported teaching.
evaluate work on innovative ICT-supported teaching methods support. The school has revised these procedures at least two or more times in accordance with the established schedule (production cycles, academic years). Dissemi nation
The school has demonstrated the ability and experience to assist other schools in defining their policies (practices and procedures) in the use of innovative ICT-supported teaching and learning. The school has to share its best practices and procedures with other schools.
The application of the combined methodology to the SELFIE data showed that there has already been at least an awareness of the need to use innovative ICTsupported teaching in sample schools. However, the way of this work can vary.
There are no schools in the sample where such work is not carried out. All schools are at least introducing innovative ICT-supported teaching. In most schools (level "Awareness" on the scale), the corresponding processes are rather ad-hoc. Here, the use of innovative ICT-supported teaching methods is an initiative of individual teachers and / /or school leaders. There is a group of schools (the level of "Understanding" on the scale) where the use of innovative ICT-supported teaching is shaped into systematic work. We can say that stable forms of innovative ICT-supported teaching have appeared in these schools.
In two schools (level "Start of implementation" on the scale), there is a process of implementation and use of innovative ICT-supported teaching. Here, the use of innovative ICT-supported teaching methods is carried out unceasingly. Most of the teaching staff is involved in this process. We can say about these schools that there is an integrated support system for the use of innovative ICT-supported teaching methods, and teachers regularly use them in their daily work.
All schools that participated in the SELFIE piloting are i of UNESCO Associated
Schools. They are recognized as innovative schools. In each of them, the digital
transformation is already underway. At the same time as follows from the data we
obtained, none of them reached the “Use” and “Improvement” levels on the scale. An
analysis of the school work [
Discussion Since 1959, ICT’s have begun to increasingly penetrate school education in Russia, and it continues today. We can describe it as the renewal of education in the evolving digital environment. With some ups and downs, distinguishing several stages in it, each of the stages (computerization, early and mature informatization of education) differs in the kinds of tasks. The stage that has begun most recently is the digital transformation of education. Schools are moving along the path of updating education in the evolving digital environment along different trajectories and at different rates. The existing models for describing this process are based presumably on quantitative assessments of its indicators, which makes it challenging to prepare meaningful conclusions about the maturity of the ongoing processes, followed by targeted recommendations for individual schools.
The proposed approach to the transformation of quantitative assessments into qualitative ones, which characterize the maturity of innovative processes, is based on the composition of characteristics, estimated from survey data. Then, using cluster analysis, groups of schools are identified and mapped to the selected maturity levels.
Based on this technique, we developed a combined methodology that overcomes the limitations of existing approaches to assess innovative ICT-supported teaching and learning based on quantitative data. The methodology makes it possible to automate the interpretation of survey results and prepare, on this basis, targeted recommendations to schools for their work improvement.
The methodology application to SELFIE data resulted in structured descriptions of innovative ICT-supported teaching work. Such descriptions can be used by school teams and educational management, for an impact assessment of certain projects and activities on actual changes in the use of innovative ICT-supported teaching methods by schools.
The combined technique is quite universal. However, with a change in the type of educational organizations, the use of new survey instruments, the addition of new groups of participants in the educational process, and the use of a different scale of the maturity model, it will require appropriate adjustment and refinement.