=Paper=
{{Paper
|id=Vol-2393/paper_237
|storemode=property
|title=Design of Approaches to the Development of Teacher’s Digital Competencies in the Process of Their Lifelong Learning
|pdfUrl=https://ceur-ws.org/Vol-2393/paper_237.pdf
|volume=Vol-2393
|authors=Nadiia Balyk,Yaroslav Vasylenko,Galina Shmyger,Vasyl Oleksiuk,Anna Skaskiv
|dblpUrl=https://dblp.org/rec/conf/icteri/BalykVSOS19
}}
==Design of Approaches to the Development of Teacher’s Digital Competencies in the Process of Their Lifelong Learning==
https://ceur-ws.org/Vol-2393/paper_237.pdf
Design of Approaches to the Development of Teacher’s
Digital Competencies in the Process of Their Lifelong
Learning
Nadiia Balyk [0000-0002-3121-7005], Yaroslav Vasylenko [0000-0002-2954-9692],
Galina Shmyger [0000-0003-1578-0700], Vasyl Oleksiuk [0000-0003-2206-8447]
and Anna Skaskiv [0000-0002-3548-2383]
Ternopil Volodymyr Hnatiuk National Pedagogical University
2 M. Kryvonosa St., Ternopil, Ukraine
{nadbal, yava, shmyger, oleksyuk, skaskiv}@fizmat.tnpu.edu.ua
Abstract. At present, various strategies and initiatives focused on innovation of
educational technologies in higher pedagogical education are offered in
Ukraine. The study of the state of the formation of teachers’ digital competenc-
es in the process of their professional development has been carried out on the
basis of Ternopil Volodymyr Hnatiuk National Pedagogical University.
The article analyzes foreign and national approaches and strategies to the
development of teachers’ digital competences. The results of the study, aimed
to determine the features of mastering digital competencies in the process of
teachers’ professional development and their lifelong learning, are presented. In
total, 258 teachers from Ternopil and Ternopil region (Ukraine) took part in this
research. The study combines a variety of statistical tools and techniques in the
real contexts of higher education. The research has been carried out to deter-
mine the characteristics of elements that measure the digital competency of the
professional development. The results were processed based on the Item Re-
sponse Theory (IRT). This article demonstrates the utility of the standardized
LD χ2 statistic and the M2 statistic as provided in the software IRTPRO, but not
available readily in most IRT programs and not discussed commonly in peda-
gogical papers for IRT.
On the basis of the research carried out at the Ternopil Volodymyr Hnatiuk
National Pedagogical University, the strategy for the professional development
of digital competencies of teachers in the process of their lifelong learning has
been developed, which takes into account the results of the analysis of the crite-
ria and indicators inherent for the qualitative improvement of qualifications,
that have been determined by international standards and studies of professional
institutions.
Keywords: digital competencies, approaches, professional development, teach-
er training, lifelong learning.
1 Introduction
Society's digitalization involves the need to create strategies for the development of a
modern educational digital environment. As digital technologies are becoming central
part of everyday work, teachers are made to rethink and transform educational tradi-
tions through new technologies and learn throughout their lives. This problem re-
quires the creation of approaches to the development of teachers’ digital competences
in the process of their lifelong learning.
The reform of Ukrainian education involves a new educational strategy that focus-
es on the pupils and on the competence learning. This approach involves fundamental
changes in the professional priorities and school teachers’ roles [23]. Teachers must
adapt their professional competences in accordance with the requirements of the mod-
ern digital technologies development. Therefore, professional qualification improve-
ment and lifelong learning are of paramount importance for the development of
teachers’ digital competencies.
2 Justification of the problem
Digital education is a multifunctional concept that includes the structure, culture and
goals of schools, new roles of teachers and pupils. Increasing the efficiency of digital
pedagogical education requires special attention to the acquisition of digital compe-
tences in the process of professional development of teachers and their lifelong learn-
ing [6; 22; 3; 2].
Digital competence regards the ability to use digital technology effectively and to
function properly in a digital society, which is an essential part of lifelong learning
[20]. Acquiring digital competence refers to the learning to adapt the culture with
strong technological, informational and communicative elements.
The problem of distinguishing the maim competences in digital education and
teacher professional training is relevant and important today. The research [18] focus-
es is on the approaches to the development of digital competences in educational con-
texts. The author analyzes international studies over the past 10 years from the point
of view of politics, organizational infrastructure, strategic leadership, as well as of
lectures and their practices.
Hall R., Atkins, L. and Fraser, J. [10] reviewed a variety of four-level digital com-
petency structures that determine critical digital interest in achievement progress from
the basic requirements to the demonstration of expert, transformational skills, practice
and knowledge.
The levels of digital competence, specified in the DigEuLit project [14], changed
from the digital competency, general skills and approaches to the digital use and pro-
fessional application of these skills.
Competency approach is becoming a standard of pedagogical innovations [5] and a
major factor in the reformation of education system [11; 12].
In recent years, pedagogical aspects of the digital competence have been discussed
[7; 27; 13]. From J. [7] confirms that the pedagogical aspects of digital competence
should be considered not only at the level of teachers’ competence, but also at the
administrative level of school organization.
Ottestad, G., Kelentrić, M., Guðmundsdóttir, G. [15] described the use of ICT for
pedagogical and didactic purposes in Norwegian pedagogical curriculums. In the
context of their research, pedagogical education is of paramount importance for the
development of digitally competent teachers. They offer three main dimensions to
describe the professional competence of teachers: the Generic digital competence, the
Didactic digital competence and the Professionally-oriented digital competence.
The strategy of integrating digital competences into the professional teachers’ de-
velopment has been analyzed in the study [19]. The model includes a frame of 7 digi-
tal competences, 78 units of digital competences divided into three levels of compe-
tence development: the Basic Knowledge, the Knowledge Deepening and the
Knowledge Generation.
Tømte C., Kårstein, A., Olsen, D. [24] have revealed that the development of pro-
fessional digital competences in the whole world is poorly developed at the level of
pedagogical curriculum management and there is no complex approach to the devel-
opment of such competencies in most of the curriculum programs. In addition, they
noted that the academic strategies of pedagogical educational institutions on this topic
are not efficient enough and that the teaching staff’ experience varies greatly. Encour-
aging the professional digital competency of students and teachers in many pedagogi-
cal curriculum programs depends on enthusiasts among teachers.
In their studies, Gudmundsdottir, G., Loftsgarden, M., Ottestad, G. [8] stated that
only a few graduates – qualified teachers were satisfied with their knowledge and
skills gained at the university for work in a class equipped with digital instruments. At
the same time, teacher practitioners were interested in further development and deep-
ening of their digital competence, even if the schools in which they work do not set
clear requirements about the use of digital technologies for teaching and learning.
There are relatively few examples when a pedagogical institution clearly describes
how digital competence may be related to what a good teacher should be, or what
digital competencies will be formed in the process of teacher training and their life-
long learning. Authors [24; 25] also note that there is the need to improve collabora-
tion between schools and pedagogical universities in order to develop approaches to
supporting teachers’ digital competences.
Digital competences continue to be a problem for pedagogical practice and educa-
tional innovation, as well as the integration of digital technologies into the learning
process. Our main attention in this research is focused on the need to develop ap-
proaches to developing the digital competence of teachers in their professional quali-
fication improvement and lifelong learning.
3 The Presentation of Main Results
3.1 Methodology for identifying the teachers’ needs in digital competences
acquisition in the context of their professional development
During the research, we have used a set of research methods, namely: theoretical –
analysis of scientific and educational-methodical literature, official documents of the
European Union and the Ministry of Education and Science of Ukraine in order to to
determine the theoretical fundamentals of the problem acquisition; empirical – obser-
vation to identify the teachers’ needs in the digital competences in the process of their
professional development and lifelong learning; development of the approaches to the
development of teachers’ digital competences in the process of their lifelong learning;
statistical methods of mathematical processing of scientific data for the research re-
sults analysis and interpretation.
The conducted research is comprised of the following stages:
1. Review of the documents to reflect the contemporary understanding of the teach-
ers’ digital competences needed in the digital society and the education system.
2. Analysis of the digital technologies impact on teachers’ professional development,
role and functions in order to identify their needs for digital competency training in
the context of professional development.
3. Creating the questionnaire for assessing the teachers’ needs in digital competence
training and advanced training in this field. The research strategy from the begin-
ning involved the use of online survey.
4. Statistical processing of results and summing up.
5. Development of the approaches in order to the develop teachers’ digital compe-
tences in the process of their lifelong learning.
At the end of 2017, the preparatory phase of the study, consisting of series of inter-
views with experts on the digital competences integration into teachers’ professional
development and lifelong learning, took place. The preparatory phase laid the founda-
tion for a clearer statement of goals and objectives, clarification of the methodology,
contributed to the formulation of research hypotheses and to the development of re-
search tools.
The data collection phase has been conducted through the online survey. To collect
data for the study, the questionnaire with four sections was used and it contained 14
questions reflecting the objectives of this study. The survey lasted from February 20
to May 5, 2018. The time of work with the questionnaire was expected approximately
on 20 minutes.
258 respondents were involved through various informational channels on anony-
mous and free-of-charge basis. The target audience of the experiment was represented
by rural teachers (n = 127) working at schools of Ternopil region and teachers of the
city of Ternopil (n = 131).
The results of the experiment have showed that 75% of teachers are women, and
the rest are men. Most respondents were between the age of 28 and 58. Regarding the
professional profile of the respondents, the experiment has revealed that 70% of the
respondents had ten or more years of pedagogical experience, 30% – less than 10
years.
The study revealed that there is a link between some of the demographic character-
istics of respondents (age and place of residence) and their need for digital compe-
tence in the context of their professional development.
The research has also ascertained that there is a significant link between all re-
spondents’ professional characteristics in particular with the subject they teach at
school, their position, their work experience and their need for the digital competence
in the context of their professional development.
In the process of research such theses have been confirmed:
Both rural and urban teachers are generally not satisfied with the existing system of
teacher’ qualifications in the field of digital competence development.
During the advance teachers’ improvement in order to develop digital competenc-
es, traditional trajectories dominate that are often characterized by the limited crea-
tivity and by the lack of innovation practices.
Study has showed that among the challenges affecting teachers’ digital competence
of acquisition in professional development and their lifelong learning, there were the
lack of funds (46%), lack of time (51%), lack of motivation for professional growth
(42%), as well as the problems, associated with the educational sector in Ukraine
(21%), that were significant to them.
The author's strategy of the designing of the approaches in order to develop digital
competences in the process of professional training and teachers’ advanced training is
based on the European Digital Comprehensive Teachers Framework – DigCompEdu
[2; 21]. The digital competency of professional development contains 14 criteria,
which are grouped into 4 groups. The selection of criteria has derived from our expe-
rience of teachers’ training organization in the training center "Educational Innova-
tion" studies in the context of their digital competencies development.
At the preparatory stage, we suggested teachers to evaluate their level of digital
competencies development. The assessment has been carried out in a 5-point scale
based on the proposed criteria (see Table 1).
Table 1. Criteria for assessing the digital competency of professional development
1. Organizational communi- The use of digital technologies for:
cations. OC1. Access to pupils and parents’ resources and
The use of digital technolo-information
gies for communication be- OC2. Communication with colleagues by means
tween institutions and a teach-of digital technologies
er with stakeholders. OC3. Access to the joint development of commu-
nication strategies of the institution
2. Professional cooperation. The use of digital technologies for:
Using digital technologies to PC1. Collaborative with other educators to im-
collaborate with other educa- plement educational projects
tion workers, sharing PC2. Sharing resources and experiences with
knowledge and experience. colleagues
PC3. Collaborative development of educational
resources
PC4. New pedagogical practices and methods
study
3. Reflexive pedagogical The use of digital technologies for:
practice. RPP1. Finding of gaps in digital competency
The use of digital technologies RPP2. Search for educational materials for ad-
for individual and collective vanced professional development
reflection, the active develop- RPP3. Appealing for help to others to improve
ment of their own digital peda- their digital pedagogical competence.
gogical practice.
4. Professional lifelong de- The use of digital technologies for:
velopment LLD1. Planning your own learning
The use of digital technologies LLD2. Updating their professional subject com-
and resources for advanced petences
professional development. LLD3. Providing opportunities for colleagues
training
LLD4. Use of online learning opportunities
We have provided five possible options (categories) of answers for each item of
questionnaire: 1 – very low level (initial level), 2 – low level, 3 – medium level, 4 –
sufficient level, 5 – high level (expert level). For further statistical analysis of the
obtained data, we used the modern theory of testing IRT. This theory compared with
the classical theory of testing has such advantages as objective estimates of task pa-
rameters and knowledge level parameters.
For statistical processing of the data obtained, we used the IRTPRO software. The
response categories of 1, 2, 3, 4, 5 were translated into item scores 0, 1, 2, 3, 4 (interi-
or codes of response categories) by this program.
As far as obtained estimates depend on the level of teachers’ digital competence
() and on the complexity of the questionnaire questions ( ), we used the assumption
of the unidimensionality of our model. That is, the probability that the participant of
the test with the level of preparedness () executes the task of difficulty () is calcu-
lated by the formula
1
𝑃(𝜃 − 𝛿) = (1)
1 + 𝑒 𝜃−𝛿
The probability of success depends, in essence, only on one parameter - the differ-
ence − 𝛿. The level of preparedness and the complexity of the task are meas-
ured in logits and are plotted on the same scale.
3.2 Statistical and mathematical analysis of research data
First, we have checked the questionnaire (test) for internal consistency. To do this, for
all 14 questions, the coefficient alpha Cronbach have been calculated. It was accepta-
ble (α=0.8604).
Local independency. One of the assumption of unidimensional IRT models is that
of local independency (LI) or conditional independence. LI is the assumption that is
the only that influences on an individual’s item response is that of the latent trait vari-
able that is measured and that no other factors (e.g., other items on the measuring
scale or another latent trait variable) is influencing individual item responses. Local
dependency can occur for numerous reasons such as when the wording of two or
more items consist of the synonyms used across items that teachers can’t differentiate
between items, but only by selecting the same response category across items.
To assess the tenability of local independency, the standardized LD χ2 statistic for
each item pair has been examined. LD statistics greater than |10| are considered large
and reflecting likely LD issues or residual variance that is not accounted for by the
unidimensional IRT model. LD statistics between |5| and |10| are considered moderate
and questionable LD, and LD statistics less than |5| are considered small and inconse-
quential.
LD statistics for 14-item five-category scale are summarized in Table 2.
Table 2. Standardized LD χ2 Statistics
Item Label 1 2 3 4 5 6 7 8 9 10 11 12 13
1 OC1
2 OC2 2.0
3 OC3 0.1 0.8
4 PC1 0.1 -0.7 0.9
5 PC2 4.0 -1.0 0.2 0.8
6 PC3 5.3 -0.8 2.5 0.2 3.5
7 PC4 -0.6 0.6 0.2 1.4 0.1 3.2
8 RPP1 4.0 0.6 4.5 0.3 1.1 1.0 1.8
9 RPP2 2.1 -0.1 0.7 0.7 0.4 3.9 3.0 1.8
10 RPP3 1.7 3.0 2.7 1.2 0.4 0.4 3.3 2.1 -0.4
11 LLD1 3.2 1.1 1.8 -0.5 7.2 1.5 2.0 1.6 1.5 0.2
12 LLD2 -0.3 -0.7 -0.9 2.5 0.7 0.3 -0.1 1.1 1.7 0.2 0.1
13 LLD3 -0.5 3.3 -0.6 1.3 -0.2 2.3 1.1 0.6 0.5 4.3 0.4 -0.1
14 LLD4 0.6 0.3 1.0 0.2 2.9 -1.6 0.2 2.1 0.4 -0.3 1.6 0.8 2.8
Overall, LD statistics for the model corresponds to the 14-item five-category scale
and shows that most LD statistics are relatively small. Based on these results, the
assumption of local independency is tenable.
Unidimensional IRT models have the assumption, known as functional form,
which states that the observed or empirical data follow the function specified by the
IRT model. In the context of the IRT model, functional form implies that all threshold
parameters are ordered and that there is a common slope within each item, although
not necessarily across items. Essentially, the comparison has been made between the
empirical data and those that were predicted by the IRT model.
In addition to assessing model-data correspondence, it is important to check if each
item refers to the category system and operates as expected. To assess whether cate-
gories usage corresponds as expected (or not) to the IRT model (14-item five-
category scale), ORF (option response function) plots of each item have been inspect-
ed.
Software has been used to generate the ORF plots for all items, IRTPRO has an
easily accessible feature of this once. Figure 1 [15] provides ORF plots for all items,
which is typical for the IRT model.
As it can be seen, the predicted ORF plots shows that all items deport themselves
as a five-category item, with a category score of 0 (very low level) and is less likely to
be selected than any other category for almost the entire competencies continuum
(i.e., between −3 and 3).
Assessing IRT Model-Data Fit. To assess the correspondence of the model to each
item, a S-χ2 item-fit statistic for polytomous data has been examined. This item-fit
statistic is provided by default in IRTPRO. For each item, S-χ2 assesses the degree of
similarity between the model-predicted and the empirical (observed) response fre-
quencies by item response category. A statistically significant value indicates if the
model corresponds to the given item.
Table 3. Item-Fit Statistics (S- χ 2 Item Level Diagnostic Statistics) for 14-Item Five-Category Scale.
Item Label S- χ2 d.f. Probability
1 OC1 52.98 45 0.1931
2 OC2 59.02 49 0.1544
3 OC3 62.28 57 0.2932
4 PC1 113.15 68 0.0005
5 PC2 49.58 45 0.2951
6 PC3 56.72 45 0.1128
7 PC4 105.00 71 0.0054
8 RPP1 90.15 63 0.0140
9 RPP2 56.31 57 0.5019
10 RPP3 60.49 59 0.4228
11 LLD1 51.80 51 0.4436
12 LLD2 78.37 60 0.0557
13 LLD3 79.52 64 0.0912
14 LLD4 91.73 67 0.0241
Given, that the length of the scale is short, the statistics have been calculated at the
1% significance level. The items fit S-χ2 statistics (see Table 3) and indicate the satis-
factory correspondence except only 1 of the 14 items (p < 0.01 for Item 4 (PC1)).
Since the correspondence of the model to this item is not acceptable, then the Item 4
has been removed, and the IRT items calibration has been performed again, and tests
of item level correspondence have proved (see Table 4).
Table 4. Final Item-Fit Statistics (S- χ 2 Item Level Diagnostic Statistics) for 13-Item Five-Category Scale.
Item Label S- χ2 d.f. Probability
1 OC1 58.76 44 0.0673
2 OC2 58.82 44 0.0666
3 OC3 63.68 53 0.1493
4 PC2 92.06 70 0.0397
5 PC3 50.40 46 0.3028
6 PC4 51.25 43 0.1812
7 RPP1 92.60 67 0.0209
8 RPP2 99.31 64 0.0031
9 RPP3 48.39 50 0.5390
10 LLD1 55.26 59 0.6148
11 LLD2 53.19 52 0.4294
12 LLD3 61.37 60 0.4277
13 LLD4 67.20 58 0.1908
In this study, to analyze model-data correspondence respectively Granded and
GPCredit models have been used and -2 LogLikelihood (-2LL) values have been
gained for each model. -2LL values for each model are shown in Table 5.
Table 5. -2 Loglikelihood values for inter models
Granded GPCredit
-2 Log Likelihood: 7782.93 -2 Log Likelihood: 7841.94
To determine which model is appropriate for our data structure, the difference be-
tween -2LL values have been analyzed if it is over than the desired value looking up
at the 𝜒2 table. As there are 13 items in the test (after calibration), p=0.01 desired
value for 𝜒2 is 27.69. As it can be seen in Table 5 for the GPCredit and the Granded
models, the difference between -2LL values is 59.01. As the gained value is over than
the intended value, it has been determined that the Granded model is more appropriate
for our data structure than the GPCredit model.
Evaluating and Interpreting Results. Given that the model assumptions are tena-
ble, the description of the item properties, including the amount of information avail-
able, now we can apply for each item, subset of items, or the entire scale. The ITR
model item parameter estimates for the 13-items scale are provided in Table 6.
Table 6. Graded Model Item Parameter Estimates, logit: a (θ - b)
Item Label a s.e. b1 s.e. b2 s.e b3 s.e. b4 s.e
1 OC1 2.82 0.32 -0.92 0.13 0.31 0.09 1.33 0.12 2.41 0.22
2 OC2 2.22 0.25 -1.01 0.14 0.53 0.10 1.53 0.14 2.95 0.33
3 OC3 2.23 0.24 -0.94 0.14 0.18 0.10 1.15 0.12 2.29 0.22
4 PC1 1.01 0.15 -0.73 0.19 0.92 0.18 2.20 0.32 3.88 0.60
5 PC2 2.67 0.29 -1.16 0.14 0.13 0.09 1.33 0.12 2.65 0.26
6 PC3 1.55 0.20 -0.49 0.14 1.21 0.15 2.51 0.28 4.33 0.76
7 PC4 1.25 0.16 -2.37 0.32 -0.75 0.16 0.77 0.14 2.09 0.26
8 RPP1 1.72 0.20 -1.06 0.16 0.13 0.11 1.24 0.14 2.34 0.24
9 RPP2 1.75 0.22 -0.78 0.15 0.66 0.11 1.59 0.16 3.33 0.43
10 RPP3 1.46 0.19 -0.36 0.13 0.66 0.12 1.95 0.22 3.54 0.49
11 LLD1 2.37 0.26 -1.23 0.15 0.08 0.10 1.07 0.11 2.30 0.21
12 LLD2 1.83 0.21 -0.85 0.14 0.37 0.10 1.23 0.13 2.42 0.25
13 LLD3 1.23 0.18 -0.34 0.15 1.14 0.17 2.15 0.28 4.21 0.67
Parameter a is the slope; b1, b2, b3, b4 present the ability to value at the thresholds
between the response-option categories for the item. Each threshold reflects the level
of generally perceived selfefficacy needed to have equal 0.50 probability by choosing
the corresponding above the given threshold. In our study there are 5 graded catego-
ries or response options, thus there are 4 b values. b1 is the threshold for the trace line
describing the probability of chosen category 2, 3, 4, or 5. b2 is the threshold for the
trace line describing the probability of chosen category 3, 4, or 5. b3 is the threshold
for the trace line describing the probability of chosen category 4 or 5. b4 is the thresh-
old for the trace line describing the probability of chosen category 5. For example, to
determine the probability that someone will choose category 2, we subtract the proba-
bility dictated by the trace line defined by b2 from that dictated by the trace line de-
fined by b1.
The slope estimates the range from 1.01 (Item 4) to 2.82 (Item 1). In general, all
items have a two level relationship with general teachers’ digital competency (first
level slope values – from 1.01 to 1.83 and the second level – from 2.22 to 2.82). But
the large level of slopes for Items 1-3, 5, 11 indicates that they have the strongest
relationship with the latent trait and measure general digital competency more pre-
cisely than other items.
Threshold parameters for the Granded model correspond to the 13-item of five-
category scale ranged from –2.37 (Item 7) to –0.34 (Item 13) for b1, from –0.75 (Item
7) to 1.21 (Item 6) for b2, from 1.07 (Item 11) to 2.51 (Item 6) for b3, from 2.09 (Item
7) to 4.21 (Item 13) for b4. The majority of b1, b2, b3 and b4 thresholds for the items
are around general digital competency level of –0.94, 0.43, 1.54, 2.98, respectively.
The range of average values of the thresholds is wide enough and they differ by more
than 1. This information implies that the used scale is the most useful in distinguish-
ing between teachers around these latent trait levels.
Each item has its own item information function (IIF) that is shaped by its slope
and thresholds. IIFs are used to identify how much empirical information each item
adds to the entire scale and where that information appears along the continuum.
IIFs are readily available in the IRTPRO once the set of items have been calibrat-
ed.
Figure 2 [8] shows IIFs for 13 items from the five-category scale.
The information function of the ideal test must have one clearly expressed extre-
mum. If the graph of the information function has a smooth, but not clearly expressed
extremum, it suggests a decrease in the effectiveness of the entire test. In the case of
several local extrema, for example, two at θ1 and θ2, the test needs to be improved. If
the number of items in the test is not big, then you need to add items that have an
intermediate complexity θ1 < < θ2 to eliminate the "failures" between adjacent ex-
tremums.
The IIFs for Items 1, 2, 3, 5 and 8 stands out the most from all other items because
it provides the most amount of information (precision). The maximum values IIFs of
these items are in the range of 1.5 to 2.0. For instance, the IIF for Item 1 (OC1) has
local extrema in four point θ = −0.92, θ = 0.31, θ = 1.33, θ = 2.41, which are the
item’s respective thresholds b1, b2, b3, b4. The items providing the least amount of
information across the continuum are Items 4 (PC2) and 13 (LLD4) as their slope
values were the lowest relative to all other items on the scale. There are pairs of items
that appear to provide nearly identical information across the continuum because their
respective IIFs are nearly identical, so that suggests that only one of these items may
be necessary. Such pairs are represented by the pairs of Items 3 and 11, Items 8 and
12.
To understand how the 13-item five-category scale works in the whole, the area
under each IIF can be summed together to create a total information function (TIF).
Each item contributes independently the unique information to the TIF and is not
dependent on other items. This is also another reason why the assumption of LI is
important. The TIF provides useful details about variable scale information on the
trait continuum. Furthermore, the TIF can be used to identify gaps in the continuum.
Useful metric to capture the amount of error around an IRT score is the expected
standard error of estimate (SEE; SEE ≈1/√information). The SEE can also be used as
a function to gauge the expected amount of errors along the continuum.
Figure 3 [26] shows graphs for changing the basic data (measured in logits) and the
standard error of measurement.
As it can be seen from Figure 3, with values from -1.5 to 3.0, the SEE is almost
constant and slightly less than 0.3, but the value of the information function is the
range from 13 to 15 (approximately constantly). Then the estimated marginal reliabil-
ity for this range is 1−0.322 0.91. The Marginal Reliability for Corresponding Pat-
tern Scores provided by the IRTPRO is 0.92. This means that for latent values greater
than -1.5 the values of the indicator variables are the most reliable. However, outside
this range of −1.5 to 3.0 marginal reliability decreases and the SEE increases. Thus, if
a more precise GSE scale was desired within this range or across more of the continu-
um, then more items are need to be added to the scale to meet the desired information
or level of expected SEE.
To summarize, the 13-item five-category scale provides precise estimates of the
scores (information 14, marginal reliability 0.92, expected SEE 0.3) for a broad
range of the continuum, −1.5 to 3.0. The maximum amount of information (precision)
is approximately 15 around latent trait estimates 1.3. However, precision and ex-
pected SEEs around score estimates worsen outside of this range. To improve score
estimates beyond this range it is need to write additional items that have thresholds
below −1.5.
According to the IRT analysis, the following conclusions can be drawn:
1. Analyzing the characteristic functions (see Figure 1) of the questionnaire ques-
tions, it is possible to note that the probability of choosing the response of category
0 (very low level (initial level) for all 14 distractors (OC1, OC2, OC3, PC1, PC2,
PC3, PC4, RPP1, RPP2, RPP3, LLD1, LLD2, LLD3, LLD4) with = -3 is within
the approximate range from 0.75 to 1.0 (fairly high limits). This means that a small
number of teachers assesses their level of digital competency according to all dis-
tractors at the initial level, which is a very positive factor at the present time. The
higher the level of the teachers’ digital competences, the smaller is the probability
to choose from the category 0 a response, which is completely natural. For most
distractors, the probability of choosing from the category 0 response falls to zero
for teachers with an average level of competencies ( = 0), the exception is for the
PC1, PC3, RPP3, LLD3, LLD4 distractors, for which the probability of choosing a
category response from 0 equals to = -3 . This means that the competencies that
correspond to these distractors are not yet well formed even among teachers with
an average level of general digital competence.
2. The probability to choose a response from the category 4 (high level (expert lev-
el)) for all 14 distractors with = 3 is within the approximate range from 0.2 to
0.8. This means that teachers who generally have a high level of overall digital
competence (or believe that it is of such a level), in the context of exact distractors,
have a very miscellaneous level of preparedness. The attention should be paid to
the development of competencies that correspond to distractors for which the cor-
responding probability is less than 0.5. These are PC1, PC3, RPP2, RPP3, LLD3,
LLD4.
3. If the graphs of characteristic functions for categories 1, 2 and 3 reach their maxi-
mum somewhere in the middle of the scale from -3 to 3, then this is normal from
the point of view of the IRT analysis. But as our study has revealed that the curve 3
for individual distractors (PC1, PC3, RPP2, RPP3, LLD3, LLD4) reaches its max-
imum at the right end of the scale for . This means that teachers who are consid-
ered to having the high level of general digital competence, in fact, the level of
their competence that corresponds to the specified distractors is not sufficient.
4. The particular concern is caused by the competences with responses from the cate-
gories 1, 2 and 3 have a less probability than 0.5 and when the maxima of these
probabilities are shifted to the right. These are the competences: PC1, RPP1, RPP2,
RPP3, LLD2, LLD3. The displacement of the maximum of probabilities, shifted to
the right, means that teachers who are considered to have an average level of com-
petences, in general, have an inadequate level of preparedness of these competen-
cies.
5. From the above mentioned, the level that correspond to the OC1, OC2, OC3, PC2
and LLD1 distractors considered to be satisfactory from the point of probabilistic
statistical analysis.
6. When analyzing S- χ 2 Item Level Diagnostic Statistics, we came to the conclusion
that the PC1 item should be removed from the questionnaire (see Table 3 and Ta-
ble 4). Indeed, if we analyze Table 1 at the content level, the attention can be
drawn to the fact that the PC1 and PC3 distractors concern in fact to one compe-
tence, which is realized in different activity directions.
7. The analysis of the information functions of the questionnaire (see Figure 2) shows
that items PC2, RPP1 and LLD4 were not informative enough in the general con-
text of digital competencies evaluation. In order to do the repeated research on
general digital competence after the practical implementation of the strategy for its
formation or individual stages of this strategy, these distractors should be correct-
ed.
8. From the graph of the general information function (see Figure 3) it is clear that the
IRT analysis gives the sufficiently complete information about the general digital
competency of the teachers. Only in cases of very low competence or close to it the
value of the general information function is low and according to it, the standard
error of estimation (SEE) is high. This indicates the fairly good selection of distrac-
tors for this study.
3.3 Development of approaches to the development of digital competences of
teachers in the process of their lifelong learning at Ternopil Volodymyr
Hnatiuk National Pedagogical University
From the study, it follows that teachers with different levels of digital competency of
professional development do not have well-formed competencies such as: working
with other educators to implement educational projects, joint development of educa-
tional resources, appeal to others to improve their digital pedagogical competence,
provision learning opportunities for colleagues, the use of online learning opportuni-
ties.
Proceeding from this, approaches to the development of digital competencies in the
process of improving of teachers' qualification at the training center "Educational
Innovation" of Ternopil Volodymyr Hnatiuk National Pedagogical University were
developed. They are based on a model for teaching teachers throughout their lives
based on the development of digital competencies [1].
Proposed approaches to the professional development of digital competences of
teachers in the process of their lifelong learning include groups of criteria for the
planning and development of organizational communications, engagement and pro-
fessional co-operation, assessment and reflexive pedagogical practice, sustainability,
and professional development throughout life.
In the process of planning and developing organizational communications, atten-
tion is paid to both the contemporary national and world context and the individual
experience of developing digital competences of teachers in the process of their pro-
fessional development and lifelong learning, namely:
─ combining a subject of the learning with context in which teachers work at the
level of school, community, region;
─ correlations of qualification improving on the development of digital competences
of teachers in the process of their professional development and lifelong learning
with standards, programs and goals at the school, community, region, and state
levels;
─ the use of digital technologies for communication of institution and teacher with
other teachers and pupils.
The group of criteria for "engagement and professional co-operation" envisages an
active role for teachers in professional co-operation, community building and motiva-
tion to share their pedagogical experience:
─ collaboration with other educators for the implementation of educational projects;
─ joint development of educational resources;
─ supporting professional co-operation, providing learning opportunities of the learn-
ing for colleagues;
─ the creation and development of professional communities with horizontal links to
ensure mutual learning and discussion of new ideas;
─ appealing for help to others to improve their digital pedagogical competence;
─ searching study materials for continuing professional development, using online
learning opportunities.
In the group of criteria "evaluation and reflective pedagogical practice" the emphasis
is on formal assessment, qualitative feedback and constant reflection:
─ demonstration of the service's compliance with the stated objectives and learning
outcomes;
─ the use of digital technologies for individual and collective reflexive pedagogical
practice;
─ feedback opportunities for those who take part in the improving of qualifications;
─ discussion of specific features related to the received knowledge, materials or skills
that will be demonstrated by a successful transition to the implementation of pro-
fessional activities;
─ adding participants to the assessment of their knowledge and skills.
The group of criteria for "sustainability and professional lifelong development" pro-
vides post-support, facilitates better motivation of educators for lifelong learning and
helps in building an individualized trajectory of professional growth in the field of
digital technologies, namely:
─ detailing of further steps after training that teachers need to apply in a new envi-
ronment;
─ proposals for continuing education through information and technical post-support;
─ provision of training opportunities for colleagues;
─ use of online learning opportunities (massive open online courses, webinars, etc.).;
─ advising on the implementation of educational innovations.
These approaches are already being implemented in practice in the process of qualifi-
cation improving of teachers and their lifelong learning at the international education-
al training center of Ternopil Volodymyr Hnatiuk National Pedagogical University.
4 Conclusions
To develop approaches to assessing the professional development of digital compe-
tences of teachers during their lifelong learning, levels of their formation were deter-
mined, as well as relevant criteria and indicators.
The results were processed based on the theory of modeling and parametrization of
tests IRT. We can state the appropriateness of choosing the standardized statistics LD
χ2 and statistics M2, presented in IRTPRO.
On the basis of the conducted research, the approaches to the professional devel-
opment of digital competence of teachers in the process of their lifelong learning are
proposed, which include the following groups of criteria: planning and development
of organizational communications, engagement and professional cooperation, assess-
ment and reflective pedagogical practice, sustainability and professional development
throughout life.
Among the main vectors of the strategy of professional development of teachers in
the context of the development of digital competencies, it should be noted: the crea-
tion and development of professional communities with horizontal links to ensure
mutual learning and discussion of new ideas; the use of digital technologies and re-
sources in the learning process, modeling of the learning process, oriented on results
and educational projects.
References
1. Balyk, N., Barna, O., Shmyger, G., Oleksiuk, V.: Model of Professional Retrain-
ing of Teachers Based on the Development of STEM Competencies, http://ceur-
ws.org/Vol-2104/paper_157.pdf (2018).
2. DigCompEdu, https://ec.europa.eu/jrc/digcompedu (2017).
3. DigComp 2.1: The Digital Competence Framework for Citizens with eight profi-
ciency levels and examples of use,
http://publications.jrc.ec.europa.eu/repository/bitstream/JRC106281/web-
digcomp2.1pdf_(online).pdf (2017).
4. Edwards, M.: An Introduction to Item Response Theory Using the Need for Cog-
nition Scale. Social and Personality Psychology Compass, 3(4), рр.507–529
(2009)
5. European Commission. Key competences for lifelong learning. A European frame
of reference. Luxembourg: Office for Official Publications of the European
Communities,
http://www.mecd.gob.es/dctm/ministerio/educacion/mecu/movilidad-europe /
competenciasclave.pdf? documentId = 0901e72b80685fb1 (2007).
6. Ferrari, A.: DIGCOMP: A Framework for Developing and Understanding Digital
Competence in Europe; Publications Office of the European Union: Luxembourg,
Volume EUR 26035,
http://publications.jrc.ec.europa.eu/repository/bitstream/JRC83167/lb-na-26035-
enn.pdf (2013).
7. From, J.: Pedagogical digital competence – between values, knowledge and skills.
Higher Education Studies, 7(2), рр.43–50 (2017).
8. Graded response model item information functions for 13 items scale,
https://drive.google.com/file/d/1-4yQ_vNRmBVHHySp9mBimWtq8IGTVe4X/view
(2019).
9. Gudmundsdottir, G., Loftsgarden, M., Ottestad, G.: Nyutdannede lærere:
Profesjonsfaglig digital kompetanse og erfaringer med IKT i lærerutdanningen.
Senter for IKT i utdanningen. (2014).
10. Hall, R., Atkins, L., Fraser, J.: Defining a self-evaluation digital literacy frame-
work for secondary educators: the DigiLit Leicester project. Research in Learning
Technology, 22 (2014).
11. Hargreaves, A.: Teaching in the knowledge society: Education in the age of inse-
curity. New York: Teachers College Press. (2003).
12. Hatlevik, O. E.: Examining the relationship between teachers’ self-efficacy, their
digital competence, strategies to evaluate information, and use of ICT at school.
Scandinavian Journal of Educational Research. 61(5), рр.555-567 (2017).
13. Lund, A., Furberg, A., Bakken, J., Engelien, K. L.: What does professional digital
competence mean in teacher education? Nordic Journal of Digital Literacy, 9(4),
рр.281–299 (2014).
14. Martin, A., Grudziecki, J.: DigEuLit: concepts and tools for digital literacy devel-
opment. Innovation in Teaching and Learning in Information and Computer Sci-
ences, 5(4). рр.249-267 (2006).
15. ORF for Items 1-14,
https://drive.google.com/file/d/1vW62WETrp1H41vRuyJJBJXwLelDS77w0/view
(2019).
16. Ottestad, G., Kelentrić, M., Guðmundsdóttir, G.: Professional Digital Competence
in Teacher Education. Nordic Journal of Digital Literacy, 9, рр. 243-249 (2014).
17. Paek, I., Han, K.: IRTPRO 2.1 for Windows (Item Response Theory for Patient-
Reported Outcomes). Applied Psychological Measurement, 37, рр. 242-252 (2013).
18. Pettersson, F.: On the issues of digital competence in educational contexts – a review
of literature. Education and Information Technologies, 23(3), рр. 1005–1021 (2018).
19. Pozos P., Torelló O.: The digital competence as a cross-cutting axis of higher educa-
tion teachers’ pedagogical competences in the european higher education area. WCES
2012. Procedia – Social and Behavioral Sciences, 46, рр.1112 – 1116 (2012).
20. Proposal for a Council Recommendation on Key Competences for Lifelong
Learning, Brussels, https://ec.europa.eu/education/sites/education/files/ recom-
mendation-key-competences-lifelong-learning.pdf (2018).
21. Shmyger, G., Balyk, N.: Formation of Digital Competencies in the Process of
Changing Educational Paradigm from E-Learning to Smart-Learning at Pedagogi-
cal University. In E.Smyrnova-Trybulska (Eds.), Monograph «E-learning Meth-
odology – Effective Development of Teachers’ Skills in the Area of ICT and E-
learning», 9, pp. 483 – 497 (2017).
22. Shyshkina, M.: Emerging Technologies for Training of ICT-Skilled Educational
Personnel. In: Ermolayev, V., Mayr, H.C., Nikitchenko, M., Spivakovsky, A.,
Zholtkevych, G. (ed.) Information and Communication Technologies in Educa-
tion, Research, and Industrial Applications. ICTERI 2013. Communications in
Computer and Information Science, 412, pp. 274-284,
https://link.springer.com/chapter/10.1007/978-3-319-03998-5_14 (2013).
23. Spirin, O., Nosenko, Yu., Іatsyshyn, A.: Current requirements and contents of
training of qualified scientists on information and communication technologies in
education. In: J. Information Technologies and Learning Tools, 56(6),
http://journal.iitta.gov.ua/index.php/itlt/article/view/632/483 (2016) (in Ukrainian)
24. Tømte, C., Kårstein, A., Olsen, D.: IKT i lærerutdanningen. På vei mot profes-
jonsfaglig digital kompetanse? Oslo: NIFU. (2013).
25. Tømte, C. Digital competence in teacher education. Learning & Teaching with
Media & Technology, рр. 173-182. (2013).
26. Total information function and expected SEE function,
https://drive.google.com/file/d/11eiFVcS96DDthG6uR7yY0UE34BNsLk_I/view (2019).
27. Wastiau, P., Blamire, R., Kearney, C., Quittre, V., Van de Gaer, E., Monseur, C.:
The use of ICT in education: A survey of schools in Europe. European Journal of
Education, 48(1), рр. 11–27 (2013).