At present, various strategies and initiatives focused on innovation of educational technologies in higher pedagogical education are offered in Ukraine. The study of the state of the formation of teachers' digital competences in the process of their professional development has been carried out on the basis of Ternopil Volodymyr Hnatiuk National Pedagogical University. The article analyzes foreign and national approaches and strategies to the development of teachers' digital competences. The results of the study, aimed to determine the features of mastering digital competencies in the process of teachers' professional development and their lifelong learning, are presented. In total, 258 teachers from Ternopil and Ternopil region (Ukraine) took part in this research. The study combines a variety of statistical tools and techniques in the real contexts of higher education. The research has been carried out to determine the characteristics of elements that measure the digital competency of the professional development. The results were processed based on the Item Response Theory (IRT). This article demonstrates the utility of the standardized LD χ2 statistic and the M2 statistic as provided in the software IRTPRO, but not available readily in most IRT programs and not discussed commonly in pedagogical papers for IRT. On the basis of the research carried out at the Ternopil Volodymyr Hnatiuk National Pedagogical University, the strategy for the professional development of digital competencies of teachers in the process of their lifelong learning has been developed, which takes into account the results of the analysis of the criteria and indicators inherent for the qualitative improvement of qualifications, that have been determined by international standards and studies of professional institutions.
Society's digitalization involves the need to create strategies for the development of a modern educational digital environment. As digital technologies are becoming central part of everyday work, teachers are made to rethink and transform educational traditions through new technologies and learn throughout their lives. This problem requires the creation of approaches to the development of teachers’ digital competences in the process of their lifelong learning.
The reform of Ukrainian education involves a new educational strategy that
focuses on the pupils and on the competence learning. This approach involves fundamental
changes in the professional priorities and school teachers’ roles [
Digital education is a multifunctional concept that includes the structure, culture and goals of schools, new roles of teachers and pupils. Increasing the efficiency of digital pedagogical education requires special attention to the acquisition of digital competences in the process of professional development of teachers and their lifelong learning [6; 22; 3; 2].
Digital competence regards the ability to use digital technology effectively and to
function properly in a digital society, which is an essential part of lifelong learning
[
The problem of distinguishing the maim competences in digital education and
teacher professional training is relevant and important today. The research [
Hall R., Atkins, L. and Fraser, J. [
The levels of digital competence, specified in the DigEuLit project [
Competency approach is becoming a standard of pedagogical innovations [
In recent years, pedagogical aspects of the digital competence have been discussed
[7; 27; 13]. From J. [
Ottestad, G., Kelentrić, M., Guðmundsdóttir, G. [
The strategy of integrating digital competences into the professional teachers’
development has been analyzed in the study [
Tømte C., Kårstein, A., Olsen, D. [
In their studies, Gudmundsdottir, G., Loftsgarden, M., Ottestad, G. [
There are relatively few examples when a pedagogical institution clearly describes how digital competence may be related to what a good teacher should be, or what digital competencies will be formed in the process of teacher training and their lifelong learning. Authors [24; 25] also note that there is the need to improve collaboration between schools and pedagogical universities in order to develop approaches to supporting teachers’ digital competences.
Digital competences continue to be a problem for pedagogical practice and educational innovation, as well as the integration of digital technologies into the learning process. Our main attention in this research is focused on the need to develop approaches to developing the digital competence of teachers in their professional qualification improvement and lifelong learning.
During the research, we have used a set of research methods, namely: theoretical – analysis of scientific and educational-methodical literature, official documents of the European Union and the Ministry of Education and Science of Ukraine in order to to determine the theoretical fundamentals of the problem acquisition; empirical – observation to identify the teachers’ needs in the digital competences in the process of their professional development and lifelong learning; development of the approaches to the development of teachers’ digital competences in the process of their lifelong learning; statistical methods of mathematical processing of scientific data for the research results analysis and interpretation.
The conducted research is comprised of the following stages: 1. Review of the documents to reflect the contemporary understanding of the teachers’ digital competences needed in the digital society and the education system. 2. Analysis of the digital technologies impact on teachers’ professional development, role and functions in order to identify their needs for digital competency training in the context of professional development. 3. Creating the questionnaire for assessing the teachers’ needs in digital competence training and advanced training in this field. The research strategy from the beginning involved the use of online survey. 4. Statistical processing of results and summing up. 5. Development of the approaches in order to the develop teachers’ digital competences in the process of their lifelong learning.
At the end of 2017, the preparatory phase of the study, consisting of series of interviews with experts on the digital competences integration into teachers’ professional development and lifelong learning, took place. The preparatory phase laid the foundation for a clearer statement of goals and objectives, clarification of the methodology, contributed to the formulation of research hypotheses and to the development of research tools.
The data collection phase has been conducted through the online survey. To collect data for the study, the questionnaire with four sections was used and it contained 14 questions reflecting the objectives of this study. The survey lasted from February 20 to May 5, 2018. The time of work with the questionnaire was expected approximately on 20 minutes.
258 respondents were involved through various informational channels on anonymous and free-of-charge basis. The target audience of the experiment was represented by rural teachers (n = 127) working at schools of Ternopil region and teachers of the city of Ternopil (n = 131).
The results of the experiment have showed that 75% of teachers are women, and the rest are men. Most respondents were between the age of 28 and 58. Regarding the professional profile of the respondents, the experiment has revealed that 70% of the respondents had ten or more years of pedagogical experience, 30% – less than 10 years.
The study revealed that there is a link between some of the demographic characteristics of respondents (age and place of residence) and their need for digital competence in the context of their professional development.
The research has also ascertained that there is a significant link between all respondents’ professional characteristics in particular with the subject they teach at school, their position, their work experience and their need for the digital competence in the context of their professional development.
In the process of research such theses have been confirmed:
Both rural and urban teachers are generally not satisfied with the existing system of teacher’ qualifications in the field of digital competence development.
During the advance teachers’ improvement in order to develop digital competences, traditional trajectories dominate that are often characterized by the limited creativity and by the lack of innovation practices.
Study has showed that among the challenges affecting teachers’ digital competence of acquisition in professional development and their lifelong learning, there were the lack of funds (46%), lack of time (51%), lack of motivation for professional growth (42%), as well as the problems, associated with the educational sector in Ukraine (21%), that were significant to them.
The author's strategy of the designing of the approaches in order to develop digital competences in the process of professional training and teachers’ advanced training is based on the European Digital Comprehensive Teachers Framework – DigCompEdu [2; 21]. The digital competency of professional development contains 14 criteria, which are grouped into 4 groups. The selection of criteria has derived from our experience of teachers’ training organization in the training center "Educational Innovation" studies in the context of their digital competencies development.
At the preparatory stage, we suggested teachers to evaluate their level of digital competencies development. The assessment has been carried out in a 5-point scale based on the proposed criteria (see Table 1).
cations. OC1. Access to pupils and parents’ resources and
The use of digital technolo- information gies for communication be- OC2. Communication with colleagues by means tween institutions and a teach- of digital technologies er with stakeholders. OC3. Access to the joint development of communication strategies of the institution
Using digital technologies to PC1. Collaborative with other educators to imcollaborate with other educa- plement educational projects tion workers, sharing PC2. Sharing resources and experiences with knowledge and experience. colleagues
practice. RPP1. Finding of gaps in digital competency The use of digital technologies RPP2. Search for educational materials for adfor individual and collective vanced professional development reflection, the active develop- RPP3. Appealing for help to others to improve ment of their own digital peda- their digital pedagogical competence. gogical practice.
velopment LLD1. Planning your own learning The use of digital technologies LLD2. Updating their professional subject comand resources for advanced petences professional development. LLD3. Providing opportunities for colleagues training
LLD4. Use of online learning opportunities
We have provided five possible options (categories) of answers for each item of questionnaire: 1 – very low level (initial level), 2 – low level, 3 – medium level, 4 – sufficient level, 5 – high level (expert level). For further statistical analysis of the obtained data, we used the modern theory of testing IRT. This theory compared with the classical theory of testing has such advantages as objective estimates of task parameters and knowledge level parameters.
For statistical processing of the data obtained, we used the IRTPRO software. The response categories of 1, 2, 3, 4, 5 were translated into item scores 0, 1, 2, 3, 4 (interior codes of response categories) by this program.
As far as obtained estimates depend on the level of teachers’ digital competence () and on the complexity of the questionnaire questions (), we used the assumption of the unidimensionality of our model. That is, the probability that the participant of the test with the level of preparedness () executes the task of difficulty () is calculated by the formula ( − ) =
1 1 + − (1)
The probability of success depends, in essence, only on one parameter - the difference − . The level of preparedness and the complexity of the task are measured in logits and are plotted on the same scale. 3.2
First, we have checked the questionnaire (test) for internal consistency. To do this, for all 14 questions, the coefficient alpha Cronbach have been calculated. It was acceptable (α=0.8604).
Local independency. One of the assumption of unidimensional IRT models is that of local independency (LI) or conditional independence. LI is the assumption that is the only that influences on an individual’s item response is that of the latent trait variable that is measured and that no other factors (e.g., other items on the measuring scale or another latent trait variable) is influencing individual item responses. Local dependency can occur for numerous reasons such as when the wording of two or more items consist of the synonyms used across items that teachers can’t differentiate between items, but only by selecting the same response category across items.
To assess the tenability of local independency, the standardized LD χ2 statistic for each item pair has been examined. LD statistics greater than |10| are considered large and reflecting likely LD issues or residual variance that is not accounted for by the unidimensional IRT model. LD statistics between |5| and |10| are considered moderate and questionable LD, and LD statistics less than |5| are considered small and inconsequential.
LD statistics for 14-item five-category scale are summarized in Table 2.
Overall, LD statistics for the model corresponds to the 14-item five-category scale and shows that most LD statistics are relatively small. Based on these results, the assumption of local independency is tenable.
Unidimensional IRT models have the assumption, known as functional form, which states that the observed or empirical data follow the function specified by the IRT model. In the context of the IRT model, functional form implies that all threshold parameters are ordered and that there is a common slope within each item, although not necessarily across items. Essentially, the comparison has been made between the empirical data and those that were predicted by the IRT model.
In addition to assessing model-data correspondence, it is important to check if each item refers to the category system and operates as expected. To assess whether categories usage corresponds as expected (or not) to the IRT model (14-item fivecategory scale), ORF (option response function) plots of each item have been inspected.
Software has been used to generate the ORF plots for all items, IRTPRO has an
easily accessible feature of this once. Figure 1 [
As it can be seen, the predicted ORF plots shows that all items deport themselves as a five-category item, with a category score of 0 (very low level) and is less likely to be selected than any other category for almost the entire competencies continuum (i.e., between −3 and 3).
Assessing IRT Model-Data Fit. To assess the correspondence of the model to each item, a S-χ2 item-fit statistic for polytomous data has been examined. This item-fit statistic is provided by default in IRTPRO. For each item, S-χ2 assesses the degree of similarity between the model-predicted and the empirical (observed) response frequencies by item response category. A statistically significant value indicates if the model corresponds to the given item.
Label OC1 OC2 OC3 PC2 PC3 PC4
-2 Log Likelihood: 7782.93 -2 Log Likelihood: 7841.94 To determine which model is appropriate for our data structure, the difference between -2LL values have been analyzed if it is over than the desired value looking up at the 2 table. As there are 13 items in the test (after calibration), p=0.01 desired value for 2 is 27.69. As it can be seen in Table 5 for the GPCredit and the Granded models, the difference between -2LL values is 59.01. As the gained value is over than the intended value, it has been determined that the Granded model is more appropriate for our data structure than the GPCredit model.
Evaluating and Interpreting Results. Given that the model assumptions are tenable, the description of the item properties, including the amount of information available, now we can apply for each item, subset of items, or the entire scale. The ITR model item parameter estimates for the 13-items scale are provided in Table 6.
Parameter a is the slope; b1, b2, b3, b4 present the ability to value at the thresholds between the response-option categories for the item. Each threshold reflects the level of generally perceived selfefficacy needed to have equal 0.50 probability by choosing the corresponding above the given threshold. In our study there are 5 graded categories or response options, thus there are 4 b values. b1 is the threshold for the trace line describing the probability of chosen category 2, 3, 4, or 5. b2 is the threshold for the trace line describing the probability of chosen category 3, 4, or 5. b3 is the threshold for the trace line describing the probability of chosen category 4 or 5. b4 is the threshold for the trace line describing the probability of chosen category 5. For example, to determine the probability that someone will choose category 2, we subtract the probability dictated by the trace line defined by b2 from that dictated by the trace line defined by b1.
The slope estimates the range from 1.01 (Item 4) to 2.82 (Item 1). In general, all items have a two level relationship with general teachers’ digital competency (first level slope values – from 1.01 to 1.83 and the second level – from 2.22 to 2.82). But the large level of slopes for Items 1-3, 5, 11 indicates that they have the strongest relationship with the latent trait and measure general digital competency more precisely than other items.
Threshold parameters for the Granded model correspond to the 13-item of fivecategory scale ranged from –2.37 (Item 7) to –0.34 (Item 13) for b1, from –0.75 (Item 7) to 1.21 (Item 6) for b2, from 1.07 (Item 11) to 2.51 (Item 6) for b3, from 2.09 (Item 7) to 4.21 (Item 13) for b4. The majority of b1, b2, b3 and b4 thresholds for the items are around general digital competency level of –0.94, 0.43, 1.54, 2.98, respectively. The range of average values of the thresholds is wide enough and they differ by more than 1. This information implies that the used scale is the most useful in distinguishing between teachers around these latent trait levels.
Each item has its own item information function (IIF) that is shaped by its slope and thresholds. IIFs are used to identify how much empirical information each item adds to the entire scale and where that information appears along the continuum.
IIFs are readily available in the IRTPRO once the set of items have been calibrated.
Figure 2 [
The information function of the ideal test must have one clearly expressed extremum. If the graph of the information function has a smooth, but not clearly expressed extremum, it suggests a decrease in the effectiveness of the entire test. In the case of several local extrema, for example, two at θ1 and θ2, the test needs to be improved. If the number of items in the test is not big, then you need to add items that have an intermediate complexity θ1 < < θ2 to eliminate the "failures" between adjacent extremums.
The IIFs for Items 1, 2, 3, 5 and 8 stands out the most from all other items because it provides the most amount of information (precision). The maximum values IIFs of these items are in the range of 1.5 to 2.0. For instance, the IIF for Item 1 (OC1) has local extrema in four point θ = −0.92, θ = 0.31, θ = 1.33, θ = 2.41, which are the item’s respective thresholds b1, b2, b3, b4. The items providing the least amount of information across the continuum are Items 4 (PC2) and 13 (LLD4) as their slope values were the lowest relative to all other items on the scale. There are pairs of items that appear to provide nearly identical information across the continuum because their respective IIFs are nearly identical, so that suggests that only one of these items may be necessary. Such pairs are represented by the pairs of Items 3 and 11, Items 8 and 12.
To understand how the 13-item five-category scale works in the whole, the area under each IIF can be summed together to create a total information function (TIF). Each item contributes independently the unique information to the TIF and is not dependent on other items. This is also another reason why the assumption of LI is important. The TIF provides useful details about variable scale information on the trait continuum. Furthermore, the TIF can be used to identify gaps in the continuum.
Useful metric to capture the amount of error around an IRT score is the expected standard error of estimate (SEE; SEE ≈1/√information). The SEE can also be used as a function to gauge the expected amount of errors along the continuum.
Figure 3 [
As it can be seen from Figure 3, with values from -1.5 to 3.0, the SEE is almost constant and slightly less than 0.3, but the value of the information function is the range from 13 to 15 (approximately constantly). Then the estimated marginal reliability for this range is 1−0.322 0.91. The Marginal Reliability for Corresponding Pattern Scores provided by the IRTPRO is 0.92. This means that for latent values greater than -1.5 the values of the indicator variables are the most reliable. However, outside this range of −1.5 to 3.0 marginal reliability decreases and the SEE increases. Thus, if a more precise GSE scale was desired within this range or across more of the continuum, then more items are need to be added to the scale to meet the desired information or level of expected SEE.
To summarize, the 13-item five-category scale provides precise estimates of the scores (information 14, marginal reliability 0.92, expected SEE 0.3) for a broad range of the continuum, −1.5 to 3.0. The maximum amount of information (precision) is approximately 15 around latent trait estimates 1.3. However, precision and expected SEEs around score estimates worsen outside of this range. To improve score estimates beyond this range it is need to write additional items that have thresholds below −1.5.
According to the IRT analysis, the following conclusions can be drawn: 1. Analyzing the characteristic functions (see Figure 1) of the questionnaire questions, it is possible to note that the probability of choosing the response of category 0 (very low level (initial level) for all 14 distractors (OC1, OC2, OC3, PC1, PC2, PC3, PC4, RPP1, RPP2, RPP3, LLD1, LLD2, LLD3, LLD4) with = -3 is within the approximate range from 0.75 to 1.0 (fairly high limits). This means that a small number of teachers assesses their level of digital competency according to all distractors at the initial level, which is a very positive factor at the present time. The higher the level of the teachers’ digital competences, the smaller is the probability to choose from the category 0 a response, which is completely natural. For most distractors, the probability of choosing from the category 0 response falls to zero for teachers with an average level of competencies ( = 0), the exception is for the PC1, PC3, RPP3, LLD3, LLD4 distractors, for which the probability of choosing a category response from 0 equals to = -3 . This means that the competencies that correspond to these distractors are not yet well formed even among teachers with an average level of general digital competence. 2. The probability to choose a response from the category 4 (high level (expert level)) for all 14 distractors with = 3 is within the approximate range from 0.2 to 0.8. This means that teachers who generally have a high level of overall digital competence (or believe that it is of such a level), in the context of exact distractors, have a very miscellaneous level of preparedness. The attention should be paid to the development of competencies that correspond to distractors for which the corresponding probability is less than 0.5. These are PC1, PC3, RPP2, RPP3, LLD3, LLD4. 3. If the graphs of characteristic functions for categories 1, 2 and 3 reach their maximum somewhere in the middle of the scale from -3 to 3, then this is normal from the point of view of the IRT analysis. But as our study has revealed that the curve 3 for individual distractors (PC1, PC3, RPP2, RPP3, LLD3, LLD4) reaches its maximum at the right end of the scale for . This means that teachers who are considered to having the high level of general digital competence, in fact, the level of their competence that corresponds to the specified distractors is not sufficient. 4. The particular concern is caused by the competences with responses from the categories 1, 2 and 3 have a less probability than 0.5 and when the maxima of these probabilities are shifted to the right. These are the competences: PC1, RPP1, RPP2, RPP3, LLD2, LLD3. The displacement of the maximum of probabilities, shifted to the right, means that teachers who are considered to have an average level of competences, in general, have an inadequate level of preparedness of these competencies. 5. From the above mentioned, the level that correspond to the OC1, OC2, OC3, PC2 and LLD1 distractors considered to be satisfactory from the point of probabilistic statistical analysis. 6. When analyzing S- χ 2 Item Level Diagnostic Statistics, we came to the conclusion that the PC1 item should be removed from the questionnaire (see Table 3 and Table 4). Indeed, if we analyze Table 1 at the content level, the attention can be drawn to the fact that the PC1 and PC3 distractors concern in fact to one competence, which is realized in different activity directions. 7. The analysis of the information functions of the questionnaire (see Figure 2) shows that items PC2, RPP1 and LLD4 were not informative enough in the general context of digital competencies evaluation. In order to do the repeated research on general digital competence after the practical implementation of the strategy for its formation or individual stages of this strategy, these distractors should be corrected. 8. From the graph of the general information function (see Figure 3) it is clear that the IRT analysis gives the sufficiently complete information about the general digital competency of the teachers. Only in cases of very low competence or close to it the value of the general information function is low and according to it, the standard error of estimation (SEE) is high. This indicates the fairly good selection of distractors for this study. 3.3
teachers in the process of their lifelong learning at Ternopil Volodymyr
From the study, it follows that teachers with different levels of digital competency of professional development do not have well-formed competencies such as: working with other educators to implement educational projects, joint development of educational resources, appeal to others to improve their digital pedagogical competence, provision learning opportunities for colleagues, the use of online learning opportunities.
Proceeding from this, approaches to the development of digital competencies in the
process of improving of teachers' qualification at the training center "Educational
Innovation" of Ternopil Volodymyr Hnatiuk National Pedagogical University were
developed. They are based on a model for teaching teachers throughout their lives
based on the development of digital competencies [
Proposed approaches to the professional development of digital competences of teachers in the process of their lifelong learning include groups of criteria for the planning and development of organizational communications, engagement and professional co-operation, assessment and reflexive pedagogical practice, sustainability, and professional development throughout life.
In the process of planning and developing organizational communications, attention is paid to both the contemporary national and world context and the individual experience of developing digital competences of teachers in the process of their professional development and lifelong learning, namely: ─ combining a subject of the learning with context in which teachers work at the level of school, community, region; ─ correlations of qualification improving on the development of digital competences of teachers in the process of their professional development and lifelong learning with standards, programs and goals at the school, community, region, and state levels; ─ the use of digital technologies for communication of institution and teacher with other teachers and pupils.
The group of criteria for "engagement and professional co-operation" envisages an active role for teachers in professional co-operation, community building and motivation to share their pedagogical experience: ─ collaboration with other educators for the implementation of educational projects; ─ joint development of educational resources; ─ supporting professional co-operation, providing learning opportunities of the learning for colleagues; ─ the creation and development of professional communities with horizontal links to ensure mutual learning and discussion of new ideas; ─ appealing for help to others to improve their digital pedagogical competence; ─ searching study materials for continuing professional development, using online learning opportunities.
In the group of criteria "evaluation and reflective pedagogical practice" the emphasis is on formal assessment, qualitative feedback and constant reflection: ─ demonstration of the service's compliance with the stated objectives and learning outcomes; ─ the use of digital technologies for individual and collective reflexive pedagogical practice; ─ feedback opportunities for those who take part in the improving of qualifications; ─ discussion of specific features related to the received knowledge, materials or skills that will be demonstrated by a successful transition to the implementation of professional activities; ─ adding participants to the assessment of their knowledge and skills. The group of criteria for "sustainability and professional lifelong development" provides post-support, facilitates better motivation of educators for lifelong learning and helps in building an individualized trajectory of professional growth in the field of digital technologies, namely: ─ detailing of further steps after training that teachers need to apply in a new environment; ─ proposals for continuing education through information and technical post-support; ─ provision of training opportunities for colleagues; ─ use of online learning opportunities (massive open online courses, webinars, etc.).; ─ advising on the implementation of educational innovations.
These approaches are already being implemented in practice in the process of qualification improving of teachers and their lifelong learning at the international educational training center of Ternopil Volodymyr Hnatiuk National Pedagogical University. 4
To develop approaches to assessing the professional development of digital competences of teachers during their lifelong learning, levels of their formation were determined, as well as relevant criteria and indicators.
The results were processed based on the theory of modeling and parametrization of tests IRT. We can state the appropriateness of choosing the standardized statistics LD χ2 and statistics M2, presented in IRTPRO.
On the basis of the conducted research, the approaches to the professional development of digital competence of teachers in the process of their lifelong learning are proposed, which include the following groups of criteria: planning and development of organizational communications, engagement and professional cooperation, assessment and reflective pedagogical practice, sustainability and professional development throughout life.
Among the main vectors of the strategy of professional development of teachers in the context of the development of digital competencies, it should be noted: the creation and development of professional communities with horizontal links to ensure mutual learning and discussion of new ideas; the use of digital technologies and resources in the learning process, modeling of the learning process, oriented on results and educational projects.