Validation of the Digital Teaching Competence Questionnaire (COMDID-A) in the Mexican context

Abstract

We present a validation of the Digital Teaching Competence Questionnaire (COMDID-A), used to measure the level of digital competencies of teachers at the Autonomous University of Chiapas. A documentary review of the existing instruments to measure the digital competencies of teachers was conducted. The COMDID-A instrument was selected due to its focus on the evaluation of teachers' digital competences in the university context, as well as its adaptability to different cultural and linguistic contexts. Subsequently, a thorough analysis of the instrument was conducted, based on theoretical references and the experience of experts in the area of education. Adjustments and modifications were made to the original instrument to adapt it to the Mexican context and improve its relevance and reliability. The results obtained indicate that the COMDID-A instrument is dependable and relevant for its use in the Mexican context. Quantitative and qualitative analyzes show that the instrument is capable of effectively measure digital competencies.

Keywords

• Cross-cultural adaptation
• expert judgment
• content validity
• digital competence1

1. Introduction

After the outbreak of the COVID-19 pandemic, the prevailing need to train teachers in digital skills became evident [ 1 ]. Although the pandemic took many higher education institutions by surprise, the incorporation of digital technologies into classrooms was already underway, although insufficiently [ 2 ]. The emerging use of technologies to mitigate the effects of the global shutdown revealed that, in many cases, the assessments conducted were not adequate to determine the level of intervention required and the areas where the need for training is most critical [ 3 ]. This scenario has resulted in numerous trainings which not meet the specific demands related to the incorporation of technologies in the education of students [ 4 ]. In addition, the lack of a strategic focus on teacher training has produced an unequal adoption of digital tools, which in turn affects the quality of education [ 5 ]. Therefore, it is key not only to identify areas for improvement, but also to develop a teacher training model that is comprehensive, flexible and adapted to the specific context of each educational institution [ 6 ].

In this sense, training in digital skills must go beyond mere technical instruction; it must incorporate pedagogical elements that allow teachers to effectively apply technologies in their educational practice [ 7 ]. This holistic approach will not only improve the quality of teaching, but will also contribute to a more inclusive and equitable education, preparing students for the challenges of the 21st century [ 8 ]. To realize this comprehensive approach in teacher training, it is key to have assessment tools that reflect these complexities [ 9 ]. In this regard, there are different proposals to evaluate the level of digital competence, as well as various reference models or performance standards [ 10–15 ]adopted by some Latin American countries.

In this context, we have chosen to use the Digital Teaching Competence Questionnaire (COMDID-A) prepared by Lazaro-Cantabrana et al. [ 16 ] to evaluate the self-perception of Spanish teachers [ 17 ]. This selection was based on their multidimensional approach, which encompasses four key dimensions of teaching [18]. It encourages teacher self-reflection and autonomy, provides instant feedback, contrasted with other instruments [ 10–15 ]. In addition, it has been adapted in other countries, including Chile [ 16 ]. This provides an added value for the Latin American context [19]; however, its specific adaptability to the Mexican environment had not yet been evaluated. The purpose is to evaluate digital teaching competencies and identify the condition of the teachers at the Autonomous University of Chiapas (UNACH). Therefore, it/they was subjected to various tests such as content, concurrent and construct validity, as well as idiomatic and linguistic validity. By submitting COMDID-A to this process, we intend to provide it with greater solidity and validity, to be an suitable tool to evaluate the digital competencies of UNACH teachers [20,21].

1.1. Digital competences and COMDID-A

Most authors define digital competencies as an amalgam of skills, knowledge and skills which enable individuals to employ digital technologies effectively and ethically. In the academic field, these competencies are key for teachers to efficiently incorporate digital technologies into their pedagogy, contributing in this way to raising the educational level [22]. To do this, it is necessary to promote favorable attitudes toward the use of digital technologies and the ability to adapt to constantly changing technological innovations.

In order to assess the breadth and depth of digital competencies, multiple approaches to measurement have emerged[ 10–15 ]. Notably, self-perception stands out as an essential instrument, since it allows teachers to consciously identify their own level of competence in this field [ 11,23,24 ]. For the study case, and responding to the need to evaluate the self-perception of digital teaching competence (CDD) of teachers of Universidad Autónoma de Chiapas, COMDID-A becomes a reference [24]. This instrument is organized around four dimensions: D1. Didactic, curricular, and methodological approach (6 items); D2. Planning, Organization and Administration of Digital Technology Resources and Spaces (5 items); D3. Relations, Ethics and Security (5 items); D4. Personal and Professional Development (6 items). Altogether, the questionnaire includes 22 items that use a five-point Likert scale to determine different degrees of CDD (non-initiated, beginner, intermediate, expert, and transformer)[ 16 ].

This questionnaire has been applied in multiple contexts and there are several researches and publications that address its implementation and validation [ 16,17,25 ] specifically, to validate the factorial structure and validity of the construct. Palau et al. [ 17 ] Conducted a core component analysis to simplify the dataset and identify the four dimensions; however, this was intended for the European context. For the Latin American environment, there is a study in Chile, which adapted and applied it through focus group [ 16 ]. To address the need for a contextualized evaluation of CDD and to have versions matching the language and sociocultural characteristics of the Mexican population, this study focuses on the process of cross-cultural adaptation of the COMDID-A instrument, developed in Spain.

2. Methodology

This study was conducted at Universidad Autónoma de Chiapas (UNACH) as a result of the need to evaluate the CDD level, to develop a model of professional training in the institution. Initially, a documentary review was conducted to identify the most appropriate instruments to measure these competencies. Within this framework, COMDID-A has been chosen as an essential evaluative instrument; developed by specialists in the field of educational technology at the Universitat Rovira i Virgili in Tarragona, Spain [ 16 ]. This instrument stands out for its ability to measure teachers’ self-perception in four key areas through 22 descriptors and four levels of development, and is especially applicable for self-assessments in academic contexts [ 17 ]. The goal is to evaluate digital skills to determine the current level among UNACH teachers.

Valida&on Stages Stage 1: Transcultural Adapta%on

Stage 2: Content Validity

Opinion of3 Experts in the Area

Iden4fica4on of confusingques4ons.

Propose adapta4ons

Linguis4c and Idioma4c.

Qualita4ve Review of

Assessment

Organiza4on and Classifica4on according to COMDID Integrate results in Visual

Format (table)

Terminology Consistency and

Accuracy Integra4on with cross table

Implementa4on of prac4cal guide adapted by EscobarPérez and Cuervo-MarHnez

Adding Observa-on

Sec-on Integra4on in Matrix

Summary for each Dimension. Screening for

Item Adapta4on

Digitaliza4on of Resultsand Website for Distribu4on of

Instrument Selec4on of 16 Experts and Instrument

Implementa4on Assessment ofJudge

Consistency

Data collec4on.

Quan4ta4ve Validity using Fleiss’ Kappa

Iden%fy COMDID Items for Change

The choice of this instrument was based on its multidimensional approach, which covers four key aspects of teaching [26]. In addition, its adaptation in other countries, including Chile [ 16 ], adds additional value for its use in Latin American contexts [19]. However, its adaptability to the Mexican environment has not yet been applied. Since the COMDID-A instrument originated in a European context, its adaptation to the Mexican environment was imperative, culturally as well as linguistically. To achieve this, the study was divided into two interconnected phases (see Figure 1). The first stage focused on the adaptation and validation of the instrument, using the COMDID-A rubric as proposed by Lázaro-Cantabrana et al. (2018). The adaptation of the questionnaire to the Mexican environment was key. Two validation phases were conducted: The first focused on cross-cultural adaptation (linguistic, cultural, and idiomatic) which involved the participation of three experts in the educational and linguistic field. The second consisted of content validity through the judgment of 16 experts, using the proposal of Escobar-Perez & Cuervo-Martinez [27]. This process made it possible to identify the dimensions that required contextual adaptation, and the results were verified by Fleiss K.

2.1. Phase 1 Cross-cultural adaptation

Ensuring the validity of an instrument is a constant concern among researchers. Over time, validity has been interpreted in various ways and from different fields of study [28,29]. However, it remains crucial for the choice and use of an instrument. Specifically, the question is whether the instrument really evaluates what it intends to measure [30]. To achieve an adequate linguistic, linguistic, and cultural equivalence in an instrument, it needs to be used in different cultural contexts. There are theoretical proposals that serve as a guide and highlight the importance of the transition from an instrument from one culture to another [30–32]. This transition is called transcultural adaptation of the instrument [33]. While in Europe and in English-speaking countries this process is widely valued, in Latin America it is sometimes not given the necessary importance[30]. The lack of adequate procedures for translating and adapting the instruments has led to some research being considered invalid. For COMDID-A to adapt properly to the Mexican context, it was essential to have the opinion and experience of experts in the linguistic and educational field [34]. It was decided to consult three recognized UNACH teachers with wide experience in areas relevant to the study. Table 1 shows a brief description of each expert’s experience and specialty:

Experience

Ph.D. in Philosophy and Educational Sciences

Universidad Complutense De Madrid

Professor of the Faculty of Humanities Campus VI, Universidad Autónoma de

Chiapas. Expert teacher in Psychopedagogy and development of qualitative instruments.

PhD. in Contemporary Philosophy

Benemérita Universidad Autónoma de Puebla (BUAP), México. Professor of the

Faculty of Humanities, Campus VI, Universidad Autónoma de Chiapas. 16 years in teaching, expert in the phenomenology of education.

PhD. D. in Education

Campus Tuxtla, Universidad Autónoma de Chiapas.

32 years in teaching, expert in Linguistics and Languages. Faculty member of the

School of Language.

An e-mail was sent to each of the specialists with a formal invitation to participate. The COMDID-A instrument was attached to this e-mail requesting its analysis and possible proposals for modification. The guidelines provided required that they focused on the linguistic and idiomatic parts based on the following criteria: The first focused on identifying questions that could generate confusion in the Mexican context, and the second on proposing adjustments for them. After receiving their assessments, a meticulous review was conducted to discern which modifications to incorporate and how to do so, ensuring the relevance and clarity of the instrument in Mexico. The contribution of these experts not only strengthens the linguistic validity of COMDID-A, but also guarantees its cultural adaptation and specificity in the Mexican educational context. The reflections of each expert were integrated into a cross table. It is important to mention that, although observations were scarce, they were essential. For example, it was perceived that experts agreed to observe the same word twice, which could produce misinterpretations in the context where the instrument would be applied.

The proposal of Beaton et al. has been taken as a reference. [35] They established phases to ensure that the adapted questionnaire is conceptually, idiomatically, semantically, and operationally equivalent to the original. Having a Spanish version helped to ensure that the translation process did not have any major problems; however, there were still discrepancies in the language of the culture itself; therefore, the contribution of the experts helped to confront and compare the versions to identify and resolve discrepancies. Consistency and precision were sought in the terminology used, with respect to the original content of COMDID-A [35,36].

To ensure fidelity in both language and concepts, a stage of adaptation to Mexican Spanish was implemented through the judgment of experts who did not have prior knowledge of the instrument in its original version. This made it possible to detect and correct possible deviations in interpretation or conceptual meaning in the consolidated version of the instrument. The participation of these experts not only reinforces the linguistic validity of COMDID-A, but also ensures its adaptation to the culture and particularities of the educational field in Mexico.

2.2. Phase 2: Content Validity

Every instrument must go through a validation process to ensure that it is valid, reliable and that accurately measures what is proposed [37,38]. A validated instrument allows the generalization of findings [37,38], improves the quality of the study, not only increases the credibility of the study, but also facilitates efficient data collection and lays the foundation for future studies [39]. To carry out the second phase of the investigation, we adapted the proposal of Escobar-Perez and Cuervo-Martinez[27] , who establish a method for the judgment of experts and that through a practical guide carries out the process of expert judgment that includes: 1. Prepare instructions and spreadsheets 2. Select the experts and train them 3. Explain the context 4. Enable discussion 5. Establish agreement between experts by calculating consistency.

These steps are recommended by several authors [40,41], and are considered essential for conducting an expert judgment effectively. A space for general observations was added to this guide; this space ensured that the instrument was applicable to the given Mexican context. This enriched what was already established in Phase 1. Subsequently, the instrument was digitized using the LimeSurvey, software which helped its distribution and the data collection. This process ended with the creation of the www.competenciadigital.com.mx website, which served as a platform for managing the tool and the database.

The next step was the thorough selection of expert judges, an essential component to ensure the validity and reliability of the study. Following the best practices in the selection of judges methodologically, it was decided to invite specialists with a solid academic background and a vast experience in the field of educational technology [27,40,42,43]. In total, 16 experts from various states of the Mexican Republic were added (see Table 2). The inclusion of experts from different geographical areas and universities allowed for a more complete evaluation, addressing various aspects of the subject in question. This selection strategy was based on rigorous methodologies previously established by various authors [27,40,42,43], thus ensuring that the process was aligned with high quality academic standards. This holistic approach not only strengthened the validity of the study, but also set a precedent for future research in the field. Each expert was contacted by email, being mostly members of the Inter-Institutional Committee for the Evaluation of Higher Education (CIEES) with a specialty in Educational Technology. In the mail, a link to the digitized instrument was provided, accompanied by a detailed protocol, the theory supporting the instrument, a specific timeframe to complete the task, as well as clear definitions of the evaluation criteria.

After the end of the evaluation period, which lasted five months from the first contact, the data obtained was thoroughly collected. To guarantee the quantitative validity of the COMDID-A instrument, a mathematical analysis process of the collected data was implemented, and in this framework, the Fleiss K coefficient emerged as a crucial statistical indicator. Fleiss K is a metric that evaluates the degree of agreement between multiple judges or evaluators [44–46] . It is used specifically to measure the consistency of the classifications awarded by different judges to the same subjects [47,48] . A high value of the Fleiss K coefficient indicates a higher concordance among the judges, which, in turn, reinforces the reliability of the instrument in question [46].

To carry out this comprehensive quantitative analysis, a team of mathematics experts was formed. This team was led by an academic recognized in the National System of Researchers (SNI), level II and an advanced student of the Bachelor of Mathematics. The analysis focused on the evaluation of four key dimensions: Didactics, Curriculum and Methodology; Planning, Organization and Management of Digital Technological Spaces and Resources; Relational, Ethics and Security; and Personal and Professional. Each dimension was examined under four aspects: Clarity, sufficiency, coherence, and relevance. Each dimension-aspect pair included between 5 and 7 questions whose answers could be: High level, Moderate level, Low level or Does not meet the criteria.

This evaluation process gives rise to 16 matrices, which contain all the evaluations made by the experts. The results obtained in the first matrix are presented below.

The objective was to assess the level of concordance in the evaluations of the 16 judges. For this purpose, two statistical coefficients were used: Kendall’s coefficient W of and Fleiss Kappa coefficient. Considering that Kendall’s W was designed for ordinal trials, its adjustment was used for repeated trials; however, the results were mostly non-significant, which led to its dismissal.

The Fleiss Kappa coefficient was identified as the most suitable option for this study, especially since the collected valuations are presented in nominal form. Two hypotheses were formulated: The null hypothesis ($H_0$), which holds that there is no significant real agreement beyond chance, and the alternative hypothesis ($H_1$), which states that the observed agreement is statistically significant. The p-value was used to evaluate the evidence against the null hypothesis and to determine the statistical significance of the agreement observed among the judges. In this way, a summary matrix was constructed using the calculation of the Fleiss Kappa for each dimension-aspect pair, providing an integral view of the results obtained.

It is important to note that, with one exception, the results obtained are statistically significant, since the corresponding p-values do not exceed the level of significance established in 0.05. To better understand the level of agreement between the judges, we resort to the interpretation of the Fleiss Kappa proposed by Altman (1991). Noting that the median of \( K \) values is 0,352, Altman suggests that this reflects a generally weak level of agreement among judges. In other words, there is a certain discrepancy in the evaluations conducted by the different judges. To illustrate the level of agreement, we created Table 5 which we have called the "Frequency Table of Agreement Levels according to the Fleiss Kappa":

This table allows for a quick and effective visualization of how agreement levels are distributed among judges. For example, it can be observed that most judges (9 out of 16) have a "Weak" level of agreement, while only one reaches a "Good" level of agreement. This highlights the need to review and possibly adjust the evaluation tool to improve consistency among judges. Finally, after this rigorous process of evaluation and analysis, the final measurements were conducted, and the corresponding results were obtained. These results will serve as a basis for future research and methodological adjustments.

3. Results and Discussion

The research was able to adapt and validate the COMDID-A instrument for its application in Mexico. The results show that the instrument is dependable and relevant to evaluate the digital competencies of teachers at Universidad Autónoma de Chiapas. The method proposed by Escobar-Pérez and Cuervo-Martinez was used to validate and adapt COMDID-A, [27],A comment section was added to collect specific observations from the experts. This approach made it possible to obtain qualitative data that enriched the qualitative phase of the research. The comments of the experts were classified according to COMDID-A items and dimensions, which helped to identify patterns and relevant coincidences in specific items.

1.2 Processing of information 1.3 Management, analysis and creation of knowledge of information and creation of knowledge

Use digital technologies to Use digital technologies to

increase motivation and increase motivation and facilitate learning for facilitate learning for students with NEE students with NEI

Design EA activities which Design teaching-learning

involve the use of digital (EA) activities involving the technologies use of digital technologies

Serve as a model for other Be a reference for other professionals on the professionals on the responsible and safe use of responsible and safe use of digital technologies digital technologies

Categories such as "Meaning" and "Structure" were established to organize the data. Within "Structure," subcategories such as "Abbreviation" and "Grammar” were created. Items that required changes were moved to "Change Formats," placing them in the corresponding categories and subcategories. This meticulous process allowed to have a visual map of the items to be modified. A relevant change was the adaptation of the term "Special Educational Needs (NEE, for the Spanish acronyms)" to "Inclusive Education Needs (IND, for the Spanish acronyms)" for the Mexican context. See table 4. This detailed approach ensured that the COMDID-A instrument was well-founded and adapted to the Mexican context, ensuring its validity and reliability. A concordance was observed between the qualitative responses and the dimensions evaluated. In addition, tables were developed to improve the understanding of the changes made, highlighting the adaptation of terms and parameters to measure teaching strategies.

The qualitative results meet the objective of adapting and validating the instrument, since final actions were determined for item changes based on qualitative and quantitative analysis (Fleiss Kappa). The next task is to manage updating the instrument for its application in Mexico. See Figure 2 showing the visual overview of the instrument items that are proposed for changes.

4. Conclusions

The research focused on evaluating the digital competencies of teachers at Universidad Autónoma de Chiapas. It began with a documentary review to identify appropriate instruments to measure these competencies. The COMDID-A instrument developed by Lazaro-Cantabrana et al. [ 16 ] in Spain was considered as the most appropriate; therefore, a cultural and linguistic adaptation was required to be implemented in Mexico. To validate the instrument in the Mexican context, two types of validations were conducted: Idiomatic and linguistic, and quantitative through expert judgment. In the idiomatic validation, three experts were consulted to adapt the instrument to the linguistic conditions of Mexico. For content validity, 16 experts in educational technology were resorted to, and statistical methods such as Kendall's W coefficient and Fleiss's Kappa coefficient were applied.

It is essential to understand that the validation of an instrument is not an isolated process but must consider its applicability in a specific context. This thorough approach ensures that the instrument is both applicable and dependable in the Mexican context. The research not only seeks to evaluate the digital competencies of teachers, but also to contribute to the body of knowledge in the field of educational technology and teacher training in Mexico. This Mexican version of COMDID-A can be considered equivalent to the original; it is linguistically, semantically, and culturally adapted to the Mexican context. The authors are aware that any validation process is a process that requires testing other types of validity to ensure the validity of the instrument.

This research lays the foundations for future studies and the implementation of teacher training strategies in digital competencies, aligned with the needs and context of the UNACH and potentially applicable in other educational institutions in the country. That is to say, this method improves efficiency in data collection and serves as a basis for future research, highlighting the importance of its application in the various fields of study that should not be ignored.

Acknowledgements

The authors thank the experts who participated voluntarily and anonymously in this study. This study was partially funded by the Research Direction of the Universidad Peruana de Ciencias Aplicadas (UPC). [18] Lazáro Cantabrana JL, Gisbert Cervera M. Elaboración de una rúbrica para evaluar la competencia digital del docente. Rev Ciéncies LÉducació 2015:19. [19] Cisneros-Barahona AS, Marqués-Molias L, Samaniego-Erazo N, Mejía-Granizo CM. La Competencia Digital Docente. Diseño y validación de una propuesta formativa. Pixel-Bit2 2023;68:7–41. [20] Creswell JW. Research design: Qualitative, quantitative, and mixed methods approaches. 4th ed. Thousands Oaks, CA: SAGE Publications, Inc.; 2014. [21] Sireci SG. The construct of content validity. Soc Indic Res 1998;45:83–117.

https://doi.org/10.1023/a:1006985528729. [22] Paz Saavedra LE, Gisbert Cervera M, Usart Rodríguez M. Competencia digital docente, actitud y uso de tecnologías digitales por parte de profesores universitarios. Pixel-Bit, Rev Medios y Educ 2022:91–130. https://doi.org/10.12795/pixelbit.91652. [23] Janssen J, Stoyanov S. Online Consultation on Experts "Views on Digital Competence 2012. [24] Lázaro Cantabrana JL, Gisbert Cervera M. Elaboració d’una rúbrica per avaluar la competència digital del docent. Univ Tarraconensis Rev Ciències l’Educació 2015;1:48. https://doi.org/10.17345/ute.2015.1.648. [25] Silva J, Usart M, Lázaro-Cantabrana J-L. Competencia digital docente en estudiantes de último año de Pedagogía de Chile y Uruguay TT - Teacher’s digital competence among final year Pedagogy students in Chile and Uruguay. Comunicar 2019;61:33–43. [26] Lázaro Cantabrana JL. La competència digital docent com a eina per garantir la qualitat en l’ús de les TIC en un centre escolar. vol. 1. 2015. https://doi.org/10.17345/ute.2015.1.667. [27] Escobar-Pérez J, Cuervo-Martínez Á. Validez de contenido y juicio de expertos: Una aproximación a su utilización [Content validity and expert judgement: An approach to their use]. Av En Medición 2008;6:27–36. [28] Aiken LR, Yang W, Soto M, Segovia L, Binomial P, Miller JM, et al. Diseño y validación de un cuestionario para analizar la calidad en empleados de servicios deportivos públicos de las mancomunidades de municipios extremeñas. Educ Psychol Meas 2011;7:181–92. https://doi.org/10.1177/0013164412473825. [29] Cho J. Validity in qualitative research revisited. Qual Res 2006;6:319–40.

https://doi.org/10.1177/1468794106065006. [30] Gallardo-Echenique E, Marqués Molias L, Gomez Cruz OD, De Lira Cruz R. Cross-cultural adaptation and validation of the “student communication & study habits” questionnaire to the mexican context. Proc - 14th Lat Am Conf Learn Technol LACLO 2019 2019:104–9. https://doi.org/10.1109/LACLO49268.2019.00027. [31] Arribas A. Adaptación Transcultural de Instrumentos. Guía para el Proceso de Validación de

Instrumentos Tipo Encuestas. Rev Científica La Asoc Médica Bahía Blanca 2006;16:74–82. [32] Lira MT, Caballero E. Cross-Cultural Adaptation of Evaluation Instruments in Health: History and Reflections of Why, How and When. Rev Medica Clin Las Condes 2020;31:85–94. https://doi.org/10.1016/j.rmclc.2019.08.003. [33] International Test Commission (ITC). ITC Guidelines for Translating and Adapting Tests. 2nd ed. [Www.InTestCom.Org]: ITC; 2016. [34] Cardoso Ribeiro C, Gómez-Conesa A, Hidalgo Montesinos MD. Metodología para la adaptación de instrumentos de evaluación. Fisioterapia 2010;32:264–70. https://doi.org/10.1016/j.ft.2010.05.001. [35] Beaton D, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the Cross-Cultural

Adaptation of the DASH & QuickDASH Outcome Measures. Toronto: 2007. [36] Beaton D, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25:3186–91. https://doi.org/10.1097/00007632-200012150-00014. [37] Sireci SG. The Construct of Content Validity. Soc Indic Res 1998;45:83–117.

https://doi.org/10.1007/sl. [38] Almanasreh E, Moles R, Chen TF. Evaluation of methods used for estimating content validity.

Res Soc Adm Pharm 2019;15:214–21. https://doi.org/10.1016/J.SAPHARM.2018.03.066. [39] Hambleton RK, Patsula L. Adapting tests for use in multiple languages and cultures. Soc Indic

Res 1998;45:153–71. https://doi.org/10.1023/A:1006941729637. [40] Skjong R, Wentworth BH. Expert Judgment and Risk Perception. Proc. Elev. Int. Offshore Polar Eng. Conf., vol. IV, Stavanger, Norway: International Society of Offshore and Polar Engineers; 2001, p. 537–44. [41] de Arquer MI. Fiabilidad Humana: métodos de cuantificación, juicio de expertos. 1995. [42] Cabero J, Llorente M del C. La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información y comunicación (TIC) [The expert’s judgment application as a technic evaluate information and communication technology (ICT)]. Eduweb Rev Tecnol Inf y Comun En Educ 2013;7:11–22. [43] Urrutia Egaña M, Barrios Araya S, Gutiérrez Núñez M, Mayorga Camus M. Métodos óptimos para determinar validez de contenido. Rev Cuba Educ Medica Super 2015;28:547–58. [44] Cerda Lorca J, Villarroel Del P. L. Evaluación de la concordancia inter-observador en investigación pediátrica: Coeficiente de Kappa. Rev Chil Pediatr 2008;79:54–8. https://doi.org/10.4067/s0370-41062008000100008. [45] Torres J, Perera V. Cálculo de la fiabilidad y concordancia entre codificadores de un sistema de categorías para el estudio del foro online en e-learning. Rev Investig Educ 2009;27:89– 103. [46] Falotico R, Quatto P. Fleiss’ kappa statistic without paradoxes. Qual Quant 2015;49:463–70.

https://doi.org/10.1007/s11135-014-0003-1. [47] López A, Galparsoro DU, Fernández P. Medidas de concordancia : el índice de Kappa. Cad Aten

Primaria 2001:2–6. [48] Gwet KL. Large-Sample Variance of Fleiss Generalized Kappa. Educ Psychol Meas 2021;81:781–90. https://doi.org/10.1177/0013164420973080. [49] Altman DG. Practical Statistics for Medical Research. 1991.

References

[1] Molina Montalvo HI , Macías Villareal JC , CEpeda Hernández AA . Educación en tiempos de COVID- 19 : Una aproximación a la realidad en México: experiencias y aportaciones . Comunicaci . Ciudad de México: 2022 .

[2] Fernández Escárzaga J , Gabriela J , Varela D , Lorena P , Martínez M . De la educación presencial a la educación a distancia en época de pandemia por Covid 19 . Experiencias de los docentes. Rev Electrónica Sobre Cuerpos Académicos y Grup Investig 2020 ; 7 : 87 - 110 .

[3] Cruz-Aguayo Y , Hincapé D , Rodríguez C. Profesores a prueba: Claves para una evaluación docente exitosa . 2020 . https://doi.org/10.18235/0002149.

[4] Guerrero I , Kalman J . La inserción de la tecnología en el aula: Estabilidad y procesos instituyentes en la práctica docente . Rev Bras Educ 2010 ; 15 : 213 - 29 . https://doi.org/10.1590/S1413-24782010000200002.

[5] Mendoza R , Bellodas M , Ortiz C , Puelles L , Asnate E , Zambrano J . Desafíos interdisciplinarios para los docentes en el aprendizaje virtual . 2023 .

[6] Balladares-Burgos J , Valverde-Berrocoso J . El modelo tecnopedagógico TPACK y su incidencia en la formación docente: una revisión de la literatura . RECIE Rev Caribeña Investig Educ 2022 ; 6 : 63 - 72 . https://doi.org/10.32541/recie. 2022 . v6i1 . pp63 - 72 .

[7] Díaz Chamorro CM. El Modelo Tpack Como Método Pedagógico Para El Desarrollo De Competencias Digitales En Los Docentes De La Unidad Educativa “Víctor Mideros .” 2023 .

[8] Esquerre Ramos LA , Pérez Azahuanche MÁ . Retos del desempeño docente en el siglo XXI: una visión del caso peruano . Rev Educ 2021 ; 45 : 0 - 21 . https://doi.org/10.15517/revedu.v45i1. 43846 .

[9] Castañeda L , Esteve F , Adell J . Why rethinking teaching competence for the digital world? Rev Educ a Distancia 2018 : 1 - 20 . https://doi.org/10.6018/red/56/6.

[10] Agreda M , Hinojo MA , Sola JM . Diseño y validación de un instrumento de evaluación de competencia digital docente. Pixel-Bit, Rev Medios y Educ 2016 : 39 - 46 .

[11] Ferrari A. DIGCOMP  : A Framework for Developing and Understanding Digital Competence in Europe . Luxembourg: 2013 . https://doi.org/10.2788/52966.

[12] Reixach E , Andrés E , Ribes JS , Gea-Sánchez M , López AÀ , Cruañas B , et al. Measuring the Digital Skills of Catalan Health Care Professionals as a Key Step Toward a Strategic Training Plan: Digital Competence Test Validation Study . J Med Internet Res 2022 ; 24 . https://doi.org/10.2196/38347.

[13] Restrepo-Palacio S , de María Segovia Cifuentes Y. Design and validation of an instrument for the evaluation of digital competence in Higher Education . Ensaio 2020 ; 28 : 932 - 61 . https://doi.org/10.1590/S0104-40362020002801877.

[14] Zempoalteca Durán B , Barragán López JF , González Martínez J , Guzmán Flores T. Teaching training in ICT and digital competences in Higher Education System . Apertura 2017 ; 9 : 80 - 96 . https://doi.org/10.32870/ap.v9n1. 922 .

[15] Zubieta J , Bautista T , Quijano A . Aceptacio ́ n de las TIC en la docencia : una tipologıa ́ de los académicos de la UNAM. 2012 .

[16] Lázaro-Cantabrana JL , Gisbert-Cervera M , Silva-Quiroz JE . Una rúbrica para evaluar la competencia digital del profesor universitario en el contexto latinoamericano . Edutec Rev Electrónica Tecnol Educ 2018 : 1 - 14 . https://doi.org/10.21556/edutec. 2018 . 63 .1091.

[17] Palau R , Usart M , Ucar Carnicero MJ . La competencia digital de los docentes de los conservatorios. Estudio de autopercepción en España. Rev Electron LEEME 2019 : 24 - 41 . https://doi.org/10.7203/LEEME.44.15709.