Methodology of Using Structural Equation Modeling in Educational Research Liubov Panchenko[0000-0002-9979-0625] National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, 37, Peremohy Ave., Kyiv, 03056, Ukraine lubov.felixovna@gmail.com Abstract. The article deals with the problem of using structural equation modelling (SEM) methodology in educational research. It allows the researcher to build multidimensional models of the phenomena and processes that are being studied. The SEM methodology is based on many well-known methods such as correlation, regression, factor analysis, variance analysis and covariance analysis. The methodology is mainly based on deductive logic, involves the preliminary construction of a structural model of relationships between variables in order to further check for consistency with the experimental data. The article provides an example of using the SEM methodology in educational research for PhD student. An important point in preparing specialists for using SEM is to select or obtain the necessary data sets that are representative and valid. During the research the Ukrainian teacher’s self-efficacy model with SEM methodology was checked, and the obtained results were compared with the research data of the worldwide teacher’s survey – The Teaching and Learning International Survey (TALIS). The lower self-efficacy of Ukrainian teachers, especially in the student engagement block, was showed. Keywords: structure equation modelling, TALIS methodology, Ukrainian teachers, teacher’s self-efficacy, PhD student, AMOS, R. 1 Introduction 1.1 Setting of a problem In recent years, many PhD programs were organized in Ukraine. Qualitative scientific research is impossible without a systematic description of the studied phenomena; multidimensionality of the investigated phenomena requires the use of multidimensional analysis methods that are capable to identify causal relationships, latent factors, etc. A promising area in the field of multidimensional applied analysis is the structural modeling or structural equation modeling, which is becoming an increasingly popular tool for researchers in the field of education, psychology and social sciences [1-5]. Consequently, we consider it very important to train future doctors of philosophy to use it in the educational research. The popularity of the SEM methodology is evidenced by the experiment we conducted. At the request of “structural equation modeling” to search books on Amazon.com (as of March 16, 2013), we obtained 59 items, the graph of which is clearly shown in fig.1a. In the center of the graph (Fig. 1a), where 5 subgraphs can be observed, there is the third edition of the bestseller, Principles and Practice of Modeling by Structural Equations (Rex B. Kline, Principles and Practice of Structural Equation Modeling) [1]. The companion site of this publication provides methodological support and offers download syntax, data and source files for all sample books for execution in three environments EQS, LISREL and Mplus, and a comparison of simulation results. A similar experiment, conducted on March 28, 2019, (Fig. 1b), shows interest growth in structural modeling; we have 157 items. Interestingly, the fourth edition of the same bestseller has the biggest rating there. a b Fig. 1. а. Books on «structural equation modeling», Amazon.com (a – 2013. b – 2019) While solving the scientific problem of training teachers to use the SEM methodology, the following main results were obtained in past author works: the content of the simulation training by the structural equations of specialists in the field of education is revealed; the dynamics of software simulation by structural equations is analyzed; the necessity of including these means in the courses for students and graduates of higher educational institutions of Ukraine that specialize in the field of education and social sciences is substantiated [6]. The syllabuses of PhD SEM courses of leading universities are analyzed (Higher School of Economics, St. Petersburg; University of Amsterdam; University of Vaasa, Finland; University of Mannheim; Iowa State University; Brown University; University of Leuven; School of Education University of Pittsburgh; Oslo University etc). The objectives of the SEM courses are defined as follows: using structural equation modeling methodology to study the problems of social and behavioral science, understanding the strengths and flaws of the method and its limitations, teaching methods of assessment, identification models, testing their validity, interpretation, critical evaluation of scientific publications on this subject, using statistical software to perform structural equation modeling analysis, preparation of research reports in accordance with the standards of research [7]. In preparing specialists for using SEM an important point is to select or obtain the necessary data sets that are representative and valid. We offer our students the survey data from Ukrainian teachers [8-10]. On August 31, 2017, the Ukrainian Association of Educational Researchers completed the All-Ukrainian monitoring “Teaching and Learning Survey on Principals and Teachers of Secondary Education Institutions” (based on the TALIS methodology [11]). The study was conducted within the framework of the project “Teacher” and “Education Reform: Quality Assessment in an International Context”, which is implemented by the All-Ukrainian Foundation “Step by Step” with the support of the Ministry of Education and Science of Ukraine [8]. The study was attended by 3,600 teachers and 201 school principals from 201 schools, representing all regions of Ukraine. The results of the study, according to the OECD policy, are open and accessible. The aim of the article, based on the survey data of Ukrainian teachers, is to check the model of teacher’s self-efficacy with SEM methodology, and to compare obtained results with the research data of the worldwide teacher’s survey – TALIS (2013). 1.2 Analysis of recent research and publications The methodology of structural modeling has received wide recognition in the global community. The study of the basics of structural modeling has become a component of the training of researchers specializing in social sciences [7]. In Russia, the ideas of structural modeling in relation to psychology are reflected in the works of O. Mitina [3] and A. Nasledov [4]. The use of SEM with an emphasis on economic research has been studied by Ukrainian scholar A. Chorny [5]. Unfortunately, in Ukraine, structural modeling is not sufficiently used in educational and social studies in general, and in the training of researchers at universities, in particular. The aspects of the application of the SEM methodology to educational data (TALIS, 2013) are devoted to the following research. A structural equation model of determinants of the perceived impact of teachers’ professional development (the Abu Dhabi application) is reviewed in [12]. How school context and teacher’s characteristics predict distributed leadership is presented in [13]. The invariance of teachers’ sense of self-efficacy measured across countries is reviewed by R. Scherer and others [14]. 2 Results of the study TALIS (Teaching and Learning International Survey) is one of the most prestigious international comparative education projects. The project is dedicated to studying the environment and work conditions of school teachers. It has been implemented since 2008 by a research consortium under the Organization for Economic Cooperation and Development (OECD). 24 OECP countries and partner countries participated in the first wave of TALIS study in 2008, 34 – in the second wave in 2013, and 44 countries plan to participate in 2018 [8; 11]. All-Ukrainian monitoring survey of teaching and learning among school principals and teachers of general educational institutions (according to the methodology All- Ukrainian research on TALIS methodology) is an example of use of international instruments for studying national educational space and identifying the place of the Ukrainian teacher community in the international community educational context. The purpose of the research is to identify and analyze socio-demographic and professional characteristics of Ukrainian teachers and academic staff and the environment of schools on the basis of reliable comparable metrics [8]. 3600 teachers of 5-9 grades of secondary schools (level ISCED 2) and 201 school principals from 201 schools participated in the survey in 2017. Error of simple random sampling is 1.6%, the school sample selection error takes into account design effect is 2.3%. From the Ukrainian teacher’s survey file [9] we selected 3477 lines without missing values for 12 variables that represented the teacher’s self-efficacy (Table 1). Table 1. Variables of teacher’s self-efficacy Variable name Content TT2G34A Get students to believe they can do well in school work TT2G34B Help my students value learning TT2G34C Craft good questions for my students TT2G34D Control disruptive behavior in the classroom TT2G34E Motivate students who show low interest in school work TT2G34F Make my expectations about student behavior clear TT2G34G Help students think critically TT2G34H Use a variety of assessment strategies TT2G34I Provide an alternative explanation, for example, when students are confused TT2G34J Implement alternative instructional strategies in my classroom TT2G34K Get students to follow classroom rules TT2G34L Calm down a student who is disruptive or noisy A. Bandura defines self-efficacy as a personal judgment of “how well one can execute courses of action required to deal with prospective situations” [15]. He names four sources of efficacy beliefs: 1) mastery experiences; 2) vicarious experiences; 3) verbal persuasion; 4) emotional and physiological states. Professional teacher’s self-efficacy, in general, is the perception of a person's own ability to mobilize motivation, cognitive resources and behavioral activity that are needed to control the situation in order to achieve the intended purpose [15-17]. TALIS model for teacher consists of three components of self-efficacy: self-efficacy in classroom management; self-efficacy in instruction; self-efficacy in student engagement. We will conduct a factor analysis for these data. The obtained values of Kaiser- Meyer-Olkin (0.902) and Bartlett’s Test of Sphericity (13308, p <0.001) indicate that factor analysis is a suitable method for these data. The scree plot below shows three factors. Fig. 2. Scree plot: a line plot of the eigenvalues of factors Let us consider the rotated component matrix (Table 2). It demonstrates that the first factor, loading high factor weight of the attributes, is related to the class management, the second one is related to the student’s engagement, and the third one to the instruction. You can also observe that the variable “Craft good questions for my students” is more related to the factor “Student engagement” than the factor “Instruction”. As you see, three factors explain 59.1 % of variability (Table 3). Scientists identify next five steps in SEM application [1; 4]. 1. model formation. The model depicts the graphical views of the researcher about the structure of the variable and latent constructs of ties. At the same time, they decide which parameters should be fixed, and which should be left free. 2. model identification 3. model evaluation 4. checking the consistency of the model 5. model correction by adding new links and eliminating insignificant links. Let us build a model of confirmatory factor analysis with AMOS SPSS (Fig. 3). Table 2. Rotated Component Matrix Factor 1: Class Factor 2: Student Factor 3: management Engagement Instruction Control disruptive behavior in the .795 classroom Calm a student who is disruptive or .773 noisy Get students to follow classroom rules .765 Make my expectations about student .596 behavior clear Help my students value learning .816 Get students to believe they can do well .764 in school work Motivate students who show low .644 interest in school work Help students think critically .448 Craft good questions for my students .443 Provide an alternative explanation, for .785 example, when students are confused Implement alternative instructional .736 strategies in my classroom Use a variety of assessment strategies .730 Fig. 3. Initial teacher’s self-efficacy model in AMOS SPSS Table 3. Total Variance Explained Rotation Sums of Squared Loadings Component Total % of Variance Cumulative % 1 2.571 21.425 21.425 2 2.313 19.274 40.698 3 2.211 18.429 59.127 We got the following results. Number of distinct sample moments: 78; number of distinct parameters to be estimated: 27; degrees of freedom: 78–27=51. Criteria for coherence RMSEA 0.07 <0.08, that is, the model is consistent with the data. Using the data [8], we clearly compared the indicators of self-efficacy of teachers in Ukraine and in the world (Table 4, Fig. 5). The graph shows that self-efficacy of Ukrainian teachers is lower, especially in the student engagement block (variable “Get students to believe they can do well in school work” – difference was 26.4 %, “Help my students value learning” – 26%, “Motivate students who show low interest in school work” – 19.4 %). You can see the resulting teacher’s self-efficacy model in the Figure 4. Fig. 4. Resulting teacher’s self-efficacy model in AMOS SPSS Table 4. Indicators of self-efficacy of teachers in Ukraine and in the world (%) Ukrainian Variables TALIS Difference teachers A. Get students to believe they can do well in school work 59.4 85.8 26.4 B. Help my students value learning 54.7 80.7 26 C. Craft good questions for my students 82.6 87.4 4.8 D. Control disruptive behavior in the classroom 85 87 2 E. Motivate students who show low interest in school work 50.6 70 19.4 F. Make my expectations about student behavior clear 68 91.3 23.3 G. Help students think critically 69.5 80.3 10.8 H. Use a variety of assessment strategies 77 89.4 12.4 I. Provide an alternative explanation, for example, when 78.5 84.8 6.3 students are confused J. Implement alternative instructional strategies in my 87.8 81.9 -5.9 classroom K. Get students to follow classroom rules 92.9 92 -0.9 L. Calm a student who is disruptive or noisy 72 77.4 5.4 Ukrainian teachers TALIS Get students to believe they can do well in school work Implement alternative 95 Help my students instructional strategies 90 value learning in my classroom 85 80 Motivate students who Get students to follow 75 show low interest in classroom rules 70 school work 65 60 Control disruptive 55 Make my expectations behavior in the 50 about student behavior classroom clear Craft good questions Use a variety of for my students assessment strategies Calm a student who is Help students think disruptive or noisy critically Provide an alternative explanation, for example, when… Fig. 5. Comparison of teachers’ self-efficacy in Ukraine and in the world 3 Conclusions and perspectives of further research Measurements that are used in modern educational research are becoming more and more complex. SEM methodology helps researcher determine the effectiveness of educational innovations in different educational contexts, as well as model and study phenomena in their interrelations; understand the influence of latent factors, develop systemic and critical thinking. An important point in training specialists to use SEM is to select or obtain the necessary data sets that are representative and valid. For example, we offer our students such data: All-Ukrainian survey data from Ukrainian teachers. The main criteria for choosing it are: 1) an array of data is freely accessible, 2) it is large (contains 3600 lines), 3) it is accompanied by supporting documentation, 4) the array and documents have Ukrainian and English versions, 5) the array variables are simple and understandable, 6) it is possible to conduct comparative studies with the data of the International Talis Teacher’s Survey. During the research the teacher’s self-efficacy model using SEM methodology were checked, the obtained results were compared with the TALIS survey data (2013). The research demonstrated that self-efficacy of Ukrainian teachers, especially in the student engagement block, was lower. Further development of work in this direction is the creation of teaching and methodological support for modeling by structural equations in the form of a computer workshop in the AMOS and R environments for the training of researchers in the field of pedagogy and social sciences. References 1. Kline, R.: Principles and Practice of Structural Equation Modeling. The Gilford Press, New York (2010) 2. Teo, T., Khine, M.S. (ed): Structural Equation Modeling in Educational Research. Sense Publishers, Rotterdam/Boston/Taipei (2009) 3. Mitina, O.V.: Modelirovanie latentnyih izmeneniy c pomoschyu strukturnyih uravneniy (Modeling latent changes using structural equations). Eksperimentalnaya psihologiya. 1, 131–148 (2008). 4. Nasledov, A.D.: IBM SPSS Statistics 20 i AMOS: professionalnyiy statisticheskiy analiz dannyih (IBM SPSS Statistics 20 and AMOS: Professional Statistical Data Analysis). Piter, SPb (2013) 5. Chornyi, A.: Modeliuvannia latentnykh zminnykh: rozvytok, suchasnyi stan ta perspektyvy (Modeling latent variables: development, current state and prospects). Visnyk Kyivskoho nats. un-tu imeni Tarasa Shevchenka. Ser. Ekonomika. 129, 41–43 (2011) 6. Panchenko, L.F.: Modeliuvannia strukturnymy rivnianniamy yak instrument pedahohichnoho doslidzhennia (Modeling by structural equations as a tool for pedagogical research). Visn. Luhan. nats. un-tu imeni Tarasa Shevchenka: Pedahohichni nauky. 18(277), 108–114 (2013) 7. Panchenko, L.F., Razoronova, M.V.: Modeliuvannia strukturnymy rivnianniamy yak skladnyk pidhotovky doktoriv filosofii (Modeling by structural equations as a component of preparation of doctors of philosophy) Naukovyi visnyk Donbasu. 1‒2. http://nvd.luguniv.edu.ua/archiv/2016/N1-2(33-34)/5.PDF (2016). Accessed 20 Mar 2019 8. Shchudlo, S., Zabolotna, O., Lisova, T.: Ukrainian Teachers and the Learning Environment. Results of All-Ukrainian Monitoring Survey of Secondary School Teachers and Principals (by the TALIS methodology). Trek LTD, Drohobych (2018) 9. Access the full survey by TALIS methodology raw data (in SPSS): TEACHERS_DATA: https://drive.google.com/open?id=1bzh6U7MnOaFSt_1CV1BsQndCuLX_WpWt. Accessed 20 Feb 2019 10. Questionnaires (in ukr): TEACHERS_Questionnaire: https://drive.google.com/open?id=1L6SHvqpMAGPzeLkp8Ksb9E-KXdnkd0sd . Accessed 20 Feb 2019 11. TALIS – OECD Teaching and Learning International Survey – OECD. http://www.oecd.org/education/talis (2019). Accessed 20 Feb 2019 12. Badri, M., Alnuaimі, A., Yang, G., Rashidi, A., Sumaiti, R.: Structural Equation Model of Determinants of the Perceived Impact of Teachers’ Professional Development – The Abu Dhabi Application. SAGE Open. 7(2) (2017). doi:10.1177/2158244017702198 13. Liu, Y., Bellibas, M.R., Printy, S.: How school context and educator characteristics predict distributed leadership: A hierarchical structural equation model with 2013 TALIS data. Educational Management Administration & Leadership. 46(3), 401–423 (2018). doi:10.1177/1741143216665839 14. Scherer, R., Jansen, M., Nilsen, T., Areepattamannil, S., Marsh, H.W.: The Quest for Comparability: Studying the Invariance of the Teachers’ Sense of Self-Efficacy (TSES) Measure across Countries. PLOS ONE. 11(3) (2016). doi:10.1371/journal.pone.0150829 15. Bandura, A.: Self-efficacy mechanism in human agency. American Psychologist. 37(2), 122–147 (1982). doi:10.1037/0003-066X.37.2.122 16. Kremeshna, T.I.: Pedahohichna samoefektyvnist: shliakh do uspishnoho vykladannia (Pedagogical Self-Efficiency: The Way to Successful Teaching). Uman (2010) 17. Krasnoryadtseva, O.M., Kabrin, V.I., Muraveva, O.I., Podoynitsina, M.A., Chuchalova, O.N.: Psihologicheskie praktiki diagnostiki i razvitiya samoeffektivnosti studencheskoy molodezhi (Psychological practices of diagnostics and development of self-efficacy of student youth). Izdatelskiy Dom TGU, Tomsk (2014)