Students' daily interaction with the learning management system generates millions of rows of digital trace data daily, and the data can expand our understanding of self-regulation. This study employs the confirmatory composite analysis, a PLS-SEM approach, with 158 participants' data obtained from the learning management system to investigate the relation among theory-based constructs: self-regulation, learning behaviors, and academic performance. By examining the model, the estimated model has a good fitting and moderate explanatory and predictive power. The results indicate that students' data obtained from the learning management system is capable of measuring self-regulation and predicting performance. Unlike some empirical studies, after controlling discussion participation, inclass participation, file access, and video consumption, self-regulation (an executive function) has an insignificant association with academic performance, but self-regulation does moderate the effects of driving learning behaviors.
Self-regulation serves as an important set of processes for students to initiate and manage their
learning in the fast-changing world and technology-advanced environment [
In recent years, there is an increased interest in new assessments and methodologies that allow
researchers to develop critical knowledge about different dimensions of self-regulated processes [
The present study employs a learning analytics approach with the Partial Least Square Structure Equation Model (PLS-SEM) to investigate the relationship between students' digital traces as indicators of self-regulation, learning traces, and academic performance, particularly whether students' digital traces can predict achievement. Compared to the most empirical studies that investigate the relation between students' digital traces and academic performance, the present study provides a feasible approach to connecting theories to practices in learning analytics. More importantly, the present study moves from clicks to constructs in an organized and theory-based approach and operationalizes the forethought, performance, and reflection process in the social cognitive model of self-regulation [10, 14]. The two research questions of this study were: 1. How can we create a theory-based measurement of self-regulation? 2. How is self-regulation connected to students’ learning behaviors and academic performance?
The definition of self-regulation in the present study pertains to the processes that students set goals,
plan to achieve the goals, and continually monitor, react, and reflect on their plan. Self-regulation
leads to better learning, improved capabilities, and effective problem-solving. Students who are able
to regulate their learning enjoy benefits in the learning process and achieve better learning outcomes
in various contexts. For instance, first-year and second-year medical students who advance
selfregulation strategies enjoy higher academic achievement in flipped-classroom environments [
Reading the course syllabus, checking gradebook, and accessing announcements sent by instructors are grounded on the three phases in the cyclical model [10]. Because students who engage in self-regulation direct their actions, it is assumed that students’ learning behaviors are all related to self-regulation, although sometimes are not driven by self-regulation. In this case, students’ learning behaviors involve participation and learning materials access based on the course design (discussed in methodology). Participation consists of discussion and in-class participation, evaluating the quality of participation for required learning activities. Learning materials access describes how students interact with learning content offered by instructors, including file access and video consumption.
The participants for this dissertation study were from a undergraduate course designed and taught at a large Midwestern research university by the same instructor over four semesters from 2019 to 2020. The course was delivered originally in face-to-face delivery format with both formative and summative assessments. Yet because of the pandemic, the Spring 2020 course delivery was changed to the online format after March 2020. There were 158 participants with about 290,000 rows of data over the past four semesters who had already completed the course.
The course delivery approach before and during COVID pandemic remained the same. Before every lecture, students were asked to watch lecture videos, read course materials, and complete prelecture activities. During the lecture, the course instructor addressed important questions and facilitated in-class discussions. Right before the end of the lecture, students completed formative assessment questions with iclickers. Due to COVID pandemic, the course instructor moved the inclassroom discussions to Zoom but followed the same procedure, and students needed to complete the formative assessments as they do before. Before each module ended, students were assigned to participate in the online discussion in the learning management system to demonstrate understanding and applications with guided questions. After each module, students were required to complete the summative assessment, and instructors provided feedback to students.
Two types of data were collected from the learning management system: navigation data and gradebook data. Both navigation and gradebook data were collected through a customized Python program. Each student was assigned a random and unique id for de-identification. Navigation data consists of students’ digital traces during the semester, such as logins, course materials views, discussion posts and replies, and clicker usages. 3.2.
In the present study, learning behaviors are estimated with latent constructs based on students’ digital traces, including two categories: participation and learning materials access (See Table 1). Learning materials access refers to students’ access to learning materials posted by course instructors, such as articles, documents, lecture recordings, and videos. Accessing such learning materials is essential and the first step to ensuring learning occurs, and self-regulated learners are supposed to be able to control practices in learning to benefit from the learning materials provided for instructional purposes, especially in the environment where technology is used [30]. In synchronous or asynchronous learning, although students are free to choose when and how to access learning materials, they must log into the course site to access the learning materials. Evidence shows that the count of content access is a significant predictor of academic performance and student engagement. For example, in one paper, students who accessed learning materials more frequently were categorized into selective and efficient learner groups that pursued performance goals and regulated their learning. Completing the reading and media consumption behaviors also indicate the level of engagement [31]. In this case, the use of learning materials access to evaluate whether students interact with the course materials as expected in the course syllabus consists of two constructs: file access and video consumption.
File access determines whether students access files provided and required by instructors, such as articles, examples, and lecture notes. There are two indicators for the measurement of file access behaviors. First, the total number of accesses to files is used to measure the aggregated number that students access the files. Second, the percentage of files accessed measures the coverage of the files accessed, determining the percentage of assigned files students have accessed. Students are supposed to read the assigned documents for the course. According to the literature, college students read less [32, 33, 34]. The aggregation of the number of learning materials accessed can be through attention to students who regulate their learning and those who pursue performance goals. For example, students may frequently access some materials for exams, but others may regulate their learning by accessing all required learning materials step by step. While the frequency of access may be similar, the files accessed percentage differentiates learning strategies and behaviors, thus evaluating their engagement.
In addition to file access, video consumption is used as another construct to evaluate students’ behaviors guided by self-regulation. Video-based learning provides an engaged and interactive learning environment and experience for learners rather than linear broadcasting [35], allowing students to pause, forward, or rewind videos. More regulated learners tend to have a longer duration and frequency in which they engage in watching videos [36]. Therefore, two indicators are used to estimate video consumption: total minutes watched and the total number of videos watched. Total minutes watched assesses the total number of minutes students consume instructional videos, and the total number of videos watched measures the aggregated number of individual videos watched by students.
Similar to learning materials access, participation represents the actions students perform in required course activities. In this context, the course instructor designs two required learning activities: discussion and clicker questions. Students are required to participate in online discussions after each module and to answer in-class clicker questions in every lecture. Therefore, discussion participation and in-class participation are the two constructs for participation measurement.
Discussion participation describes how students participate in online discussion forums and interact with others. There are three indicators used to measure discussion participation. Collaborative work is a critical part of learning, and online discussion is one of the most effective approaches to promoting collaborative work [37]. Research has shown that online discussion participation facilitated knowledge acquisition and sustained positive effects on academic performance and achievement [38, 39]. Students participated and engaged in the process of collaborative work, such as online discussion, by reading, reflecting, and posting messages on the discussion board, suggesting that the number of times students access the discussion forum is fundamental for measuring collaborative work [40]. To accurately measure how many times students access the discussion forum, the number of unique days students access the discussion forum is used to prevent overcounting. For example, if a student accesses the discussion forum multiple times in one day, only one access will be counted. Moreover, the number of messages and the length of the message is also important to discussion participation. Students who contribute a relatively large number of messages are more active learners than those who post too fewer messages [40]. The length of the post from the beginning of the root thread post and the length of the post have been used as the metrics to measure the quality of students' collaborative work [41]. The length of a single post is often used as a proxy for the quality of the discussion, especially after students read and reflect on the messages posted by peers. Students participating in collaborative work are more likely to access the discussion frequently, read more messages, reply to other students more frequently, and write more in each post.
Beyond discussions in the asynchronous setting, in-class participation also plays a crucial role as the instructional strategy to engage students in the synchronous setting. Meanwhile, students consider class participation a crucial learning strategy [42]. Class participation has a positive relation to academic performance in higher education because students have the opportunity to interact to learning materials and time for skills practice and content assimilation [43]. In-class participation is measured by the clicker question participation. Click questions usually serve as the instructional strategy that focuses on enhancing students' participation, attendance, and attention [44]. Participation describes whether students answer clicker questions offered only in the classroom or online synchronous meeting room. If students do not show up in class or suddenly shift attention away from the lecture or activities, they are not able to complete the clicker questions. 3.3.
Students' academic performance for the course is usually determined by course grade, generated based on the weighted or unweighted course assessments [45]. While course grade is an objective measure to evaluate students' effort to advance the course content, it suffers limitations. Grades in single courses are often not normally distributed and often suffer from ceiling effects that restrict its effective range. This is partially caused by grade inflation compromising all students' grades, leading to a lack of differentiation based on a single course final grade [46]. Practically, it is challenging to distinguish students' behaviors solely based on their course grades. To address the issue, the present study evaluates students' academic performance as a latent variable constructed from a series of summative assessments. 3.4.
A confirmatory composite model is estimated to uncover the relation between self-regulation, learning behaviors, and performance using SmartPLS. SmartPLS is a popular graphical interface analytical software designed specifically for variance-based structural equation models with a partial least square approach. The proposed model includes a total of six constructs: academic performance, self-regulation, discussion participation, in-class participation, file access, and video consumption. Discussion participation, in-class participation, file access, and video consumption are categorized into learning behaviors, and these latent constructs are all directed to academic performance. Another latent construct, self-regulation, is linked to all four learning behavior constructs as well as academic performance.
The measurement and structure component of the model was evaluated separately following a two-stage evaluation recommendation [47]. In the first stage, each latent construct in the measurement component was evaluated for indicator reliability, internal reliability, convergent validity, and discriminate validity. For reliability, indicators should have indicator reliability higher than 0.7 except those indicators are retained for content validity. The composite reliability should be between 0.7 and 0.9. Any construct with a value higher than 0.9 is not desirable for indicator redundancy avoidance, which probably compromises content validity [47, 48, 49]. For validity, the average variance extracted (AVE) from the constructs was calculated by obtaining the grand mean of the squared loadings of the indicators for convergent validity. An AVE value of 0.5 is desirable because less than half of the variance remains in the measurement error than extracting from the constructs. The discriminant validity was assessed with heterotrait-monotrait ratio (HTMT) to ensure each construct was distinguished from other constructs. HTMT calculates the ratio between betweentrait and within-trait correlations, obtaining the mean of correlations of indicators across all constructs over the mean of the average correlations of the indicators in the same construct [49]. If the HTMT value is significantly smaller than 1, the two constructs are clearly discriminated [47, 50]. The HTMT was computed with bootstrapping procedure with 10,000 subsamples to obtain a 95% confidence interval for hypothesis testing.
In the second stage, path coefficient, collinearity, and explanatory and predictive power were examined to evaluate the structural component of the model. Variance inflation factor (VIF) was utilized to measure collinearity. Any VIF below five indicates no substantial collinearity effect on the structural component [47, 51]. Then a 10,000-subsample bootstrapping procedure with a 95% confidence interval was performed to assess the relevance and the significance of path coefficients between the two constructs. The was used to evaluate the explanatory power. Although PLS-SEM aims at maximizing the variance explained, the model may overfit the data with an excessive of 0.9 or higher [47, 50]. The procedure uses the model estimates generated from the training set to predict the values for the indicators of the dependent constructs from the holdout sample. The divergence between the actual and predicted values indicates the predictive power: the lower the divergence, the higher the predictive power. The mean absolute error (MAE) was used to compute the divergence for predictive power evaluation. Comparable to the root mean square error (RMSE), MAE assumes the equal weight of all errors, which is less sensitive to extreme values. Since the prediction error distribution might be non-symmetric, MAE was preferable to RMSE. The divergence of MAE was calculated for both PLS and LM in the SmartPLS software. If all MAE values obtained from the linear regression model benchmark (LM) are greater than the values obtained from PLS, the model has high predictive power [47]. If the values obtained from PLS are greater than the values obtained from LM, the model lacks predictive power. If some values from PLS are greater than the values from LM, the model has medium predictive power.
A total of 290,004 log records obtained from the learning management system for 158 students across four semesters were used to explore the association between proposed latent constructs and academic performance. Confirmatory composite analysis was used to estimate the path coefficients among all proposed latent constructs and academic performance, the explanatory and predictive power of the model, as well as measurement reliability and validity. Because the confirmatory composite analysis used in the study was a non-parametric method, a bootstrapping procedure was employed to obtain the standard errors of the estimated coefficients to determine t values and corresponding p values and the confidence interval for the stability of the estimates. Based on the bootstrapping procedure, loadings and path coefficients were tested for significance to determine whether there were non-zero effects.
The model estimation successfully converges in eight iterations, indicating that there is no
problem with data [47, 50, 52, 53]. The results show that the data fit the model well. Discussion
participation, in-class participation, and video consumption are the three constructs that significantly
predict academic performance.
Path
Self-Regulation → Discussion Participation → Academic Performance
Self-Regulation → File Access → Academic Performance
Self-Regulation → In-Class Participation → Academic Performance
Self-Regulation → Video Consumption → Academic Performance
Self-Regulation → Academic Performance
Note. * p <.05. ** p <.01. *** p <.001.
As shown in Table 2, all indicators have indicator reliability higher than 0.7 except announcement
views (ANN), which is close to 0.7. This indicator is retained in the model because it improves the
content validity. ANN represents the self-reaction behaviors students perform, an essential phase for
self-regulation [
AVE is used to establish the convergent validity of the latent constructs. All AVE values are above the threshold of 0.5, suggesting that the construct explains more than 50% of the variance for its indicators. For the single-item construct IN, AVE is not the appropriate method for the convergent validity because the loading for the indication is fixed at 1 (see Table 2).
HTMT values of the 95% upper bound is obtained to measure discriminant validity from a bootstrapping procedure with 10,000 subsamples since PLS-SEM is a non-parametric method (see Table 3). Because all HTMT values of the 95% upper bound are below 1, meaning that all two latent constructs are empirically distinct. 4.2.
Variance inflation factor (VIF) is applied to evaluate the collinearity among all latent constructs. The ideal VIF value is close to or below 3. In the present study, all the VIF values in the proposed model are below 3, suggesting collinearity issue is not found among all proposed constructs (see Table 2).
Path coefficients assess the hypothesized relations among the constructs. The path coefficients were standardized values obtained from a bootstrapping procedure with 10,000 subsamples. Figure 2 shows the path coefficients between academic performance and all other constructs. Discussion and in-class participation have a significant coefficient of 0.37 and 0.29 on academic performance. Video consumption has a significant coefficient of 0.24 on academic performance, but the other learning material access construct, file access, has a non-significant coefficient of 0.07 on academic performance. Self-regulation also has a non-significant coefficient of 0.02 on performance.
Because the relation between self-regulation and academic performance is not significant, the indirect effects are then analyzed (see Table 4). The indirect effects between self-regulation and academic performance via discussion participation, in-class participation, and video consumption are all significant and at 0.23, 0.1, and 0.07. The indirect effect between self-regulation and academic performance via file access is 0.03 and insignificant. The sum of indirect effects between selfregulation and academic performance was 0.432 and significant.
The coefficient of determination is used to assess the model's explanatory power. Table 2 shows that 64% of the variance in academic performance is explained by the combination of learning behaviors and self-regulation. Beyond the performance, 38% of the variance in discussion participation and 14% of the variance in in-class participation are explained by self-regulation. Twenty percent of the variance in file access and 8% of the variance in video consumption are explained by self-regulation.
Predictive power is utilized to evaluate whether the model could produce generalizable findings. Table 2 shows the mean absolute error (MAE) divergence between PLS and linear regression model benchmark (LM) for four assessments are 0, -0.02, -0.04, and -0.02, meaning that three of four MAEs from the PLS model are smaller than the predicted LM model. Since the majority of MAE differences between PLS and LM are fewer than 0, suggesting that the model has a moderate predictive power [47].
This study moves from correlating clicks to creating and validating the possibility of measuring theoretical learning constructs from digital traces. Self-regulation is not a significant predictor of academic performance after controlling learning behaviors, but self-regulation is significantly associated with all learning behaviors. The present study measures self-regulation based on students' digital traces with various data sources to capture students' learning behaviors impacted by selfregulation and on academic performance. It is a practical and meaningful approach that we hope will be increasingly adopted by the learning analytics research community and self-regulation researchers. Because not all the variables are directly observable, learning analytics researchers could extract more information from students' digital traces to assess learning rather than assuming every component of learning is observable by implementing educational measurement concepts. The exploratory and predictive power estimated suggest that the current method increases the accuracy and variance explained by indicators or constructs for the outcome variable.
Further, the present study provides a meaningful and valid measurement and model for selfregulation, advancing the operational theories. The measurement of self-regulation should not only be captured by think-aloud protocols, but it should also be students' behaviors or actions recorded in the learning management system. The current measurement overcomes the large expense happened to think-aloud protocols and captures self-regulation without disrupting some of the key processes. The usage by the learner is intended to promote instant feedback for self-regulated learning. Students could receive both behavior-based and performance-based feedback to sharpen self-regulation and improve performance.
Additionally, the study enables professionals to conveniently model self-regulation and learning behaviors at the course level with a relatively small sample. It is particularly beneficial for instructors and professionals eager to monitor students' learning and improve self-regulation. First, as the learning management system allows professionals to retrieve students' digital traces in nearly realtime, instructors and learning experience designers could evaluate teaching and learning with an evidence-based approach rather than expensive and disrupting methods. Second, learning experience designers usually revise and adjust their design to improve self-regulated learning by communicating with instructors and students, lacking action-based information. This model allows instructors and designers to overview how students monitor their learning and make decisions. Both designers and instructors could use this model to improve the design and motivate students. For example, instructors can encourage students to access all required documents before preceding forward by restricting the access until items in previous modules are completed before specific deadlines. Third, the current model echoes the significant role of instructors in how students regulate their learning, especially in a dynamic learning environment. Without prompt feedback to students (announcement and grade) and an expectation-specific syllabus, students would have fewer chances to engage in selfregulation and thus suffer from the course. Last, the present study brings opportunities for learning analytics and self-regulation researchers to reconsider how to use student data.
There are two limitations to the present study. First, data used to model self-regulation and learning behaviors represent how students engage in the learning management system. Since learning occurs in a lifelong and diverse ecosystem, the assessment of learning does not lie on single elements built with an exclusive data view. Data not included in the respective modeling is equally important, if not more important, to the dataset used to model student learning [54]. While data utilized in this study consists of all students' digital traces, it neither represents all students' learning activities nor any offline tasks. This limitation is also a common limitation in all learning analytics research. Second, more evaluation should be done with various course designs. Factors in course design and construction, such as the content in announcements, learning objectives, type of assessments, grading schema, or syllabus written, vary course by course, and a slight change in any of the factors may influence students' expectations and actions, affecting how students regulate their learning.
Future studies can focus on three aspects of self-regulation with students' digital traces. First, the current results show that self-regulation is a significant predictor of learning behaviors, but the role of self-regulation is unclear. Understanding the relation between self-regulation and other learning behaviors could improve the current model. It is important to focus on what role self-regulation plays (mediator or moderator) and how self-regulation affects learning behaviors in the learning management system. Second, additional in-depth qualitative data could also be used to approach selfregulation with self-reported attitudes, perceptions, and strategies. Comparing students' perceptions, attitudes, and behaviors in the learning management system could reveal more about how students regulate their learning. Third, the goal of the present study is to evaluate and measure self-regulation and its role in the learning process. Although the model’s moderate predictability allows instructors to make data-informed decisions to assist those who suffer in class, the model itself does not predict self-regulated learning strategies used in empirical educational data mining studies. Further studies may focus on the construct-based approach of prediction for self-regulation strategies. Finally, the present study applies the reflective measurement that changes in a specific type of construct would result in changes in all indicators, but changes in one of the indicators would not result in a change in the construct. Therefore, it is necessary to explore how self-regulation relates to academic performance with formative measurement. [10] B. J. Zimmerman, From cognitive modeling to self-regulation: A social cognitive career path,
Educational Psychologist 48 (2013) 135-147. [11] J. R. Segedy, J. S. Kinnebrew, G. Biswas, Using coherence analysis to characterize self-regulated learning behaviors in open-ended learning environments, Journal of Learning Analytics 2.1 (2015) 13-48. [12] G. Biswas, J. S. Kinnebrew, J. R. Segedy, Using a cognitive/metacognitive task model to analyze students learning behaviors, in: International Conference on Augmented Cognition, Springer, Limenas Chersonisou, Greece, 2014, pp. 190-201. [13] G. Biswas, R. S. Baker, L. Paquette, Data mining methods for assessing self-regulated learning, in: D. H. Schunk, J. A. Greene (Eds.), Handbook of self-regulation of learning and performance, Routledge, New York NY, 2017, pp. 388-403. [14] B. J. Zimmerman, Attaining self-regulation: A social cognitive perspective, in M. Boekaerts, P. R.
Pintrich, M. Zeidner (Eds.), Handbook of Self-Regulation, Academic Press, San Francisco, 2000, pp. 13–39. [15] P. H. Winne, Modeling self-regulated learning as learners doing learning science: How trace data and learning analytics help develop skills for self-regulated learning, Metacognition and Learning (2022) 1-19. [16] M. M. Chang, Enhancing web‐based language learning through self‐monitoring, Journal of
Computer Assisted Learning 23.3 (2007) 187-196. [17] J. C. Sun, Y. T. Wu, W. I. Lee, The effect of the flipped classroom approach to OpenCourseware instruction on students’ self‐regulation, British Journal of Educational Technology 48.3 (2017) 713729. [18] Y. Li, X. Li, Y. Su, Y. Peng, H. Hu, Exploring the role of EFL learners’ online self-regulation profiles in their social regulation of learning in wiki-supported collaborative reading activities, Journal of Computers in Education 7.4 (2020) 575-595. [19] J. W. Lin, C. W. Tsai, The impact of an online project-based learning environment with group awareness support on students with different self-regulation levels: An extended-period experiment, Computers & Education 99 (2016) 28-38. [20] I. Boor, S. Cornelisse, How to encourage online self-regulation of students, Communications of the Association for Information Systems 48.1 (2021) 211-217. [21] S. Korkmaz, I. H. Mirici, Converting a conventional flipped class into a synchronous online flipped class during COVID-19: University students’ self-regulation skills and anxiety, Interactive Learning Environments (2021) 1-13. doi: 10.1080/10494820.2021.2018615 [22] S. Zhou, Y. Zhou, H. Zhu, Predicting Chinese university students’ e-learning acceptance and selfregulation in online English courses: Evidence from emergency remote teaching (ERT) during COVID-19, Sage Open 11.4 (2021) 1-15. [23] F. A. Rowe, J. A. Rafferty, Instructional design interventions for supporting self-regulated learning: Enhancing academic outcomes in postsecondary e-learning environments, Journal of Online Learning and Teaching 9.4 (2013) 590-601. [24] M. Barak, Motivating self-regulated learning in technology education, International Journal of
Technology and Design Education 20.4 (2010) 381-401. [25] C. Harrington and M. Thomas, Designing a Motivational Syllabus: Creating a Learning Path for
Student Engagement, Stylus Publishing, 2018. [26] R. Lam, Formative use of summative tests: Using test preparation to promote performance and self-regulation, The Asia-Pacific Education Researcher 22.1 (2013) 69-78. [27] K. M. Cauley, J. H. McMillan, Formative assessment techniques to support student motivation and achievement, Journal of Educational Strategies 83.1 (2010) 1-6. [28] N. R. Aljohani, A. Daud, R. A. Abbasi, J. S. Alowibdi, M. Basheri, and M. A. Aslam, An integrated framework for course adapted student learning analytics dashboard, Computers in Human Behavior 92 (2019) 679-690. [29] F. Wu, L. Huang, R. Zou, The design of intervention model and strategy based on the behavior data of learners: A learning analytics perspective, in: International Conference on Hybrid Learning and Continuing Education, Springer, New York, 2015, pp: 294-301 [30] E. Delen, J. Liew, The use of interactive environments to promote self-regulation in online learning: A literature review, European Journal of Contemporary Education, 15.1 (2016) 24-33. [31] J. Jovanović, D. Gašević, S. Dawson, A. Pardo, N. Mirriahi, Learning analytics to unveil learning strategies in a flipped classroom, The Internet and Higher Education 33.4 (2017) 74-85. [32] K. Baier, C. Hendricks, K. W. Gorden, J. E. Hendricks, L. Cochran, College students' textbook reading, or not, in: American Reading Forum Annual Yearbook, volume 28, 2008, pp: 1-8. [33] M. A. Clump, J. Doll, Do levels of reading course material continue? An examination in a forensic psychology graduate program, Journal of Instructional Psychology 34.4 (2007) 242-246. [34] B. J. Phillips, F. Phillips. Sink or skim: Textbook reading behaviors of introductory accounting students, Issues in Accounting Education 22.1 (2007) 21-44. [35] K. Shephard, Questioning, promoting and evaluating the use of streaming video to support student learning, British Journal of Educational Technology 34.3 (2003) 295-308. doi: 10.1111/14678535.00328 [36] P. G. de Barba, D. Malekian, E. A. Oliveira, J. Bailey, T. Ryan, G. Kennedy, The importance and meaning of session behavior in a MOOC, Computers & Education 146 (2020) 103772. [37] D. R. Garrison, M. Cleveland-Innes, Facilitating cognitive presence in online learning: Interaction is not enough, The American Journal Of Distance Education 19.3 (2005) 133-148. [38] S. Goggins, W. Xing, Building models explaining student participation behavior in asynchronous online discussion, Computers & Education, 94.3 (2016): 241–251. [39] L. Williams, M. Lahman, Online discussion, student engagement, and critical thinking, Journal of Political Science Education 7.2 (2011) 143-162. [40] V. P. Dennen, Pedagogical lurking: Student engagement in non-posting discussion behavior,
Computers in Human Behavior 24.4 (2008) 1624–1633. [41] C. Kent, E. Laslo, S. Rafaeli, Interactivity in online discussions and learning outcomes, Computers & Education 97.3 (2016) 116–128. [42] R. E. Landrum, Teacher-ready research review: Clickers, Scholarship of Teaching and Learning in Psychology 1 (2015) 250 –254. [43] D. García-Pérez, J. Fraile, E. Panadero, Learning strategies and self-regulation in context: How higher education students approach different courses, assessments, and challenges. European Journal of Psychology of Education 36.2 (2021) 533-550. [44] M. Credé, S. G. Roch, U. M. Kieszczynka, Class attendance in College. Review of Educational
Research 80.2 (2010) 272–295. [45] M. Richardson, C. Abraham, R. Bond, Psychological correlates of university students' academic performance: A systematic review and meta-analysis, Psychological Bulletin 138.2 (2012) 353-387. [46] V. E. Johnson, Grade inflation: A Crisis in College Education. Springer Science & Business Media,
New York, 2006. [47] J. F. Hair, G. T. M. Hult, C. M. Ringle, M. Sarstedt, A Primer on Partial Least Squares Structural
Equation Modeling (PLS-SEM), Sage Publications, Los Angeles, 2021. [48] J. F. Hair, C. M. Ringle, M. Sarstedt, PLS-SEM: Indeed a silver bullet, Journal of Marketing theory and Practice 19.2 (2011) 139-152. [49] J. Henseler, C. M. Ringle, M. Sarstedt, A new criterion for assessing discriminant validity in variance-based structural equation modeling, Journal of the Academy of Marketing Science 43.1 (2015) 115-135. [50] J. Henseler, G. Hubona, P. A. Ray, Using PLS path modeling in new technology research: Updated guidelines, Industrial Management & Data Systems 116.1 (2016) 2-20. [51] G. Cho, H. Hwang, M. Sarstedt, C. M. Ringle, Cutoff criteria for overall model fit indexes in generalized structured component analysis, Journal of Marketing Analytics 8.4 (2020) 189-202. [52] J. F. Hair, J. J. Risher, M. Sarstedt, C. M. Ringle, When to use and how to report the results of PLS
SEM, European Business Review 31.1 (2019) 2-24. [53] W. W. Chin, J. H. Cheah, Y. Liu, H. Ting, X. J. Lim, T. H. Cham, Demystifying the role of causalpredictive modeling using partial least squares structural equation modeling in information systems research, Industrial Management & Data Systems 120.12 (2020) 2161–2209. [54] W. Greller, H. Drachsler, Translating learning into numbers: A generic framework for learning analytics, Journal of Educational Technology & Society 15.3 (2012) 42-57.