Classroom observation and post-observation feedback allow for understanding and analyzing teaching practices with the aim of improving teaching efectiveness. Due to systematic classroom observation that requires trained observers, this process is often slow, error-prone, and unscalable. Even its use to provide feedback in a massive, regular, and substantive way is limited. To overcome these dificulties, various researchers have built tools to provide automated feedback to instructors, and with the rise of generative artificial intelligence, they have made progress in providing scalable, regular, and timely feedback. However, the information these tools provide to instructors presents dificulties associated with information overload, limited actionability, lack of intent to use it, and concerns about surveillance and control. To address these problems, it is proposed that the integration of instructors' contexts, needs, and experiences are facilitating conditions that favor the use of automated feedback on teaching practices provided by generative artificial intelligence. A human-centered design approach is proposed to model an automated feedback tool to assess synchronous online classes delivered through video conferences. To conceptualize teaching practices, the International Comparative Analysis of Learning and Teaching framework is considered, which provides a set of dimensions for assessing teaching efectiveness. Finally, factors are evaluated that influence the perceived benefits of the use of automated feedback provided by generative artificial intelligence to support self-reflection in the context of online higher education.
the synchronous modality, mediated by videoconferencing tools, has established itself as a key element [17]. Synchronous classes seek to replicate the in-person classroom experience by allowing real-time interaction between instructors and students [18], while maintaining similarities with traditional face-to-face classes [19].
To conceptualize teaching practices, the International Comparative Analysis of Learning and Teaching (ICALT) framework will be used, which provides a set of dimensions to assess teaching efectiveness. This choice is based on the validity and reliability of this instrument for observing and measuring efective teaching, which has been studied in multiple international educational contexts [20].
Finally, perceived benefits are one of the main reasons people adopt a technology, as they reduce the perceived risk of adopting and using it. In this study, we consider the perceived benefit to refer to the degree to which an instructor perceives that the use of automated feedback generated by generative artificial intelligence will bring significant improvements to their teaching [21].
The large number of features obtained from the video recordings results in an information overload.
When this information is provided to instructors, it is dificult that they focus on the aspects relevant to
the analysis of their teaching practices [
Furthermore, the feedback provided by these tools presents access barriers due to its dificulty to
understand [
Additionally, the feedback provided by these solutions presents challenges regarding instructors’ willingness to use it [22]. Because these tools analyze transcribed lecture text, instructors have concerns about the accuracy of the transcript and how it is analyzed for automated feedback provision. This situation can afect the instructors’ intention to use feedback.
Finally, instructors’ appreciation for automated feedback is limited due to concerns about the use
of data for monitoring and accountability purposes [
The research aims to assess the factors that influence the perceived benefits that the use of automated feedback on teaching practices provided by generative artificial intelligence.
Thus, we aim to answer the following general research question.
What are the perceived benefits for online higher education instructors of receiving feedback provided by a generative model to support reflection on their teaching practices in synchronous video conference classes?
The research question and the general objective define the scope of this research. Therefore, to ensure a systematic approach to the problem statement, specific objectives and research questions have been defined. Each objective and its respective question have been designed to address a dimension of the problem posed, together contributing to closing the identified gaps and fulfilling the overall purpose of the study.
The specific research questions (RQ) that guide the entire project are as follows.
RQ1: How does a human-centered design approach contribute to the modeling of an automated feedback tool for teaching practices?
RQ2: What characteristics of the feedback produced by generative artificial intelligence are perceived as useful by the instructors who receive it?
RQ3: What benefits do online instructors perceive when receiving feedback from a generative model to support self-reflection on their teaching practices?
Finally, this research can be summarized in the following research-specific objectives (SO).
SO1: To model the components of an automated feedback tool based on generative artificial intelligence using a human-centered design approach.
SO2: To analyze the characteristics of automated feedback that facilitate the use of the tool in self-reflection by instructors.
SO3: To evaluate the perceived benefit for online instructors of receiving feedback provided by a generative model to support self-reflection on their teaching practices.
Contribution 1: Human-Centered Automated Feedback
By involving instructors in a design process to model an automated feedback tool, with the goal of reflecting their needs, values, and experiences, relevant feedback information will be identified, ofering actionable insights that guide self-reflection and promote improvement.
Contribution 2: Useful and Practical Automated Feedback
Automated feedback of online classes will be addressed using the ICALT framework, which establishes a set of teaching practices that can be evaluated. By determining the characteristics of automated feedback that online instructors consider useful, guidelines will be proposed for its integration into self-reflection, thus facilitating interpretation and promoting the use of feedback in order to improve teaching quality.
Contribution 3: Automated Feedback for Instructor Self-Reflection
By mitigating instructors’ fears related to surveillance and control, the self-reflection processes will be assist by generating automated feedback that considers data ethics and information privacy instructors’ concerns.
We have completed a Proof of Concept (PoC) to evaluate a video processing pipeline for synchronous video conference lectures, which has been presented at IEEE EDUCON 2025.
Furthermore, we are preparing a systematic review of the literature focused on the use of generative models for class analysis. This review will focus on the models employed, the class characteristics analyzed, and the actionable insights they generated. By integrating the findings of this systematic review, we aim to refine our approach and maintain the novelty of our work, considering the rapid evolution of generative models.
Finally, two complementary studies are being conducted that address ethical and privacy issues in automated feedback solutions. Both studies will provide in-depth information on biases and transparency problems in these kind of sociotechnical systems. The findings will help update the current tool by incorporating a more ethical approach and promoting algorithmic transparency in the solution.
The research methodology for this doctoral thesis is based on a mixed methods approach [23]. The design approach adopted to structure the study is design-based research (DBR) [24].
This design approach was chosen because (i) it fits the research objectives, (ii) it facilitates the connection between educational research and a real-world problem, (iii) it allows for collaboration between researchers and educators in the design process, and (iv) it emphasizes the iterative process over the final evaluation of the technological product designed.
Figure 1 shows a diagram of the DBR cycles with each of their stages.
The research is structured through three DBR cycles. In each cycle, we will select the methods that best fit the context and the actors involved to achieve a more comprehensive interpretation of the results. Furthermore, at the end of each cycle, we will achieve a specific research objective so that by the end of the DBR, we will have achieved all the proposed objectives.
In the first cycle, the focus will be on identifying the feedback needs of online education instructors in multiple contexts.
During the second cycle, the focus will be on identifying the characteristics that make automated feedback useful for online education instructors.
In the final cycle, the focus will be on evaluating the conditions that afect perceived benefit on the use of automated feedback for self-reflection by online higher education instructors.
In this study, the transferability of the results will be prioritized over their generalization. To ensure the quality and validity of the research, an interpretive approach will be adopted.
In addition, a deep understanding of the phenomena and situations studied will be sought, while also establishing an ongoing dialogue with the informants.
Finally, in order to ensure the transferability of the research process, the following strategies will be applied: • Data from diferent participants and from diverse contexts will be used. • Multiple data collection methods will be used to achieve triangulation and complementarity of the findings.
• Detailed descriptions of the study contexts will be provided.
This research will follow the ethical principles of artificial intelligence in education to ensure the protection of participants throughout all phases of the study [16].
Furthermore, the study will be submitted for review by the Ethics Committee of the Universidad de Santiago de Chile (USACH), ensuring compliance with international and local ethical regulations for studies in education and technology.
Before data collection, informed consent will be requested from all participating instructors, explaining study objectives, data collection procedures, and their right to withdraw at any time without consequences.
Interviews and online classes will be recorded with the participants’ prior consent and subsequently transcribed and anonymized to protect their identity. The confidentiality of the information will be guaranteed by assigning codes to responses and securely storing data in protected repositories.
This work is funded by the National Agency for Research and Development of Chile. ANID BECAS/DOCTORADO NACIONAL 21250717.
[9] R. Wang, D. Demszky, Is ChatGPT a good teacher coach? measuring zero-shot performance for scoring and providing actionable insights on classroom instruction, in: Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications, BEA2023, 2023, pp. 626–667. doi:10.18653/v1/2023.bea-1.53. [10] R. Alfredo, P. Mejia-Domenzain, V. Echeverria, D. Rahayu, L. Zhao, H. Alajlan, Z. Swiecki, T. Käser, D. Gašević, R. Martinez-Maldonado, Teamteachingviz: Benefits, challenges, and ethical considerations of using a multimodal analytics dashboard to support team teaching reflection, in: Proceedings of the 15th International Learning Analytics and Knowledge Conference, LAK2025, 2025, pp. 58–69. doi:10.1145/3706468.3706475. [11] Ó. Cánovas Reverte, P. González Férez, F. García Clemente, F. Pardo García, Analyzing wooclap’s competition mode with ai through classroom recordings, IEEE Revista Iberoamericana de Tecnologias del Aprendizaje (2024). doi:10.1109/RITA.2024.3458865. [12] U. Lee, Y. Jeong, J. Koh, G. Byun, Y. Lee, H. Lee, S. Eun, J. Moon, C. Lim, H. Kim, I see you: teacher analytics with gpt-4 vision-powered observational assessment, Smart Learning Environments 11 (2024) 48. doi:10.1186/s40561-024-00335-4. [13] K. Wiley, Y. Dimitriadis, M. Linn, A human-centred learning analytics approach for developing contextually scalable k-12 teacher dashboards, British Journal of Educational Technology 55 (2024) 845–885. doi:10.1016/j.caeai.2024.100325. [14] Y. Dimitriadis, R. Martínez-Maldonado, K. Wiley, Human-centered design principles for actionable learning analytics, Research on E-learning and ICT in education: Technological, pedagogical and instructional perspectives (2021) 277–296. doi:10.1007/978-3-030-64363-8_15. [15] N. Blanchard, M. Brady, A. M. Olney, M. Glaus, X. Sun, M. Nystrand, B. Samei, S. Kelly, S. D’Mello, A study of automatic speech recognition in noisy classroom environments for automated dialog analysis, in: Proceedings of the 17th International Conference in Artificial Intelligence in Education, AIED2015, 2015, pp. 23–33. doi:10.1007/978-3-319-19773-9_3. [16] A. Nguyen, H. N. Ngo, Y. Hong, B. Dang, B.-P. T. Nguyen, Ethical principles for artificial intelligence in education, Education and information technologies 28 (2023) 4221–4241. doi:10.1007/s10639-022-11316-w. [17] C. K. Massner, The use of videoconferencing in higher education, in: J. S. František Pollák, R. Vavrek (Eds.), Communication management, IntechOpen, 2021, pp. 75–94. doi:10.5772/intechopen. 99308. [18] A. Francescucci, M. Foster, The viri classroom: The impact of blended synchronous online courses on student performance, engagement, and satisfaction., Canadian Journal of Higher Education 43 (2013) 78–91. doi:10.47678/cjhe.v43i3.184676. [19] S. Cilligol Karabey, S. Karaman, Identifying pedagogical design and implementation of synchronous virtual classrooms, The International Review of Research in Open and Distributed Learning 25 (2024) 132–154. doi:10.19173/irrodl.v25i2.7584. [20] W. Van de Grift, Quality of teaching in four european countries: A review of the literature and application of an assessment instrument, Educational research 49 (2007) 127–152. doi:10.1080/ 00131880701369651. [21] P. Dubey, K. K. Sahu, Students’ perceived benefits, adoption intention and satisfaction to technology-enhanced learning: examining the relationships, Journal of Research in Innovative Teaching & Learning 14 (2021) 310–328. [22] T. Nazaretsky, J. N. Mikeska, B. Beigman Klebanov, Empowering teacher learning with ai: Automated evaluation of teacher attention to student ideas during argumentation-focused discussion, in: LAK23: 13th International Learning Analytics and Knowledge Conference, 2023, pp. 122–132. doi:10.1145/3576050.3576067. [23] J. W. Creswell, J. D. Creswell, Research design: Qualitative, quantitative, and mixed methods approaches, 5th ed., SAGE Publications, 2017. [24] T. Amiel, T. C. Reeves, Design-based research and educational technology: Rethinking technology and the research agenda, Journal of educational technology & society 11 (2008) 29–40. URL: https://www.learntechlib.org/p/75072/.