The emergence of Generative Artificial Intelligence (GenAI) offers transformative opportunities to enhance Personalized Learning (PL) in higher education. This research-in-progress proposes leveraging GenAI to advance PL by using Conversational Agents (CAs) designed through prompt engineering. A first proof-of-concept is demonstrated in a 3D model of the Scrovegni Chapel (Capella Arena) as a learning environment tailored for a Renaissance and Baroque art course at the Open University of Israel. By crafting prompts for synthetic students, we show how we can train the required CAs before real students are presented with self-paced 3D virtual reality (VR) learning environments. Synthetic learning data is presented as a new resource for Learning Analytics (LA) that supports pedagogical design.
The rise of Generative Artificial Intelligence (GenAI) offers higher education opportunities to
enhance content creation, student engagement, and personalized learning (PL). However, their
integration requires educators and learners to develop competencies for understanding how to
effectively use these technologies while taking the appropriate measures to minimize their known
limitations. Clear strategies emphasizing critical thinking, fact-checking, and ethical use are
essential, alongside addressing challenges like bias, misuse, and the need for human oversight [
The early forms of a CA emerged with the rise of Apple’s Siri, Microsoft’s Cortana, Amazon’s
Alexa, and Google Assistant [
We present an initial proof-of-concept (POC) of the use of GenAI-supported CAs in a 3D model of The Scrovegni Chapel (Capella Arena) as a VR learning environment tailored for a Renaissance and Baroque art course at the Open University of Israel (OUI). Our POC demonstrates that synthetic learning data can simulate diverse student interactions with CAs, offering a new type of resource for Learning Analytics (LA) to augment current pedagogical design practices.
Immersive VR is intended to provide learners with a personalized learning experience that captures
their attention, engages them, and reduces diversions. In this context, CAs can provide guidance
and help learners navigate VR environments [
Students may be engaged in immersive VR environments. However, to ensure their understanding, they must be able to identify specific elements, actions, contextual significance, and meaning. CAs can be instrumental in providing real-time feedback and personalized assessment of students’ open-ended answers (either written or spoken). Savetti et al. [11] demonstrated the ability of CAs to provide this state-of-the-art feedback within the context of medical training simulations.
VR learning environments may be particularly effective in enhancing learning of complex, unstructured content and skills that lack definitive answers and often require students to adapt to specific circumstances, make assumptions, or devise innovative solutions. CAs may provide flexible feedback, ask for clarifications, and direct students to address issues that may affect their solutions. Nursing education exemplifies these advantages by gradually augmenting traditional simulations with GenAI learning capabilities in their complex learning settings. A recent scoping review on the use and effectiveness of AI-powered chatbots as teaching tools in nursing education indicates that although these technologies are still evolving, chatbots, which simulate patient interactions, create a controlled practicing environment where students can refine their critical thinking and decisionmaking skills, by providing continuous, personalized feedback and adapting their responses in realtime to the changing learning needs. Some of these chatbots are already integrated into simulationbased learning involving VR [12]. Hence, CAs, with their enhanced capabilities, have the potential to facilitate better learning of complex knowledge and skills.
CAs can emulate collaborative learning by conducting conversations with the learner. Zielke et al. [13] examine social learning in a collaborative augmented reality environment with pedagogical CAs as learning companions. They analyze a use case of medical school students who learn how to interview a patient with stroke symptoms. Their results show that the use of CAs increased selfefficacy as well as conceptual and procedural learning. However, Zielke et al. [13] also indicate current challenges related to the rapidly evolving natural language processing (NLP) technology.
In this research-in-progress, we adopt the Design Science Research (DSR) paradigm, guided by the three-cycle process proposed by Hevner [14]. This POC was conducted according to Hevner's relevance, design, and rigor cycles to achieve practical applicability and theoretical advancement. The relevance cycle ensures the research addresses a clearly defined, real-world problem. Meanwhile, the design cycle emphasizes the iterative creation and evaluation of a designed artifact to resolve this problem. Finally, the rigor cycle situates the study within the existing body of knowledge, ensuring methodological validity and contributing to ongoing scholarly discourse. These interconnected cycles provide a comprehensive framework for examining the POC.
The relevance cycle for this research-in-progress focuses on the problem of evaluating personalized VR learning experiences for students using GenAI voice-activated CA. A secondary goal is to allow human oversight to ensure a safe, reliable, and responsible learning environment that can address the control and ethical challenges of using GenAI, drawing on Shneiderman’s Human-Centered Artificial Intelligence (HCAI) framework [15].
The design cycle included several stages towards the final research artifact. First, we created a robust knowledge base in the organizational Retrieval-Augmented Generation (RAG). We then populated it with the relevant course materials (e.g., a textbook chapter, presentation slides, video subtitles, Zoom transcripts, and open-access YouTube captions). Second, we created a dual-role prompting strategy that simulates a dialogue between a synthetic student and the CA. The synthetic student was designed to explore the 3D model of Scrovegni Chapel [16], navigate to points of interest, and ask open-ended questions based on the topic's learning objectives. The CA was designed to provide on-demand guidance, contextual information about the items within the 3D model, and feedback to the synthetic student according to the instructor’s evaluation rubrics. An orchestration prompt was designed to facilitate the dialogue between the two entities. Third, the course instructor supervised the synthetic dialogue between the CA and the synthetic student to improve both prompts. Fourth, the CA prompt was integrated into a voice-activated GenAI tool alongside the knowledge base. The course instructor evaluated the tool to properly handle irregular questions and unlikable scenarios that might harm real students or provide incorrect responses. The CA prompt was again refined according to the errors it made. Future steps will include introducing the CA to real students with the supervision of their course instructor, and finally allowing them to use the voice-activated GenAI tool asynchronously.
The rigor cycle is grounded in Shneiderman's HCAI model [15]; this approach integrates AI’s strengths with human judgment to develop adaptable and trustworthy systems. Creating personalized learning experiences for pedagogical conversational agents requires meticulous preparation and deliberate human oversight to ensure safety, reliability, and ethical responsibility. Our future research will document the improvement process of the CA first by the synthetic student and then by the instructor to mitigate the abovementioned risks of implementing GenAI technologies. This approach contributes to Berg's [17] emerging research on synthetic data within the field of Learning Analytics (LA) [18]. Synthetic data is suggested by Berg [18] as an intermediary solution before actual data is available and bureaucratic processes are approved or support scaling, methodological development, and evaluation. Using synthetic students facilitates iterative testing and improvements to the original CA pedagogical prompt design, ensuring the conversational agent can effectively navigate a wide range of pedagogical learning scenarios. Within the context of LA, the evaluation is firstly directed at the synthetic students according to the specific rubrics outlined by the pedagogical design as a “form of documentation of pedagogical intent that can provide the context for making sense of diverse sets of analytic data" [19]. The pedagogical prompt design of the CA conveys the instructor's intentions and aligns them with the course’s pedagogical goals and evaluation rubrics. 4. The case of conversational agents in the 3D model of Scrovegni
The Scrovegni Chapel, commonly referred to as the Arena Chapel or Capella Arena is a historic church in Padua, Italy, celebrated for its remarkable frescoes painted by Giotto di Bondone. Commissioned by Enrico Scrovegni in the early 14th century, the chapel was built near a Roman arena, which gave rise to its alternate name. Giotto's fresco cycle, completed around 1305, narrates the Virgin Mary's and Jesus Christ's lives with groundbreaking use of perspective and emotional expression, marking a significant evolution in Western Art. Recognized as part of the 14th-century fresco cycles in Padua, the chapel was inscribed as a UNESCO World Heritage Site in 2021, highlighting its enduring cultural and artistic importance [16].
In the art course Renaissance and Baroque, a 3D model of the Scrovegni Chapel was created to offer an immersive exploration of its artistic brilliance. This virtual reconstruction features an invisible central staircase connecting three distinct chapel levels. Students can navigate these levels by walking around and viewing Giotto's magnificent frescoes from the eye level. This design allows for an intimate appreciation of the chapel's intricate details, bringing the grandeur of Renaissance art into a dynamic, interactive experience. By virtually "walking" through the chapel, students gain a spatial understanding of the frescoes' placement and how their narratives unfold within the architectural context. Unlike static 2D images or textual descriptions, the model allows students to experience the chapel as a three-dimensional space designed for religious ceremonies, where its theatrical and performative elements come to life. The model enables students to grasp the interconnected narratives within the frescoes, emphasizing their spatial relationships, emotional depth, and moral lessons. Additionally, the immersive experience helps students connect with the chapel's historical and cultural context, understand its patrons' motivations, and reflect on the societal and theological statements conveyed through its art. Through this multisensory and embodied learning, students develop a nuanced appreciation of Giotto's genius and the broader significance of religious art as an active, lived experience rather than a static visual artifact. A 3D learning environment is a natural test case for evaluating the potential uses of personalized learning, allowing students to openly explore the course’s content in an immersive and experiencedriven setting. Moreover, this 3D environment is pedagogically suitable for audible CAs since adding a visual representation of an avatar might impede the effectiveness of the pedagogical focus on the immersive experience.
To augment the 3D model Scrovegni Chapel, we collected the relevant learning materials from the course's virtual learning environment (VLE): A chapter from the OUI course textbook; a transcript of a video lecture, and the relevant content pages from the VLE. This content was imported to the organizational RAG and served as context for the Large Language Model (LLM) [20]. We then crafted three prompts: The CA which was trained on the learning materials; a synthetic student who is curious but has little knowledge in baroque arts; and an orchestration CA trained to proactively suggest a dialogue between the pedagogical CA and the synthetic student (The prompt for the orchestration CA is available here: https://qr.openu.ac.il/01367). We used the ChatGPT 4o LLM to support the crafting of the prompts and for running the synthetic dialogue. By analyzing several synthetic dialogues, we were able to gradually improve both the prompt of the synthetic student and the prompt of the CA. The final synthetic dialogue gave us the green light to move forward (A sample discussion of a synthetic student with the CA is available here: https://qr.openu.ac.il/01368). Finally, the orchestration CA was connected to Play.ai, a voice AI platform specializing in conversational voice agents, offering real-time text-to-speech application programming interfaces (APIs). The voice-based CA created a natural experience of a personal expert orienting the way inside the 3D model of Scrovegni Chapel while being able to answer any question on the different art pieces. Subsequently, the course instructor evaluated the voice version of the CA prompt ensuring its reliability, safety, and trustworthiness before presenting it to real students (A sample evaluation dialogue of the course instructor with the audio-based CA is available here: https://qr.openu.ac.il/01359).
This research-in-progress presents a POC for crafting personalized learning experiences in 3D environments with a GenAI-based CA and using synthetic learning data as a new source for LA. The CA successfully passed the course instructor’s critical evaluation. The voice conversations of the instructor with the CA demonstrated its ability to provide students with an engaging learning environment that gives them real-time feedback and enables better learning of complex knowledge and skills, in a social collaborative atmosphere.
After completing the entire cycle from conceptual design to POC, in the coming semester, we plan to let real students interact with the CA during a live course session with an instructor present, who may tend to any unforeseen issues, observe the students’ personalized learning experiences while interacting with the CA, and gain more insights on the learning process. Thereafter, we intend to present students with an asynchronous version of Scrovegni Chapel 3D model enhanced with a voice-based CA.
We would like to thank Beni Zaks, Head of Learning Experience at the Open University of Israel, who, as a gifted VR and 3D artist, developed all the 3D objects within the 3D model of Scrovegni Chapel and documented the evaluation dialogue of the CA with the course's instructor.
During the preparation of this work, the authors used Grammarly in order to: Grammar and spelling check, paraphrase and reword. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. [11] F. Salvetti, B. Bertagni, I. Contardo, R. Gardner, J. Rudolph, R. Minehart, Feedback reimagined: generative AI and conversational avatars as your new training partners, in: Lecture notes in networks and systems, Springer Nature Switzerland, Cham, 2024, pp. 133–144. doi:10.1007/978-3-031-73427-4_13. [12] L. J. Labrague, S. A. Sabei, Integration of ai-powered chatbots in nursing education: A scoping review of their utilization, outcomes, and challenges, Teaching and Learning in Nursing (2024). doi:10.1016/j.teln.2024.11.010. [13] M. A. Zielke, D. Zakhidov, T. Lo, S. D. Craig, R. Rege, H. Pyle, N. V. Meer, N. Kuo, Exploring social learning in collaborative augmented reality with pedagogical agents as learning companions, International Journal of Human–Computer Interaction (2024) 1–26. doi:10.1080/10447318.2024.2323280. [14] Hevner, A. R. (2007). A three cycle view of design science research. Scandinavian Journal of
Information Systems, 19(2), 87-92. https://aisel.aisnet.org/sjis/vol19/iss2/4 [15] B. Shneiderman, Human-Centered artificial intelligence: reliable, safe & trustworthy, International Journal of Human–Computer Interaction 36.6 (2020) 495–504. https://doi.org/10.1080/10447318.2020.1741118. [16] G. Pisani, Scrovegni chapel: giotto's revolution, Skira Editore, 2021. [17] A. M. Berg, S. T. Mol, G. Kismihók, N. Sclater, The role of a reference synthetic data generator within the field of learning analytics., J. Learn. Anal. 3.1 (2016).doi:10.18608/jla.2016.31.7. [18] G. Siemens, D. Gašević, Special issue on learning and knowledge analytics, in: Educational
Technology & Society, 15(3), 2012, pp. 1–163. [19] L. Lockyer, E. Heathcote, S. Dawson, Informing pedagogical action: Aligning learning analytics with learning design, in: American Behavioral Scientist, 57(10), 2013, pp. 1439–1459. [20] J. Li, Y. Yuan, Z. Zhang, Enhancing LLM factual accuracy with RAG to counter hallucinations: A case study on domain-specific queries in private knowledge-bases, in: arXiv preprint arXiv:2403.10446, 2024.