This paper presents a pedagogically grounded protocol for integrating generative AI into educational settings as a dialogic partner, rather than a source of ready-made answers. Drawing on Bruner's theory of education and dialogic instructional models, the protocol leverages Socratic questioning and “Devil's Advocate” strategies to promote metacognition, critical thinking, and learner awareness. A four-phase protocol guides students from task comprehension to reflective self-assessment through structured AI interaction. A pilot implementation in a university-level programming course showed that students perceived the protocol as helpful in supporting learning, problem-solving, and self-awareness, despite the higher cognitive demands. These findings suggest that dialogically structured AI interactions can foster slow, reflective learning and support the development of learning-to-learn skills.
The emergence of generative artificial intelligence (AI) poses significant pedagogical questions. While
AI holds considerable promise for enhancing learning through personalization, immediate feedback, and
adaptive support [
This study takes as its starting point the premise that well-established pedagogical methods for fostering metacognition and critical thinking can be enacted through an AI system that operationalizes their principles in a structured, scalable way.
As a concrete implementation of this approach, we propose a protocol in which AI is integrated into
classroom dialogue as a cognitive partner–enhancing, rather than replacing, pedagogical interaction.
Our approach is grounded in Bruner’s theory of education as a cultural and dialogic process [
Building on this foundation, we develop a protocol of AI-mediated dialogic inquiry, informed by the
instructional principles articulated by Collins and Stevens [
The novelty of our contribution lies in the formalization of this protocol: a principled integration of dialogic educational roles into AI behavior, with the goal of fostering metacognition, critical thinking, 2nd Workshop on Education for Artificial Intelligence (edu4AI 2025, https:// edu4ai.di.unito.it/ ), Co-located with ECAI 2025, the 28th European Conference on Artificial Intelligence which will take place on October 26, 2025 in Bologna, Italy * Corresponding author. $ marta.valentini-1@unitn.it (M. Valentini); alberto.montresor@unitn.it (A. Montresor) 0009-0006-7310-1891 (M. Valentini); 0000-0001-5820-8216 (A. Montresor)
© 2025 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
and learner self-awareness. Beyond task support, it helps students learn how to use AI in reusable,
controllable ways, via concrete and repeatable routines. Our central research questions are:
• How can a protocol based on AI-mediated dialogic inquiry be designed to foster learning?
• How is the proposed protocol perceived by university students in terms of supporting metacognitive
awareness and critical thinking?
To explore these questions, we design an AI peer agent that alternates between two dialogic functions:
• Socratic questioning: a disciplined sequence of open-ended questions designed to help learners
surface assumptions, diferentiate what is known from what is not, and critically examine their
reasoning [
This paper presents a first proof-of-concept implementation of the protocol in a university-level programming course. Qualitative exploratory data were collected from pre- and post-intervention questionnaires and AI–student interaction logs. These findings will inform a subsequent implementation in an introductory course, with more systematic data collection and evaluation.
Recent research has raised concerns about the use of AI in education when it replaces, rather than
supports, student learning [
We situate our approach within the tradition of dialogic inquiry [
To this end, our protocol integrates two complementary dialogic strategies: Socratic questioning and
Devil’s Advocate interventions. These are well-established pedagogical techniques for fostering critical
thinking and self-awareness [
Recent empirical work suggests that AI systems embodying these roles can support deeper
engagement than traditional, answer-oriented models. For example, Favero et al. [
An important feature of our approach is its attention to the regulation of cognitive load and pacing.
Learning is not instantaneous: it requires time for uncertainty, error, and revision. Our protocol
accommodates this by activating the AI only after the learner has engaged autonomously with the task,
thus supporting slower, more deliberate thinking processes [
In doing so, our work contributes to emerging eforts aimed at formalizing pedagogically grounded
interactions with AI to foster metacognitive awareness and critical thinking [
The protocol, structured into four distinct phases, guides students from their initial engagement with the task through to metacognitive reflection (and, where applicable, collective sharing). Its primary objective is to foster students’ ability to formulate high-quality questions, engage in critical reasoning, develop metacognitive awareness, and build autonomy in approaching new problems–through a structured interaction with an AI agent. Figure 1 summarizes the dialogic protocol we used: students first attempt the task independently and document their reasoning, then select one of two guided AI interventions (Socratic or Devil’s Advocate), and finally complete a critical review form (GF) before submitting. This visual provides the end-to-end flow we analyze in the following section. The protocol can be applied individually or in small groups.
Phase
Core Student Activity
Illustrative interaction 0 – Task Assignment 1 – Question Generation & Refinement 2 – Response Drafting & AI Feedback 3 – Validation & eAnngsawgeeringumideitnagcoqguneisttivioens, Metacognitive Reflection
Review the GF task brief, Task excerpt: reorganize the package hierarchy; introduce an interface identifying how the project must that exposes the core methods; rename the legacy class to remove be extended through structural ambiguity; move shared logic into an abstract base class; and implement a refactoring. specialised concrete subclass.
Generate an initial question set, S → AI: “What is an abstract class? How can I refine this question or what iterate with AI-driven else should I ask?” refinements, and pick the best AI → S: “Try these prompts: How does an abstract class differ from an using the mini-rubric. interface? When is an abstract class preferable to interfaces alone?” Develop a preliminary solution S → AI: “Here are my answers. Play devil’s advocate—challenge my and ask the AI for Socratic assumptions and surface any weaknesses.” and/or Devil’s Advocate AI → S: “You say an interface is effectively an abstract class, but that isn’t questions. accurate. Can you explain the difference and typical use cases for each?”
GF: “After the AI session, did you revise your answers? Did you ask yourself:
What visibility should the fields have? And the methods? How can you reflection, and label good vs. improve your interaction with the AI? Review good vs. poor prompts to poor prompt examples. improve future interactions."
FG= Form Guida P hS=asstuede0n:teTask Assignment: The instructor introduces the task–for instance, the implementation of a code component–along with general guidelines and task-specific instructions. During this initial phase,
Si tratta di estratti dai log. students become familiar with the objectives of the activity and with the operational format they will follow in the subsequent phases.
Phase 1: Question Generation and Refinement: Students independently generate an initial set of questions, using metacognitive transfer techniques if needed to stimulate divergent thinking. They then submit these questions to the AI with prompts such as “What other questions should I consider?” or “How can I improve my questions?”. From the AI’s suggestions, students select the most relevant questions and refine them using a rubric designed to assess their relevance and depth. Phase 2: Task Execution and AI Feedback: Students respond to the final set of questions and work independently to complete the task. At this point, the AI intervenes in one or both of the following ways: using a Socratic approach, it poses guiding questions to support the logical progression of student reasoning; using a Devil’s Advocate approach, it challenges assumptions through provocative questions designed to expose weaknesses and stimulate further reflection. Students deliberately choose one of these guided AI interventions; the choice is not randomized but aligned with their perceived need. This fosters a metacognitive process that strengthens students’ self-awareness and their awareness of AI use. Phase 3: Validation and Metacognitive Reflection: A structured Google Form presents guiding questions focusing on both the content of the task and the interaction with the AI, that encourage students to reflect on the overall efectiveness of the process and to identify and correct any conceptual misunderstandings. Finally, they select examples of “good” and “poor” prompts and justify their classification from a metacognitive perspective in a shared document. Instructors did not pre-validate AI turns; chat logs were monitored only to verify protocol flow and extract illustrative snippets.
Context and setting Programmazione 2 (CS2) is an advanced course in object-oriented programming in Java, delivered during the 2nd semester of the 1st year of the Computer Engineering degree at the University of Trento. Alongside lectures, students attend in-person lab sessions. The protocol was implemented during lab session 2, involving approximately 86 students organized into 29 groups of 2–3 members each. Two additional sessions (lab sessions 6 and 7) applied the same protocol, but involved only a limited number of students and did not yield analyzable data; hence, they are not discussed here. Learning tools Each student group was provided with: (i) a dedicated instance of ChatGPT-4o, supplied by the university and configured to centrally record interaction logs—this traceability will allow for a later analysis of student–AI interaction and identification of pedagogical refinements; (ii) a structured Google Form (GF) guiding students through the activity step by step: each phase of the protocol (question generation, AI feedback, metacognitive reflection, etc.) included instructions and optional ifelds to record questions, answers, and observations; (iii) a short evaluation rubric used to assess the quality of the questions (both student-generated and AI-suggested) and to support decisions about which to explore or discard; (iv) a collaborative document in which students recorded examples of “good” and “poor” prompts, along with a metacognitive justification of their choices. Procedure After a brief theoretical introduction emphasizing the value of reflective–rather than passive–use of AI, students were presented with the four-phase protocol and the supporting digital tools. They then began working on the first of seven tasks comprising the lab activity, managing their time autonomously to complete the remaining ones. Figure 1 illustrates the application of the protocol to Task 1 (Refactoring of the Block class hierarchy) as an example. After completing the first task, students applied the same sequence to the six remaining tasks.
In line with our research questions on fostering deeper learning and supporting metacognitive awareness and critical thinking, the data analysis conducted in this study was descriptive and exploratory, consistent with the proof-of-concept nature of our proposal.
Data were collected from two anonymous questionnaires administered before and after the lab activity. The pre-questionnaire, completed by 86 students at the beginning of the course, focused on students’ previous experience with AI tools, their beliefs about AI’s usefulness, and some metacognitive aspects. The post-questionnaire focused on students’ subjective evaluation of the protocol and the learning experience. Chat logs were not used for data analysis but only to monitor the flow of interaction and extract illustrative examples (see Figure 1). The post-questionnaire was administered after the final session, near the end of the course, when participation was limited. Only 17 students completed the form. As items were context-specific and self-reported, and the post sample was small, results are exploratory and should be interpreted with caution.
Although not designed for direct comparison with the post-questionnaire, the pre-questionnaire provided useful context. About 71% (61/86) of students reported daily use of AI tools, and 86% (74/86) rated positively their familiarity1. AI was considered useful primarily for identifying bugs in code (77%, 66/86) and generating test cases (77%, 66/86). In contrast, its usefulness for learning to debug or test was rated lower (56%, 48/86 and 52%, 45/86 respectively). Open-ended responses mainly framed AI as a 1On a 5-point scale, we considered score 4 and 5 as positive practical support tool, with few references to metacognitive or reflective uses. This context informed the interpretation of the post- data, highlighting how the protocol may have expanded students’ perception of AI toward reflective use.
Regarding the post-questionnaire, the data suggest a positive evaluation of the protocol. Students found the strategies helpful for improving their awareness and problem-solving skills. Specifically, 53% (9/17) and 59% (10/17) rated the Socratic questioning phase and the Devil’s Advocate phase positively, respectively; 47% (8/17) appreciated the metacognitive transfer phase, and 47% (8/17) valued the final review phase. These results indicate a perceived added value from structured AI interaction in terms of clarity, awareness, and reasoning stimulation.
Regarding perceived learning outcomes, 65% (11/17) of students agreed that the protocol helped them identify errors, 65% (11/17) reported it improved their testing abilities, and 82% (14/17) said it helped them better understand core Java concepts. Despite the increased cognitive efort required, students rated the learning benefits as worthwhile. Open-ended feedback acknowledged the length of the activity but emphasized its usefulness in supporting deeper conceptual understanding.
On critical thinking, 53% (9/17) of students agreed that they critically evaluated the AI’s questions and reflected more deeply on the task because the AI ofered questions instead of direct answers.
In terms of metacognition, 65% (11/17) reported that the protocol helped them reflect on their own learning process, and 59% (10/17) stated that the Socratic and Devil’s Advocate phases helped them question their assumptions and hypotheses. These findings suggest that guided AI interaction, when structured through an appropriate protocol, can promote reflection on learning processes–supporting not only content acquisition but also the development of learning-to-learn strategies.
This paper introduced a four-phase protocol that reframes AI from an answer provider to a thinking companion capable of enacting established pedagogical methods. We examined how such a protocol, grounded in dialogic interaction, can foster students’ critical thinking and metacognitive awareness.
In this preliminary implementation, the protocol appeared to prompt students to reason critically about both the task at hand and the AI’s responses. The analysis of the post-questionnaire answers suggests that the protocol was perceived as a meaningful support for learning.
The protocol promotes learning by requiring students to generate and refine their own questions (activation), engage with targeted challenges that induce cognitive conflict, and reflect explicitly on their learning process (metacognition). It fostered deliberate, slow thinking, which students valued despite the additional time required. The four phases of the protocol structure cognitive engagement and transforms the AI into a dialogic scafold. As highlighted in the literature, passive use of AI can reduce students’ cognitive efort but leads to impoverished learning; in contrast, structured AI interaction requires greater investment but promotes growth and metacognitive awareness.
However, several limitations must be acknowledged. The study was conducted in a single disciplinary context, involved a small sample limiting the generalizability of findings. Moreover, the analysis relied partly on self-reported data and did not include objective measures of critical thinking and metacognition.
These limitations guide our future research agenda. A follow-up study in an introductory programming course will implement the protocol multiple times and incorporate validated rubrics to assess metacognitive awareness and critical thinking. Interaction logs will be analyzed through both qualitative and quantitative methods, focusing on the evolution of dialogic practices over time.
Future work will also involve designing and testing additional interaction protocols grounded in other well-established pedagogical frameworks, to explore how efectively they can be amplified through AI support. These trials will include applications in non-STEM disciplines, examining how diferent subject domains may shape the suitability and impact of protocol-mediated AI scafolding. In parallel, we plan to extend the system architecture to build a community of AI agents capable of interacting with each other and with the learner.
The planned extensions converge on a single overarching aim at the heart of education: fostering students’ active construction of knowledge and metacognitive awareness. Structured dialogue with AI, governed by pedagogically grounded protocols, can provide a practical path toward achieving this goal. Declaration on Generative AI During the preparation of this work, the authors used ChatGPT in order to: Grammar and spelling check, Paraphrase and reword. After using this tool/service, the author reviewed and edited the content as needed and take full responsibility for the publication’s content. [18] H. Fakour, M. Imani, Socratic wisdom in the age of AI: a comparative study of ChatGPT and human tutors in enhancing critical thinking skills, Frontiers in Education 10 (2025) 1528603.