Persuasive AI Feedback: Enhancing Student Emotions and Engagement in Higher Education Omar Alsaiari1,2 , Nilufar Baghaei1 1 School of Electrical Engineering and Computer Science, The University of Queensland ,St Lucia 4072, Australia 2 Department of Computer Science, College of Science and Arts, Najran University, Sharurah 68341, Saudi Arabia Abstract This study focuses on the employment of persuasive technology with artificial intelligence (AI) to enhance student’s emotions and engagement in educational environments. By integrating persuasive elements such as praise, personalization, reminders, and emojis into AI-driven feedback, we aim to bridge the gap in understanding their effects on students’ emotions and engagement levels. Anchored in the control and value theory of achievement emotions, our mixed-methods research will assess these impacts using the Achievement Emotions Questionnaire-Short (AEQ-S) and engagement data from the RiPPLE platform. We hypothesize that such AI feedback strategies will significantly improve students’ emotional experiences and engagement with learning tasks. This inquiry contributes to educational technology by offering insights into designing emotionally intelligent AI feedback systems, potentially enriching the learning experience for university students. Expected outcomes include practical guidance for leveraging AI in creating more engaging and supportive educational settings. Keywords Persuasive AI, AIED, Student Engagement, Control-Value Theory, Learnersourcing 1. Introduction The integration of Artificial Intelligence (AI) in education has opened new frontiers in how learning experiences are designed and delivered. Among these advancements, AI-driven feed-back mechanisms stand out for their potential to significantly enhance student engagement and improve learning outcomes [1]. This study is situated at the intersection of persuasive technol-ogy and AI. This approach leverages AI systems to deliver feedback that is not only informative but also motivational, incorporating elements such as praise, personalization, reminders, and the use of emojis (see Table 1) to enrich the learning experience of university students. Despite the growing interest in applying AI in educational contexts, there remains a notable gap in understanding the specific impact of these persuasive elements on students’ emotions and level of engagement. Addressing this gap, our research draws on the control and value theory of achievement emotions to provide a theoretical framework for assessing how persuasive AI influences students’ emotional responses with educational tasks. The theory posits that students’ emotions are significantly shaped by their perceptions of control over and the value of their learning activities, suggesting that AI-driven feedback designed with persuasive elements could positively affect these perceptions. In: Kiemute Oyibo, Wenzhen Xu, Elena Vlahu-Gjorgievska (eds.): The Adjunct Proceedings of the 19th International Conference on Persuasive Technology, April 10, 2024, Wollongong, Australia EMAIL: uqoaslai@uq.edu.au (O. Alsaiari); n.baghaei@uq.edu.au (N. Baghaei) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings The aims of this study are twofold: first, to evaluate the extent to which persuasive AI can enhance students’ emotions, and second, to increase students’ engagement with the RiPPLE platform and the AI feedback tool. To achieve these objectives, we will employ a mixed-methods approach, centering around the Achievement Emotions Questionnaire-Short (AEQ-S) [2] for a nuanced measurement of emotional responses, alongside with an analysis of quantitative data on engagement extracted from RiPPLE. By exploring the potential of persuasive AI to foster more engaging and emotionally sup- portive learning environments, this study aims to contribute valuable insights into the design of AI systems tailored for educational purposes. The anticipated findings are expected to not only advance academic knowledge in the domain of educational technology but also offer practical guidance for educators and developers seeking to leverage AI to enhance the quality of educational experiences. Through this research, we hope to illuminate pathways toward more effective, engaging, and emotionally resonant educational technologies. Study Hypotheses: H1: Students receiving generative AI (GPT-3.5) feedback on the quality of their work that includes persuasive elements(such as praise,reminder, and visual aids like emojis) will report higher levels of positive learning-related emotions (e.g., joy, pride) compared to students receiving generative AI (GPT-3.5) feedback that solely focuses on the qualitative aspects of student work without employing persuasive elements. H2: Students receiving generative AI (GPT-3.5) feedback that strictly addresses the quality of student work will report higher levels of negative learning-related emotions (e.g., anger, anxiety) compared to students receiving generative AI (GPT-3.5) feedback enhanced with persuasive elements. H3: The incorporation of persuasive technology elements in AI feedback (such as praise, reminder, and visual aids like emojis) will significantly enhance university students’ engagement with RiPPLE and AI feedback tool. H4: AI systems that include persuasive elements will be more effective in influencing students to adopt the feedback and suggestions provided, leading to improved learning outcomes. 2. Background and Related Work The fusion of technology and education has rapidly evolved, introducing persuasive technology and artificial intelligence (AI) to enhance learning experiences. Persuasive technology in education, through tools and strategies like persuasive SMS and web 2.0 applications, has shown to positively impact students’ learning attitudes and engagement, with studies by Behringer et al.[3], Goh et al.[4], Filippou et al.[5], and Widyasari et al.[6] underscoring its effectiveness. Similarly, the domain of Artificial Intelligence in Education (AIEd) has expanded, with AI technologies including personalized reminders and automated feedback becoming increasingly prevalent. Despite its growth, the full pedagogical potential of AIEd and its implications for student agency and self-regulation are not yet fully understood [7, 8]. Recent advancements in AI-generated feedback have demonstrated its utility in enhancing student learning out- comes, with research showing its comparability or superiority to human feedback [9, 10, 11]. Nonetheless, concerns about students’ over-reliance on AI for learning support have been raised, highlighting the need for balanced integration [8]. Emotional persuasion within educational settings has also garnered attention, emphasizing the significant role of emotional and rational persuasion strategies in shaping students’ learning experiences and outcomes. The effectiveness of technology-mediated persuasion, such as ClassDojo, in fostering social-emotional learning, along with the impact of educators’ emotional displays on student attitudes, has been highlighted by Williamson (2017), Van Kleef et al.[12], and others [13, 14]. These studies collectively advocate for the strategic use of persuasive techniques to enhance students’ emotional engagement and overall educational achievement. This background underscores the critical intersection of persuasive technology, AI, and emotional persuasion in education, setting the stage for our investigation into how these elements can be cohesively integrated to optimize learning environments. Table 1 AI Feedback Persuasion Techniques Technique Example in Writing Feedback Personalization Feedback based on each student’s contributions and moderating style [15] Social Role & Ex- Designating an AI as a tutor, portraying it as a specialist in a particular field pertise [15] Emojis[16, 17, 18] Using emojis as a form of nonverbal and visual persuasion Reminder [15] When providing feedback, students are also reminded to complete any out- standing tasks Praise [15, 19] Acknowledging and valuing students’ hard work and contributions with ap- preciation and positive affirmation 3. Research Methodology 3.1. Introduction to RiPPLE RiPPLE, an adaptive educational system, is based on learnersourcing and is designed to involve students in creating various learning resources, such as multiple-choice questions (MCQs) [20, 21]. This process encompasses drafting question content, tagging relevant topics, generating potential answers, selecting the correct one, and formulating an explanatory rationale. Beyond MCQs, RiPPLE allows for the creation of worked examples and general notes, integrating diverse elements like text, images and videos. A significant feature of RiPPLE is its commitment to high-quality student-generated content. The system employs several moderation methods, including staff and peer reviews, and has recently integrated AI feedback. This feedback is used both before a resource is submitted and during the peer moderation process. Once resources are vetted and added to a course’s repository, they become available for others in the course to use, attempt, and provide feedback on. Moreover, users can rate and comment on these resources, fostering a collaborative and interactive learning environment within the platform. 3.2. Methodology This study will employ a mixed-methods approach, integrating both quantitative and qualitative analyses to investigate the emotional and engagement responses of university students to AI feedback within a learnersourcing environment. At the heart of our methodological framework will be the use of the Achievement Emotions Questionnaire-Short (AEQ-S) [2], designed to measure a wide array of emotions, including enjoyment, hope, pride, anger, anxiety, and shame. Participants will be asked to rate their emotional experiences on a scale from 1 (strongly disagree) to 5 (strongly agree), in alignment with AEQ-S’s aim to comprehensively assess academic emotions. Additionally, the study will incorporate an assessment of student engagement, drawing inspiration from the findings of Kay (2011) [22] on the impact of web-based learning tools. To quantitatively evaluate user engagement, we will analyze data derived from interactions on the RiPPLE platform. This data analysis is intended to provide insights into the behaviors and interactions of users within the platform, offering a valuable perspective on the influence of AI feedback on student engagement. By adopting this mixed-methods approach, we aim to gain a detailed understanding of how persuasive AI feedback affects students’ emotional states and their engagement with educational content. The findings from this study are expected to contribute to the enhancement of AI-driven educational tools. 3.3. Participants and Groups This study, approved by the Human Ethics Committee at the University of Queensland, will involve participants divided into two groups, each interacting with the RiPPLE platform in distinct ways: • Control Group: This group will interact with the RiPPLE platform receiving GenAI (GPT- 3.5) feedback that strictly addresses the quality of student work without any additional persuasive elements. This will serve as a baseline for comparison against the experimental group to evaluate the impact of standard feedback on student emotions and engagement. • Experimental Group: Participants in this group will receive GenAI (GPT-3.5) feedback that includes persuasive elements, such as praise, reminders, and visual aids like emojis. This is aimed at evaluating whether the inclusion of such elements can enhance positive learning-related emotions and increase engagement with the RiPPLE platform and AI feedback too. 3.4. Procedures The study will follow a structured process to ensure comprehensive data collection and analysis: 1. Participant Selection and Group Allocation: Students enrolled in a web design course will be randomly assigned to either the experimental or control group within the RiPPLE platform, to ensure an even distribution of participants. 2. Initial Engagement and Training Session: All participants will attend a lab session designed to introduce them to the functionalities of RiPPLE, which will include a detailed tutorial and a hands-on practical exercise. 3. First Questionnaire Administration: At the conclusion of round 1, a questionnaire will be administered to gauge initial emotional responses and engagement levels. 4. Long-term Assessment: To evaluate the enduring impact of persuasive AI feedback, a final questionnaire will be distributed at the end of the semester. 4. Future Work Our completed pilot study, soon to be published, involved 521 undergraduate and graduate students for a 13-week course in ”Introduction to Web Design.” The study provided key insights into student engagement and emotional responses, such as joy, yet noted that engagement predominantly adhered to the minimum requirements for grade contributions. Reflecting on these outcomes, our future endeavors will pivot towards a nuanced examination of the impact of visual versus textual persuasive elements within educational feedback mechanisms. We aim to dissect and compare the efficacy of these differing persuasive approaches in elevating student engagement and emotional reactions. This critical analysis is designed to uncover strategies that not only encourage students to exceed baseline engagement levels identified in the pilot study but also deepen their connection with the educational material. 5. Conclusion As we prepare to embark on this research, our goal is to explore the influence of persuasive AI elements, such as personalized feedback, social role and expertise cues, and the integration of emojis, on the emotional responses and engagement levels of students in higher education. The study, underpinned by the control and value theory of achievement emotions, seeks to not only interrogate these dynamics but also to frame a comprehensive theoretical understanding of the effects of persuasive AI in educational settings. While we are yet to collect empirical data, the anticipated insights aim to inform the design of AI feedback systems that are more attuned to the emotional and cognitive needs of students. This exploratory phase is crucial for setting the stage for future empirical research, enhancing our grasp of how AI can be optimized to foster more effective and emotionally engaging learning environments. We are eager to undertake this investigation and contribute to the broader discourse on employing AI to elevate the quality of educational experiences. References [1] F. Kamalov, D. Santandreu Calonge, I. Gurrib, New era of artificial intelligence in education: towards a sustainable multifaceted revolution, Sustainability 15 (2023) 12451. [2] M. Bieleke, K. Gogol, T. Goetz, L. Daniels, R. Pekrun, The aeq-s: A short version of the achievement emotions questionnaire, Contemporary Educational Psychology 65 (2021) 101940. [3] R. Behringer, M. Soosay, S. B. Gram-Hansen, P. Øhrstrøm, C. G. Sørensen, C. Smith, J. Mikulecká, N. Winther-Nielsen, M. Winther-Nielsen, E. Herber, Persuasive technology for learning and teaching–the europlot project, in: Proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching, 2013. [4] T.-T. Goh, B.-C. Seet, N.-S. Chen, The impact of persuasive sms on students’ self-regulated learning, British Journal of Educational Technology 43 (2012) 624–640. [5] J. Filippou, C. Cheong, F. Cheong, Designing persuasive systems to influence learning: Modelling the impact of study habits on academic performance (2015). [6] Y. D. L. Widyasari, L. E. Nugroho, A. E. Permanasari, Persuasive technology for enhanced learning behavior in higher education, International Journal of Educational Technology in Higher Education 16 (2019) 1–16. [7] O. Zawacki-Richter, V. I. Marín, M. Bond, F. Gouverneur, Systematic review of research on artificial intelligence applications in higher education–where are the educators?, Interna- tional Journal of Educational Technology in Higher Education 16 (2019) 1–27. [8] A. Darvishi, H. Khosravi, S. Sadiq, D. Gašević, G. Siemens, Impact of ai assistance on student agency, Computers & Education 210 (2024) 104967. [9] T. Wan, Z. Chen, Exploring generative ai assisted feedback writing for students’ written responses to a physics conceptual question with prompt engineering and few-shot learning, arXiv preprint arXiv:2311.06180 (2023). [10] M. Afzaal, A. Zia, J. Nouri, U. Fors, Informative feedback and explainable ai-based recom- mendations to support students’ self-regulation, Technology, Knowledge and Learning (2023) 1–24. [11] T.-S. Chang, Y. Li, H.-W. Huang, B. Whitfield, Exploring efl students’ writing performance and their acceptance of ai-based automated writing feedback, in: Proceedings of the 2021 2nd International Conference on Education Development and Studies, 2021, pp. 31–35. [12] G. A. Van Kleef, H. Van den Berg, M. W. Heerdink, The persuasive power of emotions: Effects of emotional expressions on attitude formation and change., Journal of Applied Psychology 100 (2015) 1124. [13] R. F. Adler, F. Iacobelli, Y. Gutstein, Are you convinced? a wizard of oz study to test emotional vs. rational persuasion strategies in dialogues, Computers in Human Behavior 57 (2016) 75–81. [14] L. Linnenbrink-Garcia, E. A. Patall, R. Pekrun, Adaptive motivation and emotion in education: Research and principles for instructional design, Policy Insights from the Behavioral and Brain Sciences 3 (2016) 228–236. [15] H. Oinas-Kukkonen, M. Harjumaa, Persuasive systems design: Key issues, process model, and system features, Commun. Assoc. Inf. Syst. 24 (2009) 28. URL: https: //api.semanticscholar.org/CorpusID:263865812. [16] J. Ge, U. Gretzel, Emoji rhetoric: a social media influencer perspective, Journal of marketing management 34 (2018) 1272–1295. [17] E. Ayan, et al., Descriptive analysis of emoticons/emoji and persuasive digital language use in whatsapp messages, Open Journal of Modern Linguistics 10 (2020) 375. [18] Z. Lv, Emojis or not? how emotional vividness influences brand attitude in social media, in: 2020 International Conference on E-Commerce and Internet Technology (ECIT), IEEE, 2020, pp. 159–162. [19] B. J. Fogg, Persuasive technology: using computers to change what we think and do, Ubiquity 2002 (2002) 2. [20] H. Khosravi, K. Cooper, K. Kitto, Riple: Recommendation in peer-learning environments based on knowledge gaps and interests, arXiv preprint arXiv:1704.00556 (2017). [21] H. Khosravi, K. Kitto, J. J. Williams, Ripple: A crowdsourced adaptive platform for recommendation of learning activities, arXiv preprint arXiv:1910.05522 (2019). [22] R. Kay, Evaluating learning, design, and engagement in web-based learning tools (wblts): The wblt evaluation scale, Computers in Human Behavior 27 (2011) 1849–1856.