The increasing availability of easy-to-access generative Artificial Intelligence (AI) tools, like ChatGPT, calls for a need in education to devise new learning scenarios in view of their potential and challenges. Learning Analytics (LA) can play an important role in the understanding and optimization of responsible and reflective uses of AI tools in education. This paper contributes to the exploration of this role, adopting a human-centered perspective. The paper studies the perspectives of professors and students participating in a 'generative AI for learning' training at a public university in Spain. Considering these perspectives, the paper discusses new requirements for learning analytics in these new learning scenarios using AI. The perspectives highlight the potential of these tools as learning assistants, enabling improved use of study time, stimulating creativity, and facilitating personalized feedback. Stakeholders also points out several ethical concerns and risks that may hinder learning. These preliminary results emphasize the need for LA to differentiate between AI-assisted and AI-complement actions and human intelligence at work, aligned with pedagogical intentions. The paper formulates high-level constructs for learning analytics differentiating those actions, illustrated with examples. The paper also discusses that ethical concerns, like student inequality in accessing advanced tools, should be factored into analytics and decision-making tools.
Generative Artificial intelligence (AI), including ChatGPT and many other text-to-everything
tools, has revolutionized the concept of creation over recent months. The revolution is justified as
it is currently very difficult to detect if text, images, or sound have been created by a human or
generated by technology, with productions that can reach (relatively) good quality levels [
The opportunities essentially focus on their potential role to act as “assistants” in the creation process undertaken with learning purposes i.e., if used in a mindful way by humans [7]. Examples of assistance include support to inspiration (e.g., brainstorming arguments – or software code, or images in text-to-image …), facilitating scaffolding and self-evaluation (e.g., feedback on one’s own writing), or engaging in content-rich conversations (e.g., with simulated experts, in Socratic conversations, adaptive questioning) [8]. The challenges relate to ethical implications affecting academic integrity (e.g., plagiarism or fraud), trust (biases and reliability concerns in the generated text), the interest behind organizations providing the tools (e.g., commercial interests, power concentration), privacy, environmental impact, and inequality in tool accessibility (as knowledge and resources are required to exploit these tools to their full potential) [9]. These opportunities and challenges become more significant when considering the new skills learners might require in a world with AI [10].
All these considerations pave the way not only for new learning designs that employ AI tools to aid in learning tasks [6, 7, 11], but also for novel methods of providing student feedback [12], personalized learning [13], and advanced techniques for analyzing educational data [14]. The emergence of new educational scenarios and analysis techniques influences the formulation of learning analytics, both in terms of data collection in alignment with learning designs, and available analysis methods, all of which come with heightened ethical considerations [15, 16]. This paper contributes to the exploration of these implications, adopting a human-centered perspective in this initial exploratory phase. It does so by amplifying the voices of higher education professors and learners through a comprehensive participatory process [11]. The data was collected as part of 'generative AI for learning' training initiatives aimed at professors and students across all disciplines at a public university in Spain. The primary research questions driving this work are: RQ1. What opportunities do educators and learners foresee in generative AI for facilitating innovative learning designs? RQ2. What are the challenges? RQ3. To what extent does this introduce new requirements and ethical implications for learning analytics? Section 2 focuses on the study with educators and students answering RQ1 and RQ2, while section 3 elaborates an answer to RQ3 considering the results of the study.
To study the opportunities and challenges that educators and learners foresee in generative AI for facilitating the creation of innovation learning designs (RQ1, RQ2), we collected the perspectives of these stakeholders in the context of training courses devoted to the topic of 'generative AI for learning'. In the case of the professors, the training was a professional development action offered by a public university in Spain open to its professors (all disciplines in the university). In the case of the students, the training was offered as an optional course (with credit recognition) and it was also offered to all students in the university (from undergraduate to master and doctorate). So, participants were recruited via an open call within the university. The number of interested professors and students exceeded the number of available spots, so more courses will be offered in the future. In this paper, we analyze data of the first activity proposed in the courses. That activity involved reflecting (anonymously, voluntarily) on their prior knowledge and views regarding the opportunities and challenges that generative AI presents for education. Out the total number of participants, 19 students and 16 professors shared with the researchers their reflections for analysis (in a written document with two open questions for opportunities and challenges). Table 1 presents an analysis of the reflections from both students and professors. The content analysis was inductive and emerging categories were only counted one by participant. Regarding opportunities (RQ1), students (13 out of 19) emphasized the potential to make better use of their study time due to the capabilities of generative AI. These capabilities include providing rapid, personalized feedback and assistance with specific tasks (6/19), stimulating creativity (3/19), and offering flexible learning times (2/19). Professors shared these opinions, citing the opportunity to focus more on worthwhile, higher-order learning tasks (5/16) that foster the development of critical thinking skills. Both groups, particularly educators, expressed the necessity for a shift in the teaching role towards more of a tutoring approach. When it comes to potential challenges (RQ2), students expressed particular concern over the reliability of AI-generated text (10 out of 19) and privacy issues related to user data (4/19). Professors displayed more concern towards academic fraud and the design of assessments (13/16), though they also acknowledged the reliability challenge (7/16). Furthermore, some students voiced worries about potential cognitive dependency on these tools, a decrease in human cognitive efforts, and other factors that might impede learning. Professors, on the other hand, recognize the need to reevaluate their learning designs, aiming to limit the potential for thoughtless tool usage (particularly when used outside the classroom), mitigate inequality effects, and better highlight the importance and value of teacher-led learning sessions.
These results stress specific requirements and ethical implications for learning analytics (RQ3). Stakeholder comments stressing the role of AI as assistants in a learning process that is more efficient, but should be also effective (so it does not limit but boost the learning), but also as complements (see students’ perspectives on the opportunities, e.g., in the generation of some products, Table 1) suggest requirements for the formulation of learning analytics constructs. In particular, they suggest that learning analytics should be capable of differentiating among several types of learners' actions listed in Table 2. “Human intelligence (HI)” being practiced (e.g., composing one’s own self-explanation) (In Table 1, Opportunity to practice critical thinking skills, interdisciplinarity skills, … Boost creativity…, Limit problems: Undesired decrease of human effort, cognitive dependence…, address reliability and bias… ). “AI complement (AIc)” enhancing the authenticity or interest of the learning process (e.g., creating a visual representation of the self-explanation) (In Table 1, Complement processes through the generation of products (images, prototypes) not expected to be created as part of the learning objectives).
The analytics about learners’ actions should be in alignment with the pedagogical intentions in the learning designs. These designs may focus on developing higher-order thinking skills and nurturing human intelligence capabilities, through reflective and critical utilization of generative AI tools (see opportunities and challenges pointed out by the stakeholder). For example, a learning design may entail an expectation of learners’ activities in which HI actions happen after every AIa action (e.g., improving one own self-explanation after each brainstorming iteration). Another example may involve interactions with the AI (AIc) that complement previous learning activities completed by the human (HI) (Table 3).
Example of specific scenario In a scenario of GenAI as a support to inspiration (e.g., brainstorming arguments): the educator may expect students improve their own self-explanation (need for analytics of action HI), after each brainstorming iteration prompting the AI (need of analytics for action AIa). 2 A learning design may entail an expectation for a In a scenario of GenAI as a support to visualize ideas le learning activity in which after “Human intelligence (HI)” (e.g., image generation): xapm aaccttiioonnss,. students perform a “AI complement (AIc)” tchoenceedputcbatyotrhme mayseelxvpeesc(tnseteuddefonrtsatnoaldyetivcesloofpaaction E HI) and after that, to generate an image that visually represent the concept (need of analytics for action AIc).
Some stakeholders, especially professors, stress facets of the ethical implications derived from the availability of those AI tools. Efforts in learning analytics should consider the ethical implications tied to inequality (e.g., avoiding the penalization of learners who do not use certain versions of tools that require payment or data sharing) and environmentally costly uses that do not justify the learning gains.
This paper presents a preliminary human-centered exploration towards understanding the implications that generative Artificial Intelligence brings to the design and implementation of Learning Analytics. It aims at initiating the basis for continued investigation and stakeholder engagement.
The study underscores the necessity for the evolution of teaching roles, learning designs, and learning analytics to accommodate these tools responsibly. It emphasizes the need for analytics to differentiate between AI-assisted actions and human intelligence at work, aligned with pedagogical intentions. These perspectives of professors and students suggest a formulation of high-level constructs for LA differentiating several types of learners’ actions (AI assistant, Human intelligence, AI complement).
In the future, more research is needed to iterate and consolidate the constructs, also by defining more fine-grained constructs that differentiate between different types of assistance, human intelligence training, and complements. Stakeholders perspectives analyzed in this paper already distinguish different types of assistance (Table 1), while the types of human intelligence actions (e.g., human reflection, types of human adaptation of AI outputs, … ) needs further exploration. This study will be also extended in the future with additional perspectives from the stakeholders, also incorporating their thoughts and proposed examples after the training about the functioning of generative AI and its impact on education.
Moreover, the LA community need to develop understanding about what analytical techniques are more suitable to analyze student interactions with generative AI. The tensions arising from ethical implications, such as inequality (not all students have access to the most advanced tools), should be considered in the analytics and the subsequent decision-making support tools, like visualizations or in recommendations.
LA plays a crucial role in the goal of fostering critical and reflective use of AI tools in education, enhancing learning while addressing concerns.
This work has been partially funded by AEI/10.13039/501100011033 (PID2020-112584RB-C33, RED2022-134284-T, CEX2021-001195-M). The author also acknowledges the support by ICREA under the ICREA Academia programme and the Department of Research and Universities of the Government of Catalonia (SGR 00930). [5] Sharples M. Automated essay writing: an AIED opinion. International Journal of Artificial Intelligence in Education. 2022 Dec; 32(4):1119-26. [6] Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E. and Krusche, S., ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences (2023): 103, p.102274. [7] Salomon, Gavriel, David N. Perkins, and Tamar Globerson. "Partners in cognition: Extending human intelligence with intelligent technologies." Educational researcher 20, no. 3 (1991): 2-9. [8] Sabzalieva E, Valentini A. ChatGPT and artificial intelligence in higher education: quick start guide.
UNESCO report, 2023. URL: https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPTand-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf [9] Hernández-Leo, D., Directions for the responsible design and use of AI by children and their communities: Examples in the field of Education, In Charisi, V., Chaudron, S., Di Gioia, R., Vuorikari, R., Escobar Planas, M., Sanchez Martin, J.I. and Gomez Gutierrez, E., Artificial Intelligence and the Rights of the Child : Towards an Integrated Agenda for Research and Policy, EUR 31048 EN, Publications Office of the European Union, Luxembourg, 2022, ISBN 978-92-76-51837-2, doi:10.2760/012329, JRC127564, pp. 73-74. [10] Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence (2022): 3, 100056. [11] Qadir J. Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON) 2023 May, pp. 1-9.
IEEE. [12] Dai, W., Lin, J., Jin, F., Li, T., Tsai, Y.S., Gasevic, D. and Chen, G., Can large language models provide feedback to students? A case study on ChatGPT. Preprint 2023, URL: https://edarxiv.org/hcgzj?trk=public_post_main-feed-card-text [13] Koyuturk, C., Yavari, M., Theophilou, E., Bursic, S., Donabauer, G., Telari, A., Testa, A., Boiano, R., Gabbiadini, A., Hernandez-Leo, D. and Ruskov, M., Developing Effective Educational Chatbots with ChatGPT prompts: Insights from Preliminary Tests in a Case Study on Social Media Literacy. arXiv preprint 2023, URL; arXiv:2306.10645. [14] Amarasinghe, I., Marques, F., Ortiz-Beltran, A., and Hernández-Leo, D., Generative Pre-trained Transformers for Coding Text Data? An Analysis with Classroom Orchestration Data, 18th European Conference on Technology Enhanced Learning, Aveiro, Portugal, 4-8 September 2023. [15] EC, Digital Education Action Plan (2021-2027) https://education.ec.europa.eu/focus-topics/digitaleducation/action-plan Accessed 01/06/2023. URL: https://www.eduref.net/features/what-grades-can-aiget-in-college/. [16] Sarmiento, J.P., and Wise, A.F, "Participatory and co-design of learning analytics: an initial review of the literature." In LAK22: 12th International Learning Analytics and Knowledge Conference, 2022, pp. 535-541.