Generative artificial intelligence (GenAI) is now capable of performing tasks that we have considered intellectually demanding. There are justified concerns that this will undermine the agency of both educators and students, if tools are poorly designed, poorly used, or imposed - with consequences for education and the future of work. This short paper contributes practical examples pointing the potential for GenAI to promote critical analysis as part of intellectually demanding tasks, by both students and educators. However, this depends on appropriate usage. The paper then briefly discusses how we may balance the benefits and risks of human cognitive offloading to AI, as a perspective on human agency.
2 https://twitter.com/ylecun/status/1659332688786882560
Social media platforms such as Twitter have established themselves as influential channels for public
discourse and opinion, although the quality of conversation is of course highly variable with platform
and community. In a tweet, a well-known AI researcher argued that “AI doomers”, who are proponents
of strong AI regulation, would also have called for the banning of pens and pencils. This is an argument
by analogy [
The examples so far have been very short: the arguments have made a single ‘move’, which GPT could recognise and comment on. Let us now consider a more complex case. In March 2023, a large number of eminent thinkers wrote an open letter calling for a pause in building large language models.3 Achieving widespread media coverage, this provoked extensive debate, including a letter of rebuttal from another set of academics and industry researchers.4 This seemed an authentically rich argument to test GenAI.
I asked Bing Chat (now Copilot) to access the letter online and identify the key claim and arguments. It provided a reasonable textual summary, output as a set of bullet points summarising key arguments. However, it is well established that students struggle to critique arguments, and that rendering them visually as an argument map can help them understand the key elements of the argument (this is a form of concept map tuned specifically to show multiple perspectives, and the key features of arguments such as supporting/challenging claims/evidence). I asked it to generate a map, but it could not. However, when asked, it confirmed that it understood Argdown, which is a markdown notation for argument maps. It generated this in a code window, which I pasted into the Argdown web app,5 resulting in a map (Figure 3).
Examination of the argument map reveals to what extent this was a rigorous analysis, but also illustrates ‘hallucination in argument mapping’ (Figure 4). Hallucinations of two types were found. Firstly, the red underline signals incorrect classification of a premise using incorrect, or indeed madeup argument schemes. There is to my knowledge no such argument type as Argument from responsibility, or Argument from precaution. Argument from omission seems to be a jumbling of Fallacy of omission and Argument from ignorance.
Secondly, there were hallucinated summaries. This node apparently reads well as a summary, but the authors do not talk about researchers at all.
Asking students to perform critical evaluations of AIgenerated argument maps should serve as assurance of learning about the subject matter, but can also provide important insights for them into the limitations of AI, if students are equipped and empowered to see through hallucinations. 3 https://futureoflife.org/open-letter/pause-giant-ai-experiments 4 https://www.dair-institute.org/blog/letter-statement-March2023 5 https://argdown.org
Conversational agents are exciting for education since they are, by definition, premised on learning through dialogue — hardly a novel concept. But consider this illustration of GPT’s capabilities (Figure 5).
Bing Chat’s Argdown code can also be rendered as textual outlines. Figure 7 shows the addition of the critical questions, and substitution with placeholders for students to complete.
An important educational question arises, as we see this kind of performance, namely, will the students engage in excessive cognitive offloading, and fail to learn how to do this themselves? We return to this in the discussion about user agency.
Thus far, we have focused on critical thinking and reflection around arguments, primarily with students
in mind, but equally, these are tools for any professional to test their thinking. In the next example, we
focus on a specifically instructional task, which harnesses the generative capability of LLMs more fully
to distill complex text into key themes. The text in this case is a specific ‘genre’ of writing, the Course
Intended Learning Outcome (CILO). CILOs define what students know and can do on successful
completion of the course. As part of a well-designed curriculum, each part of a course – subjects,
modules and assessments – should all respond to its CILOs. Effective implementation of CILOs
requires both the subject matter expertise of academics and the pedagogical knowledge of learning
designers (LDs). Indeed, recent evidence points to the benefits that academics gain from working with
LDs on their online teaching, and how this transfers to their in-person teaching [
One specific element in this task that academics can struggle with is to articulate good LOs. Furthermore, these typically vary widely in quality and quantity between academics. At UTS, we are working towards summarising all courses consistently using approximately six CILOs, to achieve a better user experience as students make enrolment decisions, and to assist teaching teams in their course design and reviews. However, it is an intellectually and linguistically demanding task to distill a list of 20-30 CILOs (which is not uncommon), down to six well designed CILOs, and the university needs to implement this summarization for its entire program.
It is here that we anticipated that LLMs could assist. GenAI intranets now provide universities with authenticated, secure, private services, integrated with other internal services, and tuned to support business processes.6 In a 2-day hackathon, iterative prompt engineering informed by feedback from academics and learning designers led to the refinement of a system prompt that configured ‘CILObot’, a ChatGPT to aid in drafting these new CILOs. The system prompt incorporates widely recognised design principles (e.g. open each CILO with a verb from Bloom’s Taxonomy), with the addition of internal requirements (e.g., UTS Indigenous-CILOs), and the chatbot is grounded in a corpus of documents about CILO design.
The prototype is showing promise, and after a day’s intensive work using the Azure ‘Chat Playground’ (the ChatGPT design environment), the results for several programs in our Health faculty were validated by disciplinary experts (e.g., Figure 8). CILObot generates a coherent first draft in about 30 seconds, which can of course then be refined through further conversation with it, and edited by the teaching team. We estimate that agreeing on how to distill 20-30 CILOs into 6 would normally be a minimum of 3 hours’ meeting between the Course Director and the program’s lead academics, which represents an impressive return on investment. Next steps will test CILObot with other degree courses. 6 cf. Ithaka SR project: Making AI Generative for Higher Education:
https://sr.ithaka.org/blog/making-ai-generative-for-higher-education-2/
These capabilities are, in my view, impressive. If students were to produce argumentative reasoning as presented above, we would surely conclude that they were thinking critically, and had mastered some argumentation principles. Similarly, if an academic proposed a set of six distinctive, well expressed CILOs with complete coverage of the original CILOs, we would regard that as exactly the kind of task senior academics should be capable of. The difference, of course, is that these tasks are performed in under a minute, producing coherent drafts.
We do not need to believe that agents have the same kind of understanding as people to appreciate the value of AI being able to communicate with this fluency and precision in order to provoke critical human reflection. GenAI performs these tasks in seconds, and can iterate its analysis as often as requested. In principle7, therefore, GenAI can be used to: • offer students, academics or any other kind of analysts instant, formative feedback on draft arguments, for instance, identifying points that could be potentially attacked; • analyse a written corpus to give insights into the quantity and quality of argumentation, which could inform both LA researchers, practitioners, and educators; • analyse a written corpus in order to derive a representative set of summary themes (noting that
AI cannot ‘read between the lines’ as a human qualitative analyst does). 7 These are in principle capabilities for GenAI argument analysis and feedback, since this has not yet been tested empirically with students, to the best of my knowledge.
The pivotal question — whether we are envisioning the future of learning among students or professionals in the workplace — is the “allocation of function” between human and machine, to use the original term from ergonomics. Questions of cognitive offloading and human agency now arise, as we consider different scenarios.
If AI improves short-term productivity (e.g., faster syntheses of complex information; more creative ideas; more incisive reasoning), we might anticipate (and indeed we are already seeing in certain professions) that AI apps will embed into professional work practices. Professionals are qualified to ‘drive’ such intellectual power-tools (in contrast to students their qualifications should enable them to recognise poor AI output); they will welcome cognitive offloading in their busy lives; and if they do not use AI may find they are unable to compete with those who do. We might see this as empowering professionals — and yet we might also see a loss of agency as they are essentially forced to use AI in order to compete. Time will tell if the long-term use of AI leads to the degrading of important human capabilities, just as GPS satellite route navigation has for many young people obviated the need, and hence ability, to navigate via printed maps.
In sharp contrast, for education the story is very different. “Productivity gains” need to be judged by
a different yardstick, since while an essay written solely by GenAI in 2 minutes is a “productivity gain”
in terms of artifacts/minute, the absence of the student’s cognitive engagement fails other “KPIs” for
meaningful education. Students must build their foundational knowledge, skills and dispositions, in
order to function as citizens and professionals in the myriad contexts in which they cannot call on AI,
but must think on their feet and demonstrate diverse intelligences [
Consequently, as emphasised in a recent national report for the higher education sector, assessment
must be reformed for the age of AI [
GenAI forces us to ask when we are — or should be, as the boundary shifts — assessing joint
human+AI system performance, versus capability without AI. A consequence of this distinction is that
we must cultivate “mindful engagement”, not “mindless engagement” [
CILObot would not have been possible without the joint expertise of my colleagues Sharon Coutts, Ann Wilson, Michaela Zappia (Institute for Interactive Media and Learning), Susan Gibson & Carl Young (Data Analytics and Insights Unit), and Miguel Ramal & Olaf Reger (IT Unit).