=Paper=
{{Paper
|id=Vol-3605/10
|storemode=property
|title=Do Not Ask What Instruction-Following Large Language Models Can Do for Teaching (short paper)
|pdfUrl=https://ceur-ws.org/Vol-3605/10.pdf
|volume=Vol-3605
|authors=Fabio Massimo Zanzotto,Emanuela Bizzarri,Daniele Pasquazi
|dblpUrl=https://dblp.org/rec/conf/aixedu/ZanzottoBP23
}}
==Do Not Ask What Instruction-Following Large Language Models Can Do for Teaching (short paper)==
https://ceur-ws.org/Vol-3605/10.pdf
Do not ask what Instruction-following Large Language
Models can do for teaching
Fabio Massimo Zanzotto1, Emanuela Bizzarri2 , Daniele Pasquazi1,2
1 University of Rome Tor Vergata, Viale del Politecnico, 1, Roma, 00133, Italy
2 Liceo Scientifico “Bruno Touschek”, Grottaferrata, Roma, Italy
Abstract
In this paper, we want to explore how and if the Italian School System has adapted to the evolution of
the machines that are replacing humans in many repetitive cognitive tasks. Indeed, Instruction-
following Large Language Models are only the last evolution in this trend of replacing activity.
Calculators, symbolic equation solvers, information retrieval engines, and much more have appeared
during a long period. Many reforms of the Italian School System have been approved during this period.
We show that this cognitive evolution of machines has yet to be completely taken into account in the
different reforms. Finally, we propose a possible novel route to investigate how to take into account that
machines can help humans in repetitive cognitive tasks.
Keywords
Large Language Models, Cognitive Tasks in Machines, Italian School System1
1. Introduction
Students are right when they cheat. Indeed, everyone “cheats” in the student sense during the
normal working routine. It is commonly accepted. When cheating during exams, students are only
simulating their normal use of the knowledge resources and tools during their working life.
Years ago, students cheated by using small hand-written notes carrying relevant information
and, for the braver, by using textbooks. During exams, students were exactly doing what was done
during a normal workday: consulting notes and specialized texts. The needed capability was to
search and select what is relevant among physically available books and personal notes, which
may be stored on shelves near the desk. Occasionally, additional information can be retrieved in
libraries. The required skill was to know that some additional information may exist and then be
able to perform the search.
More recently, students’ cheating ability has been largely improved by small devices that allow
them to access the complete available knowledge as they were workers at the desk who are
improving their ability by using a desktop computer - nothing that is forbidden in real working
life. This is the information retrieval age.
Even asking someone else to perform the task for you – the apparently worst form of student
cheating - is a commonly accepted working practice: consulting. It is true that consulting is
expensive, and it should be used only when definitely necessary. In this case, the required skill is
to be able to understand whether the cost represents the added value in the current job.
Today, Instruction-following Large Language Models (IFLMs)[1], [2] , such as the notorious
ChatGPT, may play the consultant role, and thus, these models are democratizing access to
consultancy. These IFMLs can solve the task for you using the body of knowledge available in the
information retrieval era and relieving workers and, consequently, students from the burden of
selecting relevant information and aggregating a response. This is the age of large language
models.
Proceedings Acronym: Proceedings Name, Month XX–XX, YYYY, City, Country
fabio.massimo.zanzotto@uniroma2.it (F.M. Zanzotto)
0000-0002-7301-3596 (F.M. Zanzotto)
© 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Indeed, wherever there is physical or cognitive repetitiveness, machines can do a better and
faster job than humans. Taylor captured physical repetitiveness and transformed the way of
producing physical goods. Taylorism \cite{Tailorism} combined with machines has reduced the
required workforce in industrial production processes focused on producing physical goods.
Indeed, Taylor had the idea to reduce complex production processes into more straightforward,
repetitive tasks. Hence, dedicated machines at first and general-purpose programmable robots
later have taken these simple repetitive tasks and replaced workers, shrinking the workforce
needed for producing goods. Along with algorithms and knowledge representation, computers –
or artificial intelligence – are conquering tasks with cognitive repetitiveness. This is an
opportunity and a compelling challenge for the school system.
In this paper, we want to explore how and if the Italian School System has adapted to the
evolution of the machines that are replacing humans in many repetitive cognitive tasks. Indeed,
Instruction-following Large Language Models are only the last evolution in this trend of replacing
activity. Calculators, symbolic equation solvers, information retrieval engines, and much more
have appeared during a long period. Many reforms of the Italian School System have been
approved during this period. We show that this cognitive evolution of machines has yet to be
completely taken into account in the different reforms. Finally, we propose a possible novel route
to investigate how to take into account that machines can help humans in repetitive cognitive
tasks.
2. Machines and the conquering of cognitive tasks
Machines have been always invented to help humans with tedious, hard, repetitive tasks or to
empower humans with superhuman abilities such as flying. The first tasks that have been
automated are physical tasks such grinding the grain but the idea of automating also cognitive
tasks such as adding numbers dates back in centuries.
In these days, technology is overwhelming and many cognitive tasks can be automated at least
partially. Then, the compelling question is why young humans should learn to do these repetitive
cognitive tasks?
In this section, we analyze how the evolution of the machines conquered cognitive tasks to try
to understand what humans should learn to do since they are relieved from repetitive cognitive
tasks.
2.1. Calculators in Our Pockets
Basic calculus has been and is one of the most important abilities thought in primary school. It is
a must for human beings to understand numbers and how they can be combined by using simple
operators like sum, subtraction, multiplication, and division. This is the basis for living a life with
numbers that may represent value – money – or quantities in the real world. Basic operations are
then used to build more complex calculus operations in later stages.
After a first, small period of understanding the basic ideas around numbers, nearly five years
of the fresh minds of young children are forged to become executors of the calculus algorithms of
the sum, the subtraction, the multiplication and, then, the division that is more complex. These
young minds are rewarded for their ability to perform the repetitive task of executing stable and
well-known calculus algorithms. Teaching future adults to perform basic, increasingly complex
computations is one of the primary goals of primary school.
However, since basic calculus is a repetitive task, the attempt to automate it dates back prior
the advent of electronic computers. Between 1642 and 1644, Pascal used gears to build the first
arithmetic machine, which was able to sum two numbers up to a given length. Electricity has
pushed these machines to the ability to solve multiplications and divisions. In the late ’50,
Olivetti’s Divisumma combined mechanical parts with electromagnetic components and offered
the possibility to reduce the burden of computation. These machines were already faster than
average humans to perform basic calculus. Transistors and micro-transistors gave the final
acceleration to the development of the spreading of computing machines able to perform basic
and advanced calculus of standard mathematical functions. Hence, performing calculus was not
more necessary at work and even at shops. Afterward, calculators and scientific calculators
became portable around the ’80s and then invaded students' pockets and backpacks. Students
could start to “cheat” in the sense that they could start behaving as if they were at their future
office desks.
2.2. Computers and Personal Computers
Computers, firstly, and personal computers, after, have deeply demonstrated that many of the
repetitive cognitive tasks may be replaced by using a software program. The majestic film
“Hidden Figures” clearly depict the situation. If tasks are cancelled by the ability of machines,
humans should start to know how to use the machine to go beyond. Then, the wide spread of
computers and personal computers in the ’70 wiped out many repetitive cognitive tasks but
opened the doors to different jobs. Hence, students should use these tools as soon as possible.
However, the first era of personal computers did not give the possibility to carry these
wonderful machines everywhere and, hence, students could not use it in classrooms. The real
game change happened when these machines where reduced to pocket size and have been called
smartphones.
Around 2010, smartphones have brought the full power of personal computers and the web
in the pockets of the students. From that point on, it has been possible to “cheat” at large scale.
Eventually paired with smartwatches, the full possibility of accessing the network and the
cognitive abilities of the information technology suddenly entered the classrooms. There is a
wonderful possibility of simulating the real working environment during an exam.
2.3. Symbolic Calculus in Our Pockets
Smartphones gave the possibility to use symbolic calculus during lessons and exams. Indeed,
many of the exercises that students are asked to resolve fall in the idea of searching in a graph
where nodes are equations and edges are transformation laws. There are many apps that may
start from hand-written equations and may resolve them in matter of milliseconds. For example,
PhotoMath (Figure 1) easily resolves equations, does function analysis and so on.
Figure 1: PhotoMath: an app for symbolic calculus
These symbolic calculus resolvers are very useful tools from the point of view of a potential job
involving resolving equations. Then, apparently, there is no need to bother students with these
repetitive exercises. It is more convenient to spend time of students in different activities such as
deeply understanding the principles that are the basis of these equation transformations and
what their limitations are.
2.4. The Knowledge of the World in Our Pockets
Moreover, smartphones gave the possibility of accessing the whole knowledge of the web during
exams. This strongly replicate how people work today. It is generally neither possible nor useful
to try to remember every detail when it is possible to access details when needed.
Students should be able to:
• Search efficiently and to have a clue to where to search. Having all the information on the
tip of the fingers may be useless if there is not the capacity to search it.
• Fastly grab useful information from the results of a search engine. This is a very difficult
task as it consists of selecting the right documents to read and extracting the relevant
information.
Hence, when only search engines were there, exams should be designed to test these abilities
in students. There was no reason to stop students using these machines.
3. The reaction of the Italian School System
The school system underwent many reforms since the unification of Italy (see Table 1).
The first goal of the school of the united Italy was combatting the analphabetism of the Italian
population and giving the possibility to understand basic calculations. Legge Coppino introduced
sanctions for parents which were not respecting the compulsory and free education offered to
their children. Attending classes was compulsory for elementary cycle. Then the age was
progressively increased to the age of 14.
The second goal of the school system was preparing the elite and the working class. Until 1962,
the choice to be in one of the two classes was made just after the primary school. Only with Legge
31/12/1962 n. 1859, a unified cycle for children aged 11-14 has been established and this cycle
gave access to all secondary schools. Moreover, only with the LEGGE 11 dicembre 1969, n. 910
many secondary schools gave the possibility to access to any university faculty with one eventual
additional year of study. Then, the school system become a way to train citizens that may perform
any role.
Table 1
Some of the Italian Laws related to the School System
Year Law
1859 Legge Casati (Regio Decreto 13/11/1859 n.3725)
1877 Legge Coppino (Legge 15/07/1877)
1904 Legge Orlando (Legge 08/07/1904)
1911 Legge Daneo-Credaro (Legge 04/06/1911)
1923 Riforma Gentile (Many different Laws)
1939 Carta della Scuola Bottai
1962 Legge 31/12/1962 n. 1859
1997 Legge Bassanini (Legge 15/03/1997)
2003 Riforma Moratti (Legge 28/03/2003)
2008 Riforma Gelmini (Decreto Legge 01/09/2008 n. 137)
Although Legge Bassanini (1997) gave autonomy to schools, local schools had the freedom to
define the content of the courses only in 2003 and 2008 with the two reforms Moratti (2003) and
Gelmini (2008). These two reforms introduced the National Guidelines (Indicazioni Nazionali)
that replaced the Ministerial Syllabi (Programmi ministeriali). This change has given the
possibility to local schools and to local instructors to adapt the syllabus of their course to the local
curriculum provided that this adaptation is in line with the National Guidelines.
Moreover, the National Guidelines expressed in the Attachments to the Gelmini Reform are
extremely detailed for what concerns the use of machines. Many of the Attachments contains the
following words:
“The computer tools available today provide suitable contexts for representing and
manipulating mathematical objects. The teaching of mathematics offers numerous opportunities
to become familiar with such tools and to understand their methodological value. The path, when
this proves appropriate, will encourage the use of these tools, also with a view to their use for
processing data in other scientific disciplines. The use of the tools information technology is an
important resource that will be introduced in a way that is critical manner, without creating the
illusion that it is an automatic means of problem solving and without compromising the necessary
acquisition of mental calculation skills.
The wide range of content that will be addressed by the student will require the teacher to be
aware of the need for a good use of the time available. While maintaining the importance of the
acquisition of techniques, dispersion in repetitive technicalities or sterile case studies that do not
contribute in a meaningful way to understanding the problems. The in-depth of technical aspects,
although greater in the scientific high school than in other high schools, will never lose sight of
the goal of understanding in depth of the conceptual aspects of the discipline.
The main indication is: few fundamental concepts and methods, acquired in depth.”2
It seems to open the doors to what is needed, that is, imposing an overture to the use of machines
in teaching and, thus, to Large Language Models too. However, this is not the case in many public
Italian schools. Nevertheless, the option is open.
4. The Game Changer: Large Language Models
As discussed in the previous sections, machines have conquered many cognitive tasks in the past
and the Italian school system is “legally” open to use these machines during teaching. However,
in these days, teachers are trying to understand how to incorporate the new game changer that
is represented by the accessible part of these large language models (LLMs) [2]–[5], that is,
ChatGPT3, BARD4, and others.
Indeed, Instruction-following Language Models (IFLMs) seem to be able to cover many of the
repetitive tasks. As partially demonstrated in [6], [7] for Language Models, these LLMs may be
particularly good in remembering and applying patterns for solving tasks. If a pattern is
constantly repeated in training documents, it can be learnt and it can be used to solve exercises.
In Physics, Chemistry and so on, asking students to do exercises is the current tactic for helping
students to memorize rules in a deductive teaching manner. This result in a request of doing many
times the same “kind” of exercises. In this way, they are not focusing on the principles, but they
are focusing on the repetition. This is the major ability of Instruction-following Language Models
(IFLMs).
In the future job, these students will possibly use all the technology they can to solve faster
assigned tasks and IFLMs are one of the possible technologies.
IFLMs tease the Italian school system and, more in general, all the teaching activity with a
tantalizing question: “What is worth to be taught to the new generations?”. In this sense, IF-LLMs
are real game changers. The question on how to use these systems to improve teaching is not
completely the right point to address. Indeed, if the perspective does not change, teachers may
end up in being replaced by donating their knowledge to these greedy IFLMs [8]. Moreover, in a
dystopian future, these students may not become active members of the society because they can
do what machines can do.
2 In Attachment (Allegato) F of DECRETO 7 ottobre 2010, n. 211, Ministero Istruzione, Univeristà e Ricerca,
Translated with www.DeepL.com/Translator (free version)
3 http://chat.openai.com/
4 http://bard.google.com
5. Discussion and Conclusions
Conditions in working environments and in study environments should be similar in order to
prepare students to future working life. For this reason, intelligent machines should be used in
classrooms as these are used in offices.
In this paper, we analyzed the relationship between the Italian school system and the
advancement of machines doing cognitive tasks. In principle, the Italian school system is open to
the use of these cognitive machines during student training and, thus, this is good news. However,
it is completely unprepared to the challenge imposed by Instruction Following Language Models
such as ChatGPT.
We believe that the teaser of these models can help the Italian school system to renew from
the basis if students keep “cheating” as we all normally do during our working day.
References
[1] OpenAI, “GPT-4 Technical Report.” arXiv, Mar. 27, 2023. Accessed: Sep. 22, 2023. [Online].
Available: http://arxiv.org/abs/2303.08774
[2] H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models.” arXiv, Feb. 27,
2023. Accessed: Mar. 28, 2023. [Online]. Available: http://arxiv.org/abs/2302.13971
[3] B. Workshop et al., “BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.”
arXiv, Mar. 13, 2023. Accessed: May 03, 2023. [Online]. Available:
http://arxiv.org/abs/2211.05100
[4] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional
Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the
North, Minneapolis, Minnesota: Association for Computational Linguistics, 2019, pp. 4171–
4186. doi: 10.18653/v1/N19-1423.
[5] Z. Zhang, X. Han, Z. Liu, X. Jiang, M. Sun, and Q. Liu, “ERNIE: Enhanced Language
Representation with Informative Entities.” arXiv, Jun. 04, 2019. Accessed: Sep. 22, 2023.
[Online]. Available: http://arxiv.org/abs/1905.07129
[6] L. Ranaldi, E. S. Ruzzetti, and F. M. Zanzotto, “PreCog: Exploring the relation between
memorization and performance in pre-trained language models,” in Proceedings of the
international conference on recent advances in natural language processing (RANLP 2023),
Sep. 2023.
[7] L. Ranaldi et al., “The dark side of the language: Pre-trained transformers in the DarkNet,” in
Proceedings of the international conference on recent advances in natural language processing
(RANLP 2023), Sep. 2023.
[8] F. M. Zanzotto, “Viewpoint: Human-in-the-loop artificial intelligence,” Journal of Artificial
Intelligence Research, 2019, doi: 10.1613/jair.1.11345.