=Paper=
{{Paper
|id=Vol-3762/583
|storemode=property
|title=Toward the use of Generative AI to develop Computational Thinking by supporting Problem Decomposition
|pdfUrl=https://ceur-ws.org/Vol-3762/583.pdf
|volume=Vol-3762
|authors=Davide Ponzini,Giovanni Adorni,Giorgio Delzanno,Giovanna Guerrini
|dblpUrl=https://dblp.org/rec/conf/ital-ia/PonziniADG24
}}
==Toward the use of Generative AI to develop Computational Thinking by supporting Problem Decomposition==
https://ceur-ws.org/Vol-3762/583.pdf
Toward the use of Generative AI to develop Computational
Thinking by supporting Problem Decomposition
Davide Ponzini1,2,* , Giovanni Adorni1 , Giorgio Delzanno1 and Giovanna Guerrini1
1
University of Genoa, Dipartimento di Informatica, Bioingegneria, Robotica e Ingeneria dei Sistemi, via Dodecaneso 35, Genoa, 16145, Italy
2
University of Genoa, Dipartimento di Lingue e Culture Moderne, piazza Santa Sabina 2, Genoa, 16124, Italy
Abstract
We present a possible applications of generative AI to support a Computational Thinking approach to learn programming
principles with a particular focus on problem decomposition. Our approach is based on a visual tool that guides students in
decomposing a problem in smaller task, prompting ChatGPT on demand via predefined queries designed via a preliminary
prompt engineering experimental phase. The tool also provides the possibility of prompting ChatGPT to generate code in
a bottom-up manner, reusing functions generated in previous steps. We illustrate here the main ideas with the help of a
case-study.
Keywords
Generative AI, ChatGPT, Computational Thinking, Problem Decomposition
1. Introduction In our research, we are interested in possible appli-
cations of Generative AI to develop a Computational
The role of Generative AI in computing education is one Thinking approach in learning to program.
of the mostly debated issues in the last year, as evidenced Computational Thinking [8] can be defined as
by works such as [1]. Although the advantages and disad-
vantages of using tools such as ChatGPT1 to support the "The thought processes involved in formu-
teaching of computer-related subjects are not fully clear lating problems and their solutions so that
yet, there is a growing consensus that the technological the solutions are represented in a form
progress of Generative AI will require an adaptation of that can be effectively carried out by an
the educational methods currently employed. information-processing agent." [9]
In the context of introductory and advanced program-
Wing emphasized that Computational Thinking is a fun-
ming courses, tools such as ChatGPT, which are already
damental component of Computer Science, where its
integrated in the most common software development
abstractions can be executed by computational agents.
tools (take as an example GitHub’s Copilot) [2] are fre-
The shift from solving problems to effectively solving
quently exploited. These tools are designed in order to
problems and making something perform computations
help the users to best formulate the questions submitted
for us distinguishes Computational Thinking from other
to the prompt.
disciplines.
According to the current literature on computing edu-
One of the characteristics that defines Computational
cation, such as [1, 3, 4, 5, 6, 7], the most common use of
Thinking is problem decomposition, i.e., the capability
Generative AI is to generate code or to explain specific
of solving a complex problem by breaking it down into
features of existing or generated code. The use of Genera-
progressively smaller tasks.
tive AI to generate solutions to assignments and exercises
In this paper we focus our attention on the possible
is indeed a critical issue to be taken into consideration to
use of Generative AI to develop Computational Thinking
avoid plagiarism and negative effects on student learning
skills by supporting the process of Problem Decomposi-
outcomes.
tion [10] with the help of visual tools.
In particular, we examine the potential of ChatGPT to
Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- assist students in analyzing and decomposing problems
nized by CINI, May 29-30, 2024, Naples, Italy
*
Corresponding author. into smaller subtasks, employing a top-down approach.
$ davide.ponzini@edu.unige.it (D. Ponzini); This is followed by the generation of code for a possi-
giovanni.adorni@unige.it (G. Adorni); giorgio.delzanno@unige.it ble implementation, using a bottom-up approach and
(G. Delzanno); giovanna.guerrini@unige.it (G. Guerrini) potentially reusing the code generated in previous steps.
0009-0006-4282-9652 (D. Ponzini); 0000-0003-3933-4377 Our approach is based a visual tool that presents the cur-
(G. Adorni); 0000-0001-7030-1050 (G. Delzanno);
0000-0001-9125-9867 (G. Guerrini)
rent decomposition in form of a labeled tree, in which
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License nodes contain names and descriptions of each subtask.
Attribution 4.0 International (CC BY 4.0).
1
https://chat.openai.com/ Starting from the root node with the description of the
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
problem to solve, the user chooses which nodes to fur- of students may prefer not to use the tool, believing it
ther decompose. The decomposition process is designed will lead to reduced learning [11].
to be interactive and fully guided by the user until the Among the CS students who employ the tool, the most
solution is decomposed in a collection of subtasks that common usage is for generating code, followed by debug-
the user is capable to implement. The decomposition of a ging. Explaining difficult concepts falls in third place [12].
given task, i.e., a given node of the tree, can be provided Students who use ChatGPT when facing programming
manually by expanding the corresponding subtree with assignments tend to show higher efficacy and computa-
a set of labeled nodes inserted by the user. In addition it tional thinking skills [13].
can also be generated via a predefined query to ChatGPT. Currently, ChatGPT is mainly employed to automati-
The query formulation is based on a preliminary prompt cally provide students with feedback when programming
engineering experimental design phase that seems to a task. Feedback can be provided in many forms, such
provide good results in the most common coding tasks as coding hints for the next instruction [4] and code or
taken from the literature on introductory programming error explanations [14].
courses. In our approach, the prompt engineering task is When confronted with more complex problems, Chat-
entirely hidden to the user and ChatGPT is considered as GPT is less adept at providing reliable solutions. As a
an oracle to support students during the decomposition result, the user is often required to manually break down
process. the problem into smaller components and subsequently
If needed, in a second phase, users can prompt Chat- assemble the solution [15].
GPT to generate the code of the solution in a bottom
up manner. The predefined query generated by our tool
ensures that functions generated for a given subtask can 3. Methods
be reused for generating the code of the current task.
To evaluate the support offered by ChatGPT to decom-
The implementation of the tool allowed us to evaluate
pose a high-level task into smaller, easier-to-solve sub-
how well ChatGPT is able to decompose a high-level task
tasks, we designed a set of prompts for ChatGPT, as well
into smaller, easier-to-solve subtasks, and to implement
as a custom visualization tool to display the results and
decomposed tasks using a bottom-up approach.
allow the user to interact with the tool.
In the paper we present the methodology, the proposed
tasks, and the preliminary results with the help of an
example. 3.1. ChatGPT Prompts
The paper is structured as follows: In Section 2, we out- Decomposing a task into subtasks To decompose a
line the background of our study, focusing on the effects broad task into more refined subtasks we experimented
ChatGPT has on education and computational thinking. with several prompting styles.
Section 3 describes the proposed methodology and exper- Our initial approach was to include the current task
iment setup. Section 4 reports, focusing on a case study, decomposition in the prompt, so that ChatGPT would be
the results of our first experiments and discusses their aware of which tasks had already been decomposed and
limitations. Finally, in Section 5 we address conclusions how they were decomposed at each iteration. However,
and future research directions. this approach proved ineffective as ChatGPT struggled to
understand the decomposition. Various notations were
2. Background attempted, including JSON and ad hoc syntax, similar to
regular tree expressions, to describe the current struc-
ChatGPT has shown remarkable capabilities in solving ture of the decomposition, but none was found to have a
programming tasks across a range of different program- positive impact. Ultimately, two distinct prompts were
ming languages, particularly when the instructions are utilized, and ChatGPT’s conversational memory was re-
presented in a clear and unambiguous manner [7, 6]. lied upon to maintain the current decomposition state.
However, it is less effective when confronted with more The initial prompt is used when a user first wishes to
complex requests, particularly when the questions are decompose a problem. It includes a user-provided prob-
not structured in an optimal manner or when the infor- lem description and detailed instructions for ChatGPT on
mation provided in the prompt is limited [6]. how to accurately decompose it. The main requirements
Even though the tool shows great potential in the ed- we identified are:
ucational domain, it is used by a limited number of stu-
• Specifying that the task needs to be decomposed
dents for educational purposes [11]. One of the reasons
into a small number of subtasks. This require-
is that education regarding LLMs usage is very limited
ment is crucial as ChatGPT has a tendency to
and students often don’t know how to properly use these
tools to support learning [11, 5]. Furthermore, a number
immediately decompose the problem into approx- given task and, if available, review its properties and im-
imately ten steps, with emphasis on the process plementation. Alternatively, ChatGPT can be utilized by
rather than on the reasoning behind the task. the tool to automatically decompose or implement any
task. Task decomposition is only performed upon user
• Requiring the fields “name” and “description” to request, as each user’s knowledge base varies and some
be similar to the ones initially provided. This is tasks may be clear to some users but not to others.
important for keeping stylistic consistency be- Tasks can also be marked as “solved”, indicating that
tween all tasks. the user has fully understood the task and no further
• Returning the results in JSON format. This is im- decomposition is needed. For the sake of usability, solved
portant for using the output in our visualization tasks are presented in a distinct color, allowing for a
tool. We also noticed that this requirement made clear representation of the current understanding of the
the title and description more precise and each problem at all times.
subtask being assigned a unique name. Tasks can also be edited, created, or deleted manually.
However, ChatGPT does not currently reflect these ac-
• Requiring that subtasks of a given task did not tions. Automatic decomposition or implementation of
contain any element of other tasks. This is needed these tasks is not currently available.
to avoid task mix-ups.
• Ensuring there are no missing steps in the decom- 4. Case Study
position, i.e., solving all the subtasks is equivalent
to solving the original problem. The tool was employed to decompose and implement a
small number of tasks. This section presents the results
We also included an instruction indicating not to decom- for the task “Write a Python program to find the most
pose a task in case no reasonable decomposition can be trending videos, given a CSV file containing each visual-
made (for example for basic tasks). However, this proved ization”. The task has been intentionally formulated as
ineffective, as ChatGPT always ended up decomposing a broad request, to test if the program would be able to
even the simplest tasks. correctly identify its subtasks.
A simpler prompt has been designed for decomposing
subtasks. The prompt references the task by name and
requests that the same process be repeated for that task.
4.1. Task decomposition
It is important to note that each task decomposed using Figure 1 shows the decomposition of the task. ChatGPT
this prompt has been originally generated by ChatGPT. decomposed the main task in the following subtasks,
which is similar to how we would have manually decom-
Implementing a task Once the user is satisfied with posed it.
the current decomposition, they can start implement-
ing the subtasks following a bottom-up approach. The • Read CSV file: Write a Python function to read
prompt requires ChatGPT to implement the given task in the CSV file containing video views data.
a specific programming language, which can be selected • Parse CSV file: Write a Python function to parse
by the user. the data from the CSV file and extract relevant
We identified two requirements, which are: information such as video IDs and view counts.
• Using, whenever possible, functions already gen- • Calculate trending score: Write a Python func-
erated by ChatGPT for other tasks. This require- tion to calculate a trending score for each video
ment is crucial, since it can highlight the interac- based on its view count and possibly other factors
tion between a task and its subtasks. such as upload date.
• Not writing the implementation of functions • Sort videos by trending score: Write a Python
which have already been implemented, to avoid function to sort the videos based on their calculated
redundant code. trending scores in descending order.
• Retrieve top trending videos: Write a Python
3.2. Visualization tool
function to retrieve the top N videos with the highest
The tool we have created enables users to easily visual- trending scores, where N is a parameter.
ize tasks and explore their decomposition into subtasks.
Users have the option to display or hide subtasks for a We chose not to decompose basic tasks, such as read-
ing, parsing, sorting, or displaying data, as they can be
Figure 1: Decomposition for the problem “Write a Python program to find the most trending videos, given a CSV file containing
each visualization”.
easily accomplished using existing functions in Python. 4.2. Task implementation
Instead, we decided to further decompose the task “Calcu-
After obtaining the decomposition shown in Figure 1,
late trending score” because its implementation was not
we instructed ChatGPT to implement the tasks using a
easily understandable. ChatGPT decomposed the task
bottom-up approach. This involved starting from the
into:
smallest subtasks and subsequently progressing towards
• Calculate view count factor: Write a Python the top-level task.
function to calculate a factor based on the view For each subtask, ChatGPT created a function which
count of a video, possibly using logarithmic scaling solved the given problem. Each implementation also
to give more weight to videos with higher view contained a docstring describing its purpose, parameters
counts., and returned value, as well as a usage example, as shown
in Figure 2.
• Calculate time decay factor: Write a Python For tasks that were previously decomposed, ChatGPT
function to calculate a factor to represent the time was able to accurately recall and utilize the functions it
decay of a video’s popularity, considering how re- had previously implemented, as shown in Figure 2b.
cently it was uploaded.
• Combine factors into trending score: Write a 4.3. Limitations
Python function to combine the factors calculated
The main limitation of this study is the relatively small
in the previous steps into an overall trending score
number of problems that were decomposed and imple-
for each video.
mented using our system. It is possible that some tasks
which is consistent with how we would have simpli- may present unforeseen challenges that require further
fied the task. Further decompositions of its subtasks, tuning of our prompts or different approaches.
as shown in Figure 1, were also consistent with our ex- A further limitation of the study is the small number
pectations. of users who have tested the system. By testing the
system with a larger number of users, it would be possible
to ascertain whether certain tasks present issues when
decomposed.
(a) Implementation for task “Normalize view counts”. (b) Implementation for task “Calculate view count factor”. The
functions highlighted in yellow have been created when
implementing its subtasks.
Figure 2: Results of task implementation.
Additionally, as ChatGPT is frequently updated, differ- References
ent prompts may be required to obtain the same output.
[1] P. Denny, J. Prather, B. A. Becker, J. Finnie-Ansley,
A. Hellas, J. Leinonen, A. Luxton-Reilly, B. N.
5. Conclusions & Future Work Reeves, E. A. Santos, S. Sarsa, Computing educa-
tion in the era of generative AI, Commun. ACM 67
The developed tool demonstrates potential for decompos- (2024) 56–67. URL: https://doi.org/10.1145/3624720.
ing tasks into smaller subtasks and implementing them doi:10.1145/3624720.
using a bottom-up approach. The preliminary study de- [2] C. Ebert, P. Louridas, Generative AI for software
composed a limited number of problems, but the results practitioners, IEEE Software 40 (2023) 30–38.
seem quite promising. ChatGPT correctly identified how [3] C. Zastudil, M. Rogalska, C. Kapp, J. Vaughn,
to decompose the main problem into smaller subtasks S. MacNeil, Generative AI in computing educa-
and provided a well-documented implementation of each tion: Perspectives of students and instructors, in:
step. IEEE Frontiers in Education Conference, FIE 2023,
Future directions for our work include: College Station, TX, USA, October 18-21, 2023,
• Integrating the tool with ChatGPT’s APIs to IEEE, 2023, pp. 1–9. URL: https://doi.org/10.1109/
fully automate decomposition and implementa- FIE58773.2023.10343467. doi:10.1109/FIE58773.
2023.10343467.
tion functionalities;
[4] L. Roest, H. Keuning, J. Jeuring, Next-Step Hint
• Enabling users to fully modify existing decompo- Generation for Introductory Programming Using
sitions and implementations; Large Language Models, in: Proceedings of the
26th Australasian Computing Education Confer-
• Testing our system with a larger number of prob- ence, ACE ’24, Association for Computing Ma-
lems; chinery, New York, NY, USA, 2024, pp. 144–153.
• Defining precise evaluation criteria for proper doi:10.1145/3636243.3636259.
task decomposition; [5] B. Qureshi, Exploring the Use of ChatGPT as a
Tool for Learning and Assessment in Undergradu-
• Measuring if the tool can improve computational ate Computer Science Curriculum: Opportunities
thinking learning skills in users. and Challenges, in: 2023 9th International Confer-
ence on e-Society, e-Learning and e-Technologies,
• Using the tool to identify weaknesses and mis- 2023, pp. 7–13. doi:10.1145/3613944.3613946,
conception of students when facing a sequence arXiv:2304.11214 [cs].
of tasks that require common solving techniques. [6] E. L. Ouh, B. K. S. Gan, K. Jin Shim, S. Wlodkowski,
ChatGPT, Can You Generate Solutions for my Cod-
ing Exercises? An Evaluation on its Effectiveness
in an undergraduate Java Programming Course.,
in: Proceedings of the 2023 Conference on Inno-
vation and Technology in Computer Science Edu-
cation V. 1, ACM, Turku Finland, 2023, pp. 54–60.
doi:10.1145/3587102.3588794.
[7] C. Geng, Y. Zhang, B. Pientka, X. Si, Can Chat-
GPT Pass An Introductory Level Functional Lan-
guage Programming Course?, 2023. doi:10.48550/
ARXIV.2305.02230.
[8] J. M. Wing, Computational thinking, Com-
mun. ACM 49 (2006) 33–35. URL: https://doi.org/
10.1145/1118178.1118215. doi:10.1145/1118178.
1118215.
[9] J. M. Wing, Computational thinking and thinking
about computing, Philosophical Transactions of
the Royal Society A: Mathematical, Physical and
Engineering Sciences 366 (2008) 3717–3725.
[10] P. J. Rich, G. Egan, J. Ellsworth, A frame-
work for decomposition in computational think-
ing, in: Proceedings of the 2019 ACM Confer-
ence on Innovation and Technology in Computer
Science Education, ITiCSE ’19, Association for
Computing Machinery, New York, NY, USA, 2019,
p. 416–421. URL: https://doi.org/10.1145/3304221.
3319793. doi:10.1145/3304221.3319793.
[11] H. Singh, M.-H. Tayarani-Najaran, M. Yaqoob, Ex-
ploring Computer Science Students’ Perception of
ChatGPT in Higher Education: A Descriptive and
Correlation Study, Education Sciences 13 (2023)
924. doi:10.3390/educsci13090924.
[12] A. Ouaazki, K. Bergram, A. Holzer, Leverag-
ing ChatGPT to Enhance Computational Think-
ing Learning Experiences, in: 2023 IEEE Interna-
tional Conference on Teaching, Assessment and
Learning for Engineering (TALE), 2023, pp. 1–7.
doi:10.1109/TALE56641.2023.10398358.
[13] R. Yilmaz, F. G. Karaoglan Yilmaz, The effect of gen-
erative artificial intelligence (AI)-based tool use on
students’ computational thinking skills, program-
ming self-efficacy and motivation, Computers and
Education: Artificial Intelligence 4 (2023) 100147.
doi:10.1016/j.caeai.2023.100147.
[14] K. Kuramitsu, Y. Obara, M. Sato, M. Obara, KOGI: A
Seamless Integration of ChatGPT into Jupyter Envi-
ronments for Programming Education, in: Proceed-
ings of the 2023 ACM SIGPLAN International Sym-
posium on SPLASH-E, ACM, Cascais Portugal, 2023,
pp. 50–59. doi:10.1145/3622780.3623648.
[15] A. Sane, M. Albuquerque, M. Gupta, J. Valadi, Chat-
GPT Didn’t Take Me Very Far, Did It?, in: Proceed-
ings of the ACM Conference on Global Comput-
ing Education Vol 2, CompEd 2023, Association for
Computing Machinery, New York, NY, USA, 2023,
p. 204. doi:10.1145/3617650.3624947.