Toward the use of Generative AI to develop Computational Thinking by supporting Problem Decomposition Davide Ponzini1,2,* , Giovanni Adorni1 , Giorgio Delzanno1 and Giovanna Guerrini1 1 University of Genoa, Dipartimento di Informatica, Bioingegneria, Robotica e Ingeneria dei Sistemi, via Dodecaneso 35, Genoa, 16145, Italy 2 University of Genoa, Dipartimento di Lingue e Culture Moderne, piazza Santa Sabina 2, Genoa, 16124, Italy Abstract We present a possible applications of generative AI to support a Computational Thinking approach to learn programming principles with a particular focus on problem decomposition. Our approach is based on a visual tool that guides students in decomposing a problem in smaller task, prompting ChatGPT on demand via predefined queries designed via a preliminary prompt engineering experimental phase. The tool also provides the possibility of prompting ChatGPT to generate code in a bottom-up manner, reusing functions generated in previous steps. We illustrate here the main ideas with the help of a case-study. Keywords Generative AI, ChatGPT, Computational Thinking, Problem Decomposition 1. Introduction In our research, we are interested in possible appli- cations of Generative AI to develop a Computational The role of Generative AI in computing education is one Thinking approach in learning to program. of the mostly debated issues in the last year, as evidenced Computational Thinking [8] can be defined as by works such as [1]. Although the advantages and disad- vantages of using tools such as ChatGPT1 to support the "The thought processes involved in formu- teaching of computer-related subjects are not fully clear lating problems and their solutions so that yet, there is a growing consensus that the technological the solutions are represented in a form progress of Generative AI will require an adaptation of that can be effectively carried out by an the educational methods currently employed. information-processing agent." [9] In the context of introductory and advanced program- Wing emphasized that Computational Thinking is a fun- ming courses, tools such as ChatGPT, which are already damental component of Computer Science, where its integrated in the most common software development abstractions can be executed by computational agents. tools (take as an example GitHub’s Copilot) [2] are fre- The shift from solving problems to effectively solving quently exploited. These tools are designed in order to problems and making something perform computations help the users to best formulate the questions submitted for us distinguishes Computational Thinking from other to the prompt. disciplines. According to the current literature on computing edu- One of the characteristics that defines Computational cation, such as [1, 3, 4, 5, 6, 7], the most common use of Thinking is problem decomposition, i.e., the capability Generative AI is to generate code or to explain specific of solving a complex problem by breaking it down into features of existing or generated code. The use of Genera- progressively smaller tasks. tive AI to generate solutions to assignments and exercises In this paper we focus our attention on the possible is indeed a critical issue to be taken into consideration to use of Generative AI to develop Computational Thinking avoid plagiarism and negative effects on student learning skills by supporting the process of Problem Decomposi- outcomes. tion [10] with the help of visual tools. In particular, we examine the potential of ChatGPT to Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- assist students in analyzing and decomposing problems nized by CINI, May 29-30, 2024, Naples, Italy * Corresponding author. into smaller subtasks, employing a top-down approach. $ davide.ponzini@edu.unige.it (D. Ponzini); This is followed by the generation of code for a possi- giovanni.adorni@unige.it (G. Adorni); giorgio.delzanno@unige.it ble implementation, using a bottom-up approach and (G. Delzanno); giovanna.guerrini@unige.it (G. Guerrini) potentially reusing the code generated in previous steps.  0009-0006-4282-9652 (D. Ponzini); 0000-0003-3933-4377 Our approach is based a visual tool that presents the cur- (G. Adorni); 0000-0001-7030-1050 (G. Delzanno); 0000-0001-9125-9867 (G. Guerrini) rent decomposition in form of a labeled tree, in which © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License nodes contain names and descriptions of each subtask. Attribution 4.0 International (CC BY 4.0). 1 https://chat.openai.com/ Starting from the root node with the description of the CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings problem to solve, the user chooses which nodes to fur- of students may prefer not to use the tool, believing it ther decompose. The decomposition process is designed will lead to reduced learning [11]. to be interactive and fully guided by the user until the Among the CS students who employ the tool, the most solution is decomposed in a collection of subtasks that common usage is for generating code, followed by debug- the user is capable to implement. The decomposition of a ging. Explaining difficult concepts falls in third place [12]. given task, i.e., a given node of the tree, can be provided Students who use ChatGPT when facing programming manually by expanding the corresponding subtree with assignments tend to show higher efficacy and computa- a set of labeled nodes inserted by the user. In addition it tional thinking skills [13]. can also be generated via a predefined query to ChatGPT. Currently, ChatGPT is mainly employed to automati- The query formulation is based on a preliminary prompt cally provide students with feedback when programming engineering experimental design phase that seems to a task. Feedback can be provided in many forms, such provide good results in the most common coding tasks as coding hints for the next instruction [4] and code or taken from the literature on introductory programming error explanations [14]. courses. In our approach, the prompt engineering task is When confronted with more complex problems, Chat- entirely hidden to the user and ChatGPT is considered as GPT is less adept at providing reliable solutions. As a an oracle to support students during the decomposition result, the user is often required to manually break down process. the problem into smaller components and subsequently If needed, in a second phase, users can prompt Chat- assemble the solution [15]. GPT to generate the code of the solution in a bottom up manner. The predefined query generated by our tool ensures that functions generated for a given subtask can 3. Methods be reused for generating the code of the current task. To evaluate the support offered by ChatGPT to decom- The implementation of the tool allowed us to evaluate pose a high-level task into smaller, easier-to-solve sub- how well ChatGPT is able to decompose a high-level task tasks, we designed a set of prompts for ChatGPT, as well into smaller, easier-to-solve subtasks, and to implement as a custom visualization tool to display the results and decomposed tasks using a bottom-up approach. allow the user to interact with the tool. In the paper we present the methodology, the proposed tasks, and the preliminary results with the help of an example. 3.1. ChatGPT Prompts The paper is structured as follows: In Section 2, we out- Decomposing a task into subtasks To decompose a line the background of our study, focusing on the effects broad task into more refined subtasks we experimented ChatGPT has on education and computational thinking. with several prompting styles. Section 3 describes the proposed methodology and exper- Our initial approach was to include the current task iment setup. Section 4 reports, focusing on a case study, decomposition in the prompt, so that ChatGPT would be the results of our first experiments and discusses their aware of which tasks had already been decomposed and limitations. Finally, in Section 5 we address conclusions how they were decomposed at each iteration. However, and future research directions. this approach proved ineffective as ChatGPT struggled to understand the decomposition. Various notations were 2. Background attempted, including JSON and ad hoc syntax, similar to regular tree expressions, to describe the current struc- ChatGPT has shown remarkable capabilities in solving ture of the decomposition, but none was found to have a programming tasks across a range of different program- positive impact. Ultimately, two distinct prompts were ming languages, particularly when the instructions are utilized, and ChatGPT’s conversational memory was re- presented in a clear and unambiguous manner [7, 6]. lied upon to maintain the current decomposition state. However, it is less effective when confronted with more The initial prompt is used when a user first wishes to complex requests, particularly when the questions are decompose a problem. It includes a user-provided prob- not structured in an optimal manner or when the infor- lem description and detailed instructions for ChatGPT on mation provided in the prompt is limited [6]. how to accurately decompose it. The main requirements Even though the tool shows great potential in the ed- we identified are: ucational domain, it is used by a limited number of stu- • Specifying that the task needs to be decomposed dents for educational purposes [11]. One of the reasons into a small number of subtasks. This require- is that education regarding LLMs usage is very limited ment is crucial as ChatGPT has a tendency to and students often don’t know how to properly use these tools to support learning [11, 5]. Furthermore, a number immediately decompose the problem into approx- given task and, if available, review its properties and im- imately ten steps, with emphasis on the process plementation. Alternatively, ChatGPT can be utilized by rather than on the reasoning behind the task. the tool to automatically decompose or implement any task. Task decomposition is only performed upon user • Requiring the fields “name” and “description” to request, as each user’s knowledge base varies and some be similar to the ones initially provided. This is tasks may be clear to some users but not to others. important for keeping stylistic consistency be- Tasks can also be marked as “solved”, indicating that tween all tasks. the user has fully understood the task and no further • Returning the results in JSON format. This is im- decomposition is needed. For the sake of usability, solved portant for using the output in our visualization tasks are presented in a distinct color, allowing for a tool. We also noticed that this requirement made clear representation of the current understanding of the the title and description more precise and each problem at all times. subtask being assigned a unique name. Tasks can also be edited, created, or deleted manually. However, ChatGPT does not currently reflect these ac- • Requiring that subtasks of a given task did not tions. Automatic decomposition or implementation of contain any element of other tasks. This is needed these tasks is not currently available. to avoid task mix-ups. • Ensuring there are no missing steps in the decom- 4. Case Study position, i.e., solving all the subtasks is equivalent to solving the original problem. The tool was employed to decompose and implement a small number of tasks. This section presents the results We also included an instruction indicating not to decom- for the task “Write a Python program to find the most pose a task in case no reasonable decomposition can be trending videos, given a CSV file containing each visual- made (for example for basic tasks). However, this proved ization”. The task has been intentionally formulated as ineffective, as ChatGPT always ended up decomposing a broad request, to test if the program would be able to even the simplest tasks. correctly identify its subtasks. A simpler prompt has been designed for decomposing subtasks. The prompt references the task by name and requests that the same process be repeated for that task. 4.1. Task decomposition It is important to note that each task decomposed using Figure 1 shows the decomposition of the task. ChatGPT this prompt has been originally generated by ChatGPT. decomposed the main task in the following subtasks, which is similar to how we would have manually decom- Implementing a task Once the user is satisfied with posed it. the current decomposition, they can start implement- ing the subtasks following a bottom-up approach. The • Read CSV file: Write a Python function to read prompt requires ChatGPT to implement the given task in the CSV file containing video views data. a specific programming language, which can be selected • Parse CSV file: Write a Python function to parse by the user. the data from the CSV file and extract relevant We identified two requirements, which are: information such as video IDs and view counts. • Using, whenever possible, functions already gen- • Calculate trending score: Write a Python func- erated by ChatGPT for other tasks. This require- tion to calculate a trending score for each video ment is crucial, since it can highlight the interac- based on its view count and possibly other factors tion between a task and its subtasks. such as upload date. • Not writing the implementation of functions • Sort videos by trending score: Write a Python which have already been implemented, to avoid function to sort the videos based on their calculated redundant code. trending scores in descending order. • Retrieve top trending videos: Write a Python 3.2. Visualization tool function to retrieve the top N videos with the highest The tool we have created enables users to easily visual- trending scores, where N is a parameter. ize tasks and explore their decomposition into subtasks. Users have the option to display or hide subtasks for a We chose not to decompose basic tasks, such as read- ing, parsing, sorting, or displaying data, as they can be Figure 1: Decomposition for the problem “Write a Python program to find the most trending videos, given a CSV file containing each visualization”. easily accomplished using existing functions in Python. 4.2. Task implementation Instead, we decided to further decompose the task “Calcu- After obtaining the decomposition shown in Figure 1, late trending score” because its implementation was not we instructed ChatGPT to implement the tasks using a easily understandable. ChatGPT decomposed the task bottom-up approach. This involved starting from the into: smallest subtasks and subsequently progressing towards • Calculate view count factor: Write a Python the top-level task. function to calculate a factor based on the view For each subtask, ChatGPT created a function which count of a video, possibly using logarithmic scaling solved the given problem. Each implementation also to give more weight to videos with higher view contained a docstring describing its purpose, parameters counts., and returned value, as well as a usage example, as shown in Figure 2. • Calculate time decay factor: Write a Python For tasks that were previously decomposed, ChatGPT function to calculate a factor to represent the time was able to accurately recall and utilize the functions it decay of a video’s popularity, considering how re- had previously implemented, as shown in Figure 2b. cently it was uploaded. • Combine factors into trending score: Write a 4.3. Limitations Python function to combine the factors calculated The main limitation of this study is the relatively small in the previous steps into an overall trending score number of problems that were decomposed and imple- for each video. mented using our system. It is possible that some tasks which is consistent with how we would have simpli- may present unforeseen challenges that require further fied the task. Further decompositions of its subtasks, tuning of our prompts or different approaches. as shown in Figure 1, were also consistent with our ex- A further limitation of the study is the small number pectations. of users who have tested the system. By testing the system with a larger number of users, it would be possible to ascertain whether certain tasks present issues when decomposed. (a) Implementation for task “Normalize view counts”. (b) Implementation for task “Calculate view count factor”. The functions highlighted in yellow have been created when implementing its subtasks. Figure 2: Results of task implementation. Additionally, as ChatGPT is frequently updated, differ- References ent prompts may be required to obtain the same output. [1] P. Denny, J. Prather, B. A. Becker, J. Finnie-Ansley, A. Hellas, J. Leinonen, A. Luxton-Reilly, B. N. 5. Conclusions & Future Work Reeves, E. A. Santos, S. Sarsa, Computing educa- tion in the era of generative AI, Commun. ACM 67 The developed tool demonstrates potential for decompos- (2024) 56–67. URL: https://doi.org/10.1145/3624720. ing tasks into smaller subtasks and implementing them doi:10.1145/3624720. using a bottom-up approach. The preliminary study de- [2] C. Ebert, P. Louridas, Generative AI for software composed a limited number of problems, but the results practitioners, IEEE Software 40 (2023) 30–38. seem quite promising. ChatGPT correctly identified how [3] C. Zastudil, M. Rogalska, C. Kapp, J. Vaughn, to decompose the main problem into smaller subtasks S. MacNeil, Generative AI in computing educa- and provided a well-documented implementation of each tion: Perspectives of students and instructors, in: step. IEEE Frontiers in Education Conference, FIE 2023, Future directions for our work include: College Station, TX, USA, October 18-21, 2023, • Integrating the tool with ChatGPT’s APIs to IEEE, 2023, pp. 1–9. URL: https://doi.org/10.1109/ fully automate decomposition and implementa- FIE58773.2023.10343467. doi:10.1109/FIE58773. 2023.10343467. tion functionalities; [4] L. Roest, H. Keuning, J. Jeuring, Next-Step Hint • Enabling users to fully modify existing decompo- Generation for Introductory Programming Using sitions and implementations; Large Language Models, in: Proceedings of the 26th Australasian Computing Education Confer- • Testing our system with a larger number of prob- ence, ACE ’24, Association for Computing Ma- lems; chinery, New York, NY, USA, 2024, pp. 144–153. • Defining precise evaluation criteria for proper doi:10.1145/3636243.3636259. task decomposition; [5] B. Qureshi, Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergradu- • Measuring if the tool can improve computational ate Computer Science Curriculum: Opportunities thinking learning skills in users. and Challenges, in: 2023 9th International Confer- ence on e-Society, e-Learning and e-Technologies, • Using the tool to identify weaknesses and mis- 2023, pp. 7–13. doi:10.1145/3613944.3613946, conception of students when facing a sequence arXiv:2304.11214 [cs]. of tasks that require common solving techniques. [6] E. L. Ouh, B. K. S. Gan, K. Jin Shim, S. Wlodkowski, ChatGPT, Can You Generate Solutions for my Cod- ing Exercises? An Evaluation on its Effectiveness in an undergraduate Java Programming Course., in: Proceedings of the 2023 Conference on Inno- vation and Technology in Computer Science Edu- cation V. 1, ACM, Turku Finland, 2023, pp. 54–60. doi:10.1145/3587102.3588794. [7] C. Geng, Y. Zhang, B. Pientka, X. Si, Can Chat- GPT Pass An Introductory Level Functional Lan- guage Programming Course?, 2023. doi:10.48550/ ARXIV.2305.02230. [8] J. M. Wing, Computational thinking, Com- mun. ACM 49 (2006) 33–35. URL: https://doi.org/ 10.1145/1118178.1118215. doi:10.1145/1118178. 1118215. [9] J. M. Wing, Computational thinking and thinking about computing, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366 (2008) 3717–3725. [10] P. J. Rich, G. Egan, J. Ellsworth, A frame- work for decomposition in computational think- ing, in: Proceedings of the 2019 ACM Confer- ence on Innovation and Technology in Computer Science Education, ITiCSE ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 416–421. URL: https://doi.org/10.1145/3304221. 3319793. doi:10.1145/3304221.3319793. [11] H. Singh, M.-H. Tayarani-Najaran, M. Yaqoob, Ex- ploring Computer Science Students’ Perception of ChatGPT in Higher Education: A Descriptive and Correlation Study, Education Sciences 13 (2023) 924. doi:10.3390/educsci13090924. [12] A. Ouaazki, K. Bergram, A. Holzer, Leverag- ing ChatGPT to Enhance Computational Think- ing Learning Experiences, in: 2023 IEEE Interna- tional Conference on Teaching, Assessment and Learning for Engineering (TALE), 2023, pp. 1–7. doi:10.1109/TALE56641.2023.10398358. [13] R. Yilmaz, F. G. Karaoglan Yilmaz, The effect of gen- erative artificial intelligence (AI)-based tool use on students’ computational thinking skills, program- ming self-efficacy and motivation, Computers and Education: Artificial Intelligence 4 (2023) 100147. doi:10.1016/j.caeai.2023.100147. [14] K. Kuramitsu, Y. Obara, M. Sato, M. Obara, KOGI: A Seamless Integration of ChatGPT into Jupyter Envi- ronments for Programming Education, in: Proceed- ings of the 2023 ACM SIGPLAN International Sym- posium on SPLASH-E, ACM, Cascais Portugal, 2023, pp. 50–59. doi:10.1145/3622780.3623648. [15] A. Sane, M. Albuquerque, M. Gupta, J. Valadi, Chat- GPT Didn’t Take Me Very Far, Did It?, in: Proceed- ings of the ACM Conference on Global Comput- ing Education Vol 2, CompEd 2023, Association for Computing Machinery, New York, NY, USA, 2023, p. 204. doi:10.1145/3617650.3624947.