Computational Thinking (CT) is an area of great relevance today. Although its skills may be developed in various ways, one of the most common tools to learn it, train it and develop it, is through programming. From software engineering, we know that problems solved through programming may have not been solved in the most appropriate way. These symptoms are known as \bad smells". This article aims to analyze the presence of several bad smells in Scratch projects and how they relate to the development of CT skills. Therefore, we make use of a dataset of several hundreds of Scratch projects with the aim of creating a game on climate change. Our results show that bad smells can be found in all types of Scratch projects, independently of the development of CT skills they require. We discuss why the learning community should address bad smells appropriately, as they may hinder the development of abstraction, reuse and other relevant skills.
Bad (code) smells are symptoms that the problem to
be solved is not developed in the most appropriate way.
In other words, the program may run and may even
solve the problem, but it contains elements that make
it di cult to understand, to modify and to reuse [
Martin de nes code smells as follows [
For this reason, the main motivation of this research
is to analyze to what extent bad smells are present in
Scratch projects, a block-based language that is widely
used around the globe to develop CT skills. Our
research is similar to a previous one done on LEGO
MINDSTORMS EV3 and Microsoft's Kodu [
The remainder of this paper is structured as follows: In Section 2, we introduce and motivate the research goal and research questions that we address in this paper. A more detailed description about the de nition of bad smells is summarized in Section 3. Section 4 and Section 5 describe the data set used in the study, as well as the functionality and design of Dr. Scratch in more detail. Section 6 shows the results obtained after the analysis and in Section 7 a discussion is proposed based on it. Limitations and problems found are described in Section 7.1. Finally, Section 8 contains the main conclusions and future work that we envision.
The main objective of this paper is to analyze the presence of bad smells in a large set of Scratch projects.
For this, the research questions that we want to address are as follows:
RQ1. To what extent are bad habits present in Scratch projects?
In particular, we answer this question by o ering the percentage of projects that have at least one type of bad smell. This question allows to see how frequent projects show a bad smell, hinting to the relevance of the topic. We expect a signi cant number of projects to contain bad smells.
RQ2. Does the development of CT skills relate to the presence of bad smells?
We would like to nd out if the presence of bad smells correlates with the complexity of the projects. Our hypothesis is that projects that have higher degrees of CT development will have less bad smells, as these may hinder the development of CT skills.
RQ3. Do projects with more blocks have a higher number of bad smells?
More complex projects usually have more blocks. Thus having a single bad smell in a small, simple project may have less impact than in a project with hundreds of blocks. In the former case, the impact could be big, while in the latter it could be seen as an exception, with little impact.
To answer this question, for projects of the same level of CT development we calculate the ratio of the number of bad smells detected to the total number of blocks. We expect that this ratio decreases with an increase in the development of CT skills required to create the Scratch projects.
RQ4. Can we nd a relation among speci c bad smells?
As by now, we have considered all type of bad smells together. In this question, we dig into each of them separately. It may be possible that some bad smells appear more frequently in projects of lower complexity, while others appear in more complex projects.
RQ5. To which extent can bad smells be identi ed in each of the CT development phases?
Related to the previous question, we analyze how the di erent bad smell types appear in projects in the di erent stages of CT development. Therefore, we consider projects with a low complexity (basic), medium complexity (developing) and major complexity (pro ciency) and compute how often they contain a speci c type of bad smell.
We expect that several types of bad smells appear in the early phases (basic), while others are more prominent in more complex projects (pro ciency). We assume therefore that learners that achieve higher levels of complexity have overcome certain bad smells due to having developed certain CT skills, while other bad smells appear in those more complex projects. 3
Scratch is a visual programming language formed by
di erent blocks, designed for children and beginner
programmers, which contains di erent bad smells
related to the use of these blocks [
In our research, we have identi ed four di erent
types of bad smells that can be present in Scratch
projects: copy and pasted code (duplicate scripts) [
In order to analyze the presence of bad smells, as well
as their relationship with the level that users have in
CT development, a large set of projects is necessary.
The data set used in this article is the same which was
used for another, previous research [
All these snapshots were analyzed with Dr. Scratch,
of which 2,158 were erroneous in the analysis for
different reasons: the project was saved incorrectly, the
code contained special characters, etc. The nal set
of projects analyzed was 59,916 (further details can be
found in [
We have taken all snapshots of the Scratch projects
and have analyzed them with Dr. Scratch. Dr.
Scratch is a web-based tool that analyzes di erent
categories related to computational thinking based on the
blocks of the Scratch projects [
1https://drive.google.com/drive/u/0/folders/ 1tDI6nx2f6344xJAKeUeWBeTg0YzxE3bO
The outcome is a numeric punctuation based on seven categories of computational thinking: parallelism, logical thinking, ow control, user interactivity, data representation, abstraction and synchronization. For each of these abilities a project can obtain a punctuation from 0 to 3 points, according to the di erent blocks used in the Scratch project. In this way, the nal punctuation can be from 0 to 21 total points.
Based on that, there are three di erent pro les of leaner: between 0 and 7 points, Basic, between 8 and 15, Developing and between 16 and 21, Pro ciency.
Once the project is analyzed, the application will show di erent dashboards with these results, as it is possible to see in the Figure 2.
In addition to the former, Dr. Scratch identi es the four types of bad smells that we study in this work. 6
In this Section we describe the results obtained from addressing our research questions using the previously
We expected a high share of projects having bad smells, but this result is a surprise for us, as the presence of bad smells is not only more frequent than expected, but almost general. 6.2
Does the development of CT skills relate to a minor presence of bad smells? they share a ratio of around 0.5 up to 21 points, where the ratio is over 1).
We have analyzed in more detail those projects that have and do not bad smells. In particular, we want to see how the complexity of the projects is related to having a bad smell. We use the mastery required to create a project, as measured by Dr. Scratch, as a proxy for the complexity of the project.
In Figure 3 we can observe the distribution of each set of projects. We can observe that more than 50% of those projects with not bad smells have a total mastery of 0. In other words, these are skeleton projects without any content. The amount of projects with content and without any bad smell is therefore even lower than calculated in the previous research question. In addition, we see that projects with no bad smells are in the lower part of the complexity ladder.
In summary, bad smells in Scratch can be found in almost all projects. Only a small set of projects with low complexity do not have them. 6.3
Do projects with more blocks have a higher number of bad smells? One could argue that projects with more complexity (those with higher values of CT score in Dr. Scratch) usually have more blocks, and that the impact of a code smell there is lower than in less complex projects. In other words, even if more complex projects have bad smells, their presence is mitigated by the fact that these are large projects. This would imply that achieving high values of CT development means to have less bad smells.
Figure 4 visually shows the number of blocks for all projects for a given CT score (blue line) and the number of bad smells in those projects. Both curves have been normalized to their maximum values. We can observe how the two curves run almost in parallel (up to 8 points they share the same ratio, and then
nd a relation among speci c bad From Figure 5 (again, this graph is normalized), we can observe that the four di erent types of bad smells that we have studied have a similar distribution below the pro ciency level. The presence of bad smells has a similar behavior for projects up to 17 points of mastery. Then, when the mastery is above 17 points, the behavior of each of them is di erent: While duplicated scripts and bad attribute initialization continue to slightly grow, the number of dead code blocks grows more abrupt way. However, and the number of default names decreases considerably. This last trend may be because in projects with a high number of blocks and objects it is more di cult to program with the default names (Sprite 1, Sprite 2, etc) instead of personalizing them. 6.5
To which extent can bad smells be identi ed in each of the CT development phases? As already seen wit RQ3, the number of bad smells is higher when the total mastery increases. In Figure 6, we have represented the percentage of projects that have at least a speci c type of bad smell for each level of CT pro ciency. The percentage of projects from users with the basic level that have bad smells is much smaller than the percentage of projects that require a pro ciency level. This result indicates that all bad smells have an incremental evolution with the increase of CT e ciency. So, it seems that bad smells appear in early phases and instead of disappearing with the We have observed that bad smells are very common in Scratch projects. The results indicate as well that bad smells do not limit the development of a project, because it is possible to create projects at the pro ciency level even with a large amount of bad smells.
We argue that researchers, educators and learners should devote more e ort in avoiding the presence of bad smells in learning projects. The very nature of bad smells makes them di cult to identify. They are not errors and a program could run perfectly having many of them. However, their e ects are very well known in Software Engineering research. These e ects are in the long term, when maintainability of a software system is considered. In such a situation, the software has to be understood and changed, and in that situation is where these bad smells become more prominent. The large presence of bad smells indicates that understanding and changing code is not among the priorities of the projects under study, although \good code" has always been considered as that one that is easy to understand and to change. That is why we think that with the current presence of bad smells learners do not achieve their full potential of CT development skills. In our opinion, further research and tools should be envisioned and created to address this issue. 7.1
Limitations As any research, our work comes with a number of limitations that can be seen as threats to its validity.
The rst one is related to our methodology, and in
particular to the limited set of bad smells that we can
identify. We are sure that many other types of bad
smells could be thought of in Scratch. On the other
hand, we use as complexity metrics the CT scores
provided by Dr. Scratch. While there have been some
research that has studied how Dr. Scratch correlates
with other complexity metrics [
We have studied projects from a speci c environment, which may not be representative for all Scratch projects, so the generalization of results is a threat to validity. However, it should be noted that this is a case study, with the aim to raise attention on this matter. We should analyze a wider data set which includes di erent sectors of programming with Scratch, such as stories, music, animations, among others. We do not know if the behavior of bad smells could be di erent in other areas of programming. 8
Bad smells are common in Scratch projects, from basic to pro cient. As the complexity increases, bad smells do not disappear, so learners can create projects that demand a high development of CT skills having bad smells in them.
We ask for further research on this topic.
This work has been co-funded by the Madrid Regional Government, through the project e-MadridCM (P2018/TCS-4307). The e-Madrid-CM project is also co- nanced by the Structural Funds (FSE and FEDER).