This paper starts from the definition of Computational Thinking (CT) and Computational Thinking Skills for EUD and extends it to the problem of outlining a measurement construct to assess their level of mastery in individuals. Stemming from the literature re Computational Thinking and its assessment, the idea of this contribution is to rescue this discourse from that of the K-12 Computer Science (CS) education domain, where it mainly stands, and bring it into the adult (possibly working) life, where it crosses the concepts of Computational Literacy (mastery of background knowledge) and Computational Fluency (mastery of practical application of background knowledge). These concepts are then interleaving with those of subjective abilities and problems / solutions dificulties, forming an intricate network of elements to be systematized into an unicum for measurement reasons. This measurement property should be able to assess the level of subjective chances to overcome an objectively dificult problem, and having many ways to solve it. The purpose of this work is to start systematizing those concepts and their reciprocal relationships that could possibly converge into a measurement construct, and to clarify which notions should remain necessarily open and unsytematized or are simply needing further investigation.
A recent study in the EUD domain posed the challenging problem of defining Computational Thinking Skills for EUD as part of a wider framework where the construct of EUDability was defined as “the degree of concreteness, modularity, structuredness, reusability, and testability fostered by a EUD environment designed for specified end-user developers, with a specified goal to be pursued in a specified context.”[ 1, p.10]. The derived definition of Computational Thinking (CT from now on) is the following: “the acquired capability of adopting a three-stage mental process, i.e., defining the problem, solving the problem, analyzing the solution, by knowing how to apply five basic skills, i.e., abstraction, decomposition, algorithm design, generalization, and evaluation, up to an individual level of mastery.”[1, p.6]. These notions were acquired in the EUD domain as part of the problem of rescuing the concepts of Computational Literacy, Computational Fluency and the like from that of K-12 education or education in general. Most of the time, attempts were made to either define these constructs or assess them through diferent https://www.uninsubria.it/hpp/angela.locoro#0 (A. Locoro) assessment tools. The purpose of this paper is to take some of the lessons learnt from the CS education domain regarding the study of CT and of its assessment, and to transpose it to the EUD domain, where things stand quite diferently in terms of what to learn and what to measure. In this domain, whenever CT is applied to problem solving, disparate solutions exist (i.e., EUD techniques). An exact mapping of some well known EUD paradigms and the related CT Skills required to adopt such EUD solutions are again part of the above study [ibidem].
This paper is an attempt to go beyond the identification of concepts such as CT and CT Skills, their definitions, and their mapping with EUD elements. Stemming from the literature about defining CT in K-12 education and CT assessment, we will see that the main limitations in this domain lays in that the main issue is still ontological. Put it in other words, the main concerns are still that of defining CT and its assessment. Elements of definitions include: what is included in CT; what are the abilities to be identified and assessed and what should be excluded; what tools exist to assess the acquired CT skills; how to define a standard definition and a (possibly) CT standard assessment tool.
Going beyond these aspects is crucial if we want to advance the problem of CT towards its measurement aspects. This problem has to do with the need of defining a (possibly) unique construct (or variable), with basic and advanced levels, able to account for what among the skills and problem steps devised in the main CT frameworks are the easiest, what the mid level ones, and what are the most dificult ones, in the continuum space of such variable. This approach is intended to resemble the one of standard measurement tools such as the PISA test1 for measuring reading literacy. PISA defines reading literacy as “understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, develop one’s knowledge and potential, and participate in society”. In this vein, we should be able in the end to define a CT measurement construct and the related CT measurement scale. To the best of my knowledge, whatever the domain of application, we are still far from this achievement.
That said, the idea of this paper is to begin to outline what are the current achievements and what are the further steps towards a CT measurement scale.
Starting from the most high-level definitions of CT in K-12 education field, we may have an
idea of the main pillars constituting the CT essence. For example, a very recent study in the
educational domain [
However, seen from another perspective, CT may be analyzed from the point of view of
its more generic forming concepts. For example, Brennan and Resnick [
One of the most promising approaches in the direction of building a measurement construct
is that of integrating / prioritizing CT dimensions, proposed by [
This perspective lays in the direction of going beyond a single element such as the
problemsolving one, but rather considering all the elements influencing people ability to solve problems.
For example: their scholarly background knowledge, which can be called their “Computational
Literacy”, their workplace or context of routine creation and consolidation, their experience /
exposure to problem solving of the kind proposed, and to the related solutions adopted, which
can be partly overlapping with the concept of “Computational Fluency”, their individual
attitudes to creativity for example, and the like. In [
To sum up, as also remarked by Tikva and Tambouris [
What we learnt from these literature on CT education? There are many aspects that should be taken into account when considering what is the property that we want to measure when we want to assess the degree or level of mastery of an individual vs. a problem whose level of dificulty has to be defined in its turn. What is it still missing from the however rich horizon of all the CT current definitions and perspective? For example, the existence of a cognitive model able to account for how hard or easy is to “abstract, generalize, evaluate”, and the like, and to apply all of the CT skills defined, based on the level of mastery of individuals and the level of dificulty of the problem at hand.
A Computational Thinking Scale (CTS) already exists: it is composed of the five sub-scales: that
related to the five CT Skills of: Abstraction, Decomposition, Algorithmic Thinking, Evaluation
and Generalization. The paper about DEA methodology for the design of EUDability, part of
this year edition of the IS-EUD proceedings [
From Table 1 it is clear that each dimension or skill of the CT frameworks currently outlined are to be kept separated. We should start from here to be aware of what is missing from this kind of conceptualization: that of giving at least a hypothetical model of what are the skills that are deemed more dificult and what are the ones deemed easier. In a few words, the open problem is that of constructing a variable, in which the skills can be put in a continuum, from the easiest to the harder, so as to model where, on average, people’s Computational Thinking (Literacy and/or Fluency) may be positioned. Or, said otherwise, what is the level of mastery of an individual with respect to a specific activity/skill/task. This kind of construct may be anchored for example to the perspective that each skill is somehow related to each other, and to the fact that it is cognitively impossible that one skills is mastered by an individual without having acquired the full mastery of another one, strictly related to the first one, and which has to be acquired beforehand.
In other words, we need a model of individuals’ abilities and skills dificulty, upon which a measurement scale could be validated and exploited to measure the level of Computational Thinking Skills of people with respect to an EUD paradigm / technique. Without the conceptualization of this model and / or construct, we are far from the realization of an evaluation tool able to give a more realistic and as objective as possible measure of EUDability, and to generalize our discourse to that of constructing measurement scales.
After this thorough analysis of existing models for CT Skills and assessment methods, I may conclude that none of the models designed for K-12 education assessment were shown to follow any rigorous methodology in their design, nor were existing models either revisited for fitting or found to reach a validation from their assessment with a CT construct and validated items.
A cognitive model of human CT learning progression would be intended to describe how
skills are developed towards the highest “literacy and fluency” achievements, i.e., the ability to
master the declarative and procedural knowledge of, as well as the creativity, around the topic of
interest. Besides their descriptive utility, models of this kind also serve as theoretical reference
for guiding in the design of measuring instruments for assessing the learning progression of
individuals with the topic of interest, along the progression steps of a model that can be built
upon. Constructing a model of this kind is part of a rigorous methodology to construct standard
measurement scales in the CT field to measure individuals’ progression with a certain topic
(e.g., EUD) [
Following a rigorous methodology to design and implement our CT Skill model, and items
for its assessment would enrich the value of the construct and of the assessment tool. In
particular, we propose to adhere to the BAS workflow 2 as an “integrated approach to developing
assessments that provide meaningful interpretations of individuals’ work relative to the cognitive
and developmental goals” [
In [
2Available at https://bearcenter.berkeley.edu/page/about-bear.
All these many aspects are concerning whether and to which extent the required level of abstraction is in the ability of the person who is in charge of managing the problem in computational terms. Furthermore, one may assume that abstraction is inherent to diferent levels of cognitive abilities related to EUD: the level of the problem (the goal or task to be executed), the level of the object (what has to be manipulated and managed to solve the problem), and the level of the solution (the mapping of the EUD technique and its steps to be accomplished) [22, 23].
Beside self-assessment questionnaires such as the CTS presented in the previous Section, specific items need to be conceived for measuring the level of abstraction that individuals can put in place when asked to solve a problem that implies the exploitation of CT Skills. For example, for the abstraction ability some items of interest could involve the request of identifying the relevant details of an object (from the computational standpoint), of focusing only on relevant details, and the capacity of reduction to the essential details. Other items should pertain the exemplification of patterns of computation, such as for example the rule-based paradigm, by proposing tasks of the “if-this-then-that” kind [24].
Other items should ascertain the level of generalization acquired by individuals with respect to some created/modified or applied solutions (e.g., some EUD technique), together with the capacity of classifying problems and / or objects under the same solution paradigm (also part of the generalization ability) or the capacity of identifying common properties and / or solutions for diferent objects /problems with core parts in common [ 19].
In this vein, abstraction should span over diferent activities that has the common goal
of making a simplification / reduction of the problem at hand in terms of sub-problems and
combination of solution rules. Seen from this perspective, abstraction seems to also include the
two properties / abilities of decomposition and generalization [
This paper starts from the definition of Computational Thinking skills (CT) for EUD and extends it to the problem of outlining a measurement construct to assess its level of mastery in individuals. Stemming from the literature re Computational Thinking and its assessment, it crosses the concepts of Computational Literacy (mastery of background knowledge) and Computational Fluency (mastery of practical application of background knowledge). These concepts are interleaving with individuals’ subjective abilities and with problems objective dificulties, forming an intricate network of elements to be systematized into a measurement construct. The main contribution of this paper is to define the requirements for a method and to propose a tool to find a cognitive model of CT for EUD and a construct with related items for validating such construct, towards a measurement scale. This construct should take into account how CT Skills are intrinsically related to each other and that the Computational Thinking Literacy or Fluency in an individual is not static, but rather it is evolving with her attitude, context of work and work practices. Furthermore, individuals’ evolution may also pertain to the social sphere of collaboration of users in teams and small groups. Interesting investigation may also start from the concept of Abstraction, as other authors did in the CS education domain when designing CT from a learning perspective. [17] S. Atmatzidou, S. Demetriadis, Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant diferences, Robotics and Autonomous Systems 75 (2016) 661–670. [18] J. H. Hill, B. J. Houle, S. M. Merritt, A. Stix, Applying abstraction to master complexity, in: Proceedings of the 2nd international workshop on The role of abstraction in software engineering, 2008, pp. 15–21. [19] A. C. Calderon, D. Skillicorn, A. Watt, N. Perham, A double dissociative study into the efectiveness of computational thinking, Education and Information Technologies 25 (2020) 1181–1192. [20] O. Lindeberg, J. Eriksson, Y. Dittrich, Using metaobject protocol to implement tailoring; possibilities and problems, in: The 6th World Conference on Integrated Design & Process Technology (IDPT, volume 2, 2002. [21] I. Cetin, E. Dubinsky, Reflective abstraction in computational thinking, The Journal of
Mathematical Behavior 47 (2017) 70–80. [22] A. Mørch, Three levels of end-user tailoring: Customization, integration, and extension,
Computers and design in context 1997 (1997) 61. [23] J. Perrenet, J. F. Groote, E. Kaasenbrood, Exploring students’ understanding of the concept of algorithm: levels of abstraction, ACM SIGCSE Bulletin 37 (2005) 64–68. [24] B. Ur, E. McManus, M. Pak Yong Ho, M. L. Littman, Practical trigger-action programming in the smart home, in: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, Association for Computing Machinery, New York, NY, USA, 2014, p. 803–812. doi:10.1145/2556288.2557420. [25] K. J. Newton, J. Leonard, A. Buss, C. G. Wright, J. Barnes-Johnson, Informal stem: Learning with robotics and game design in an urban context, Journal of Research on Technology in Education 52 (2020) 129–147.