Development of Computational Thinking Skills through Educational Robotics

Abstract

Computational thinking (CT) is an important concept in modern education. The scientific community is not only investigating the skills involved in CT but, is also trying to establish how these skills can be developed and through what technological means. Meanwhile, a few studies have investigated the effectiveness of educational robotics (ER) as technological means which can support the development of CT but, issues of measurement of CT (i.e., using validated instruments) seem to hinder the validity of these investigations. In this paper, two quasi-experimental studies were conducted to address students' CT gains linked to their participation in ER activities. The first study was conducted at a primary school in the Eastern Mediterranean; 15 consented students participated in ER activities for five weeks. The second study included 16 students in a secondary school in the same region, who participated in ER activities for three months. Quantitative results, based on a valid measure of CT, showed that the students who participated in the ER interventions demonstrated significant improvement in their CT skills. This study extends the evidence of the potential of using ER to improve students' CT skills in K-12 contexts.

Keywords

• Computational Thinking
• Educational Robotics
• K-12

-

Computational Thinking (CT) has been characterised as a fundamental skill of the 21st century for everybody, not only for computer scientists [ 33, 34 ]. The term was coined by Wing (2006) to raise awareness of computer-based education and broaden students’ participation in the field: “computational thinking involves solving problems, designing systems, and understanding human behaviour, by drawing on the concepts fundamental to computer science” (p. 33) [37]. Wing (2016) argued that people who can efficiently use computation would have an advantage over someone without this skill. CT was initially defined by Seymour Papert (in 1980 and again in 1996) as the ability to think computationally [ 22, 23 ]. Logo Programming has been the root of CT; it was developed in 1967 by Bobrow, Feurzeig, Papert and Solomon to help children develop procedural thinking [ 20, 22, 24, 29 ]. Countries such as UK, Australia, Singapore, South Korea and Israel are already making efforts to teach Computer Science (CS) in K-12. Block-based programming is typically used to support the teaching of CT concepts; this is mostly done using Scratch [ 11, 15 ], Alice [ 31 ], Agentsheets and video prompts [ 5, 7 ], Agecubes of the Agentsheets [ 6 ], NetLogo with online interactive tests [ 30 ], and App Inventor [ 21 ].

In the last few years, the use of educational robotics (ER) in schools has become a frequent topic of research in education. ER offer a broad range of challenges and opportunities for learners to develop disruptive thinking, innovative ideas, and other learning skills needed in the classroom and outside the school [ 35 ]. Anđić et al (2015) argued that students can process information faster when they use ER for learning at school. Based on their study in secondary schools in Montenegro, the authors suggested that ER should be introduced in school curriculum to help students acquire knowledge and skills to perform various activities in their daily lives at school [ 2 ]. Afari and Khine (2017) further argued that ER can help learners to acquire essential skills such as problem-solving, collaborative skills and critical thinking [ 1 ]. According to Toh et al (2016) and Kerr (2009), the use of ER in education promotes the development of the learners’ social, cognitive, language and conceptual skills [ 18, 28 ] and can support students in the development of academic skills [ 19 ].

While Wing (2016) argued that the limited number of K-12 teachers who can teach CS is a major practical challenge in promoting CT in schools [ 32 ], ER appears to be a promising tool in supporting this aim. Indeed, ER is seen as a tool for advancing CT, coding, and engineering [ 12,13 ]. For example, Kazakoff et al. (2013) examined prekindergarten students’ sequencing ability. As a result of one-week intensive ER workshop the participants experienced increased scores in the sequencing ability test [ 17 ]. Grover (2011) reported findings from a study with middle school and high school students who participated in ER activities for 8 hours per day for five days [ 14 ]. The authors used a pre-interview and post-interview design to measure the elements and dimensions of CT as expressed by the students; results showed that the students were able to use some CT-related vocabulary and principles upon the ER intervention [ 14 ]. Moreover, Eguchi (2014) discussed students’ learning gains in problem-solving, collaboration, and communication skills after their participation in the RoboCupJunior competition [ 12, 13 ]. More recently, the study of Chalmers (2018) examined the integration of ER in an undergraduate education course in Australia and demonstrated the positive impact of ER on the introduction of CT to young learners [ 9 ]. Positive findings in the development of CT skills have also been presented by Angeli (2018) who addressed how CT can be taught to young children via the use of ER [ 3 ].

Despite the considerable attention on the intersection of ER and CT in the recent years, the literature on advancing CT through ER in K-12 is still relatively sparse [ 4, 16, 36 ]. In a recent review by Ioannou and Makridou [ 16 ] only nine empirical investigations were found to have used ER in K-12 contexts to foster students’ CT skills. These nine studies raced a number of concerns about research in the area, assessment being one such major issue. The authors argued that “the results of the nine studies should be interpreted with caution; the psychometric properties of the instruments used were not reported and therefore, there is no evidence of validity of the data produced” [ 16 ]. The present investigation extends the current research evidence on the potential of ER to support the development of CT skills, whilst it addresses the important issue of assessment using a validated instrument for the measurement of CT. The overarching question of the investigation is:

Are there gains in students’ CT skills linked to their participation in ER activities? The investigation is enacted in two quasi experimental studies. First, a study was conducted with 15 consented students in primary school who participated in ER activities for five weeks during their summer-school program. A second study was conducted with 16 consented students in secondary school who participated in ER activities over the course of three months as part of their after-school club during the school year. The rest of the paper presents methods and results from the two studies. 2

Study 1

2.1. Participants

Study 1 used a one group pretest-posttest (quasi- experimental) research design to examine if students experience gains in CT after participation in a series of ER activities.

A total of 24 students in grades 5 and 6 (aged 10-12) participated in ER activities as part of their summer-school program at a private elementary school in the Eastern Mediterranean. Yet, only 15 students provided parental consent and assent for participation in the research study (i.e., providing pre-post CT data). The 15 participants were 12 boys and three girls. In terms of age, two were 10 years old, seven were 11 years old, and six were 12 years old. None of the children had formal programming experience with or without ER.

2.2. ER activities

The studies used the "Thymio" robot together with "Scratch" to let the student program the robot (see Fig. 1). The course was organized as a series of lessons around five modules linked to CT: simple guiding commands, basic repetitive commands, repetitive commands combined with statements, basic conditions and variables, conditions and functions (see Fig. 2). The lessons and activities were developed by an experienced ER educator and author of this work.

The 15 consented students were invited to complete the CT test (see section 2.4) the day before the first ER lesson. Fast track ER lessons (2 hours each) were conducted twice a week for five weeks as a summer school-program during July and August 2017. That is, a total of 20 hours of lessons were held around ER activities on “Thymio”. The students worked in groups of four (i.e., six groups); each group had its own robot and computer (see Fig. 3). Right after the completion of the last lesson, the consented students completed again the 45-minutes CT test.

The study used the Computational Thinking Test (CTt) [ 25, 26, 27 ], adopted from Román-González (2015) [ 26 ]. The aim of CTt is to quantify the learners’ ability to formulate and solve problems by relying on the fundamental concepts of computing and using logic-syntax of programming languages [ 27 ]. This is a 28-items multiplechoice test with four-answer options and only one correct answer; the test takes 45 minutes to complete. Each item addresses one of seven computational concepts, namely: “Basic directions and sequences; Loops repeat times; Loops repeat until; If simple conditional; If/else complex conditional; While conditional; Simple functions” [ 27 ]. These computational concepts are aligned with the CSTA Computer Science Standards for the 7th and 8th grade [ 10 ] and the CT framework by Brennan, & Resnick (2012) [ 8 ]. Román-González (2015) used two main interfaces to present the test items: “The Maze” and “The Canvas” (see Fig. 4). The test has undergone a formal validation possess in the Spanish language [ 27 ] and was translated in Greek for use in the present investigation.

Fig. 4. ‘Maze’ CTt - question 8 (left), ‘Canvas’ CTt - question 26 (right)

Results

A total CTt score (out of 100) was computed for each participant, by summing up the correct answers on the 28-item CTt and adjusting to 100. A paired-samples t-test was conducted using students’ data from the two administrations of CTt. The analysis showed a statistically significant increase, t(14) = 3.091, p = .008, from pre-testing (M=52.62%; SD=11.96) to post-testing (M=61.43%; SD= 11.78), with medium effect (d = .74) based on Cohen’s (1988) guidelines.

Design the encouraging results, the design of Study 1 was susceptive to threads to internal validity mainly because it lacked a control group. That is, in the particular case we cannot be sure that the increased CTt score was due to the students’ participation in the ER activities, for example a “testing thread” is possible (i.e., students’ exposure to pre-test might have affected their scores on the post-test). Despite the limitations, Study 1 provided encouraging evidence that the intervention performed as expected (i.e., ER helped to improve CT skills for the participating students). The promising results of Study 1 lead to the fully developed ER intervention of Study 2. 3

Study 2

Study 2 used a two-group pretest–posttest (quasi- experimental) research design to examine potential differences between the experimental and control group in CT gains from pre to post testing. The research hypothesis was that students in the experimental group, participating in a 3-months ER intervention, would experience increased gains in CT compared with their counterparts in the control group. 3.1

Participants

Study 2 involved a total of 32 students in grades 7, 8 and 9 (aged 12-14) at a primary school in the Eastern Mediterranean (different from school of Study 1). A total of 16 students (all male) formed the experimental group; these students had self-selected the ER after-school club for the school year. Their mean age was 12.6 years old (ten students were 12 years old, two were 13 years old, and four were 14 years old). The control group was composed of 16 students in other after-school clubs, who consented to take the CTt during pre and post administrations, without participation in any ER lessons. These were 14 boys and 2 girls with mean age 13.1 years old (14 students were 13 years old and two were 14 years old). None of the children in experimental or control groups had previous formal programming experience with or without ER. 3.2

Procedures

From October to December 2017, the 16 students of the experimental group participated in a total of 24 lessons (2-hours ER lessons, twice a week), that is, a total of 48 hours of ER activities. Pre-post testing and intervention for the experimental group were identical to Study 1. The ER activities using “Thymio” were consistent with those of Study 1 (Fig. 2), although a much larger pool of activities was used for 48 hours of work. The 16 students of the control group simply completed the CTt during pre and post administration (i.e, same time as the experimental group). 3.3

Results

Two paired-samples t-tests were conducted, one for each group, using students’ data from the two administrations of CTt. The analysis demonstrated statistically significant CT gains for students in the experimental group, t(15) = 5.985, p < .001, from pretesting (M=60.00%; SD=19.35) to post-testing (M=76.50%; SD= 13.25), with large effect (d = 1.00, i.e., groups' mean increased by one standard deviation).

Instead, there was no statistically significant difference in the CTt scores of the control group, t(15) = .691, p = .5, from pre-testing (M=59.44%; SD=11.51) to post-testing (M=61.06%; SD= 11.49).

To further examine the comparison of the differences between the posttest and pretest scores in each treatment group, a repeated measures analysis of variance was conducted, i.e. a “time by treatment interaction” effect. A statistically significant time by treatment interaction was found, F (1,30) = 16.860, p<.001, with large effect (partial η2 =.360), indicating that students in the experimental group had statistically significant larger gains on CT compared with students in the control group (Fig. 5).

The two-group pretest–posttest design of Study 2 addressed the internal validity issues of Study 1. Study 2 examined the differences between the posttest and pretest CTt scores in each treatment group and demonstrated a significant interaction effect verifying that the ER intervention helped to improve the CTt skills of the participating students.

Discussion

Despite the considerable attention on ER and CT in the recent years, the literature on advancing CT through ER in K-12 is still relatively sparse [ 4, 16, 36 ]. In fact, there is a lack of studies on ER which attempt to measure CT using validated instruments [ 16 ]. The present investigation extends the current research evidence on the potential of ER to support the development of CT skills, whilst it addresses the important issue of assessment using a validated instrument for the measurement of CT.

“Are there gains in students’ CT skills linked to their participation in ER activities?” was the overarching question of the investigation, enacted as two quasi-experimental studies. Both studies demonstrated that the ER intervention had a positive impact on the participants’ CT skills, which is consistent with previous research evidence on ER and CT (see review by [ 16 ]). Indeed, the CT scores of the students increased statistical significantly after the intervention of ER lessons. Moderate to large effects suggest that students’ CT gains are meaningful and that the results may have practical implications for researchers, instructors, and students using ER to address CT. Also, the longer duration of the intervention in Study 2 (3 months over 5 weeks in Study 1) seems to have strengthen the effect, from modulate (Study 1) to large (Study 1); the ideal duration of the intervention is a matter that warrants further investigation.

The small sample size in both studies limits the generalizability of these findings; future studies should aim for larger samples and a better gender balance (mostly boys in the present study). Moreover, the fact that students in the experimental group of Study 2 had self-selected the ER after-school club, as opposed to a randomized-controlled trial, introduces a limitation in this work, as these students might possess characteristics different from their counterparts in the control group; future work should aim for randomized trials within experimental designs. Closing, our series of lessons and activities were developed by an experienced ER educator yet there is a need to formulate and validate a formal CT curriculum to further integrate ER and CT in educational settings.

Acknowledgment References

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 692058.

References

1. Afari , E. , and Khine , M.S. , ( 2017 ). Robotics as an Educational Tool: Impact of Lego Mindstorms . International Journal of Information and Education Technology , 7 ( 6 ), 437 - 442 . http://dx.doi.org/10.18178/ijiet. 2017 . 7 .6. 908

2. Anđić , B. , Grujičić , R. & Markuš , M.M. ( 2015 ). Robotics and Its Effects on the Educational System of Montenegro . World Journal of Education , 5 ( 4 ), 52 - 57 . https://doi.org/10.5430/wje.v5n4p52

3. Angeli , C. ( 2018 , March). Developing Third-Grade Students ' Computational Thinking Skills with Educational Robotics . In Society for Information Technology & Teacher Education International Conference (pp. 1 - 8 ). Association for the Advancement of Computing in Education (AACE).

4. Atmatzidou , S. , & Demetriadis , S. ( 2016 ). Advancing students' computational thinking skills through educational robotics: A study on age and gender relevant differences . Robotics and Autonomous Systems , 75 , 661 - 670 . https://doi.org/10.1016/j.robot. 2015 . 10 .008

5. Basawapatna , A. R. , Koh , K. H. , & Repenning , A. ( 2010 , June). Using scalable game design to teach computer science from middle school to graduate school . In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education (pp. 224 - 228 ). ACM. http://dx.doi.org/10.1145/1822090.1822154

6. Basawapatna , A. R. , Repenning , A. , & Koh , K. H. ( 2015 , February). Closing the cyberlearning loop: Enabling teachers to formatively assess student programming projects . In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 12 - 17 ). ACM. http://dx.doi.org/10.1145/2676723.2677269

7. Basawapatna , A. , Koh , K. H. , Repenning , A. , Webb , D. C. , & Marshall , K. S. ( 2011 , March). Recognizing computational thinking patterns . In Proceedings of the 42nd ACM technical symposium on Computer science education (pp. 245 - 250 ). ACM. http://dx.doi.org/10.1145/1953163.1953241

8. Brennan , K. , & Resnick , M. ( 2012 , April). New frameworks for studying and assessing the development of computational thinking . In Proceedings of the 2012 annual meeting of the American Educational Research Association , Vancouver, Canada (pp. 1 - 25 ).

9. Chalmers , C. ( 2018 ). Robotics and computational thinking in primary school . International Journal of Child-Computer Interaction , 17 , 93 - 100 .

10. CSTA. ( 2011 ). K-12 computer science standards. Retrieved from http://www.ioinformatics.org/conf/c5_CSTA.pdf

11. Dwyer , H. , Boe , B. , Hill , C. , Franklin , D. , & Harlow , D. ( 2013 ). Computational Thinking for Physics: Programming Models of Physics Phenomenon in Elementary School . Engelhardt, Churukian, & Jones (Eds.) 2013 PERC Proceedings , 133 - 136 .

12. Eguchi , A. ( 2014a ). Educational robotics for promoting 21st century skills . Journal of Automation Mobile Robotics and Intelligent Systems , 8 ( 1 ), 5 - 11 .

13. Eguchi , A. ( 2014b ). Learning experience through RoboCupJunior: Promoting STEM education and 21st century skills with robotics competition . In Proceedings of Society for Information Technology & Teacher Education International Conference.

14. Grover , S. ( 2011 , April). Robotics and engineering for middle and high school students to develop computational thinking . In annual meeting of the American Educational Research Association , New Orleans, LA. https://doi.org/10.3102/0013189X12463051

15. Grover , S. , Cooper , S. , & Pea , R. ( 2014 , June). Assessing computational learning in K-12 . In Proceedings of the 2014 conference on Innovation & technology in computer science education (pp. 57 - 62 ). ACM. https://doi.org/10.1145/2591708.2591713

16. Ioannou , A. , & Makridou , E. ( 2018 ). Exploring the potentials of educational robotics in thedevelopment of computational thinking: A summary of current research and practical proposal forfuture work Education and Information Technologies: https://doi.org/10.1007/s10639-018-9729-z

17. Kazakoff , E. R. , Sullivan , A. , & Bers , M. U. ( 2013 ). The effect of a classroom-based intensive robotics and programming workshop on sequencing ability in early childhood . Early Childhood Education Journal , 41 ( 4 ), 245 - 255 .

18. Kerr , B.A. ( 2009 ). Encyclopedia of giftedness, creativity, and talent . Thousand Oaks , Calif: Sage Publications.

19. Lee , I. , Martin , F. , Denner , J. , Coulter , B. , Allan , W. , Erickson , J. , ... & Werner , L. ( 2011 ). Computational thinking for youth in practice . Acm Inroads , 2 ( 1 ), 32 - 37 . http://dx.doi.org/10.1145/1929887.1929902.

20. Logo Foundation, http://el.media.mit.edu/logo-foundation/.

21. Morelli , R. , De Lanerolle, T. , Lake , P. , Limardo , N. , Tamotsu , E. , & Uche , C. ( 2011 , March). Can android app inventor bring computational thinking to k-12 . In Proc. 42nd ACM technical symposium on Computer science education (SIGCSE'11) (pp. 1 - 6 ).

22. Papert , S. ( 1980 ). Mindstorms: Children, computers, and powerful ideas . Basic Books , Inc.. ISBN:0-465-04627-4

23. Papert , S. ( 1996 ). An exploration in the space of mathematics educations . International Journal of Computers for Mathematical Learning , 1 ( 1 ), 95 - 123 . https://doi.org/10.1007/BF00191473

24. Papert , S. , & Harel , I. ( 1991 ). Situating constructionism . Constructionism , 36 ( 2 ), 1 - 11 .

25. Román-González M. , Moreno-León J. , & Robles G. ( 2017 ). Complementary Tools for Computational Thinking Assessment . In press, Hong Kong.

26. Román-González , M. ( 2015 ). Computational thinking Test: Design guidelines and content validation . In Proceedings of the 7th Annual International Conference on Education and New Learning Technologies (EDULEARN 2015 ) (pp . 2436 - 2444 ). https://doi.org/10.13140 /RG.2.1.4203.4329

27. Román-González , M. , Pérez-González , J. C. , & Jiménez-Fernández , C. ( 2017 ). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test . Computers in Human Behavior , 72 , 678 - 691 . https://doi.org/10.1016/j.chb. 2016 . 08 .047

28. Toh , L. P. E. , Causo , A. , Tzuo , P. W. , Chen , I. M. , & Yeo , S. H. ( 2016 ). A Review on the Use of Robots in Education and Young Children . Educational Technology & Society , 19 ( 2 ), 148 - 163 .

29. Voogt , J. , Fisser , P. , Good , J. , Mishra , P. , & Yadav , A. ( 2015 ). Computational thinking in compulsory education: Towards an agenda for research and practice . Education and Information Technologies , 20 ( 4 ), 715 - 728 . https://doi.org/10.1007/s10639-015-9412-6

30. Weintrop , D. , Beheshti , E. , Horn , M. S. , Orton , K. , Trouille , L. , Jona , K. , & Wilensky , U. ( 2014 , July) . Interactive assessment tools for computational thinking in High School STEM classrooms . In International Conference on Intelligent Technologies for Interactive Entertainment (pp. 22 - 25 ). Springer, Cham. https://doi.org/10.1007/978-3- 319 -08189- 2 _ 3

31. Werner , L. , Denner , J. , Campe , S. , & Kawamoto , D. C. ( 2012 , February) . The fairy performance assessment: measuring computational thinking in middle school . In Proceedings of the 43rd ACM technical symposium on Computer Science Education (pp. 215 - 220 ). ACM.

32. Wing 2016 - https://www.microsoft.com/en-us/research/blog/computational-thinking-10 - years-later/

33. Wing J.M. ( 2010 ). Computational Thinking: What and Why? Center for Computational Thinking , Carnegie Mellon. [Online] Available at https://www.cs.cmu.edu/~CompThink/resources/TheLinkWing.pdf [Accessed 10th March 2018 ].

34. Wing , J. M. ( 2006 ). Computational thinking . Communications of the ACM , 49 ( 3 ), 33 - 35 .

35. Xie , A. , & Huang , X. ( 2012 ). Advances in computer science and education . Heidelberg: Springer.

36. Yadav , A. , Zhou , N. , Mayfield , C. , Hambrusch , S. , & Korb , J. T. ( 2011 , March). Introducing computational thinking in education courses . In Proceedings of the 42nd ACM technical symposium on Computer science education (pp. 465 - 470 ). ACM.