=Paper=
{{Paper
|id=Vol-2128/scitech4
|storemode=property
|title=Programming for an Audience of Younger Students: Engaging Students in Creating Computational Models to Support Science Learning
|pdfUrl=https://ceur-ws.org/Vol-2128/scitech4.pdf
|volume=Vol-2128
|authors=Ashlyn Pierson,Doug Clark
}}
==Programming for an Audience of Younger Students: Engaging Students in Creating Computational Models to Support Science Learning==
https://ceur-ws.org/Vol-2128/scitech4.pdf
Programming for An Audience of Younger Students: Engaging
Students in Creating Computational Models to Support Science
Learning
Ashlyn Pierson, Vanderbilt University, ashlyn.pierson@vanderbilt.edu
Doug Clark, University of Calgary, douglas.clark@ucalgary.ca
Abstract: Research suggests that designing for an authentic external audience may support
conceptual growth by increasing engagement and offering students opportunities to reframe
their thinking. However, these ideas have not been previously tested in an empirical
comparison study. To explore if and how engaging students in programming computational
models for an external audience could support science learning, we compared the conceptual
growth, artifact quality, and classroom discourse of 6th graders designing models of tides
primarily for an external audience of 5th graders to the growth of 6th graders designing models
for their teacher and peers. We found that designing for an external audience of younger
children supports students’ conceptual growth about the mechanisms that cause tides as
evidenced by students’ pre-post assessments, models, and user guides/reports. Our analysis
suggests that designing for an external audience may have facilitated domain-specific
reasoning during whole-class discussions.
Introduction
Over the past two decades, studies have carefully chronicled the mechanisms through which constructionist
programming environments support students in restructuring their understanding of complex scientific systems
and processes (Kafai, Ching, & Marshall, 1997; Sengupta, Kinnebrew, Basu, Biswas, & Clark, 2013; Weintrop
et al., 2016; Wilensky & Resnick, 1999; Wilensky & Papert, 2010). We have located no published studies,
however, that have empirically compared the conceptual growth of students interacting with an external
audience to the conceptual growth of students interacting with an internal audience of a teacher and peers while
engaging with these constructionist modeling environments. To contribute to this line of research, we compare
two groups of 6th grade students who created computational models about tides. The current study compares
learning outcomes, artifact quality, and classroom discourse between sixth grade classrooms that designed
models of tides for an audience of younger students and sixth grade classrooms that designed models of tides as
an assignment for their teacher. More specifically, our first research question focuses on the degree to which an
external audience of younger children affects students’ conceptual learning and modeling artifacts. Our second
research question then explores the degree to which an external audience of younger children affects
opportunities for domain-specific reasoning in classroom discussions. This study therefore explores a way that
teachers could frame learning activities like computational modeling around an external audience in order to
support their students’ conceptual growth in science.
Participants and setting
This study was conducted in four 6th grade science classes with a total of 92 students in a public charter school
located in a large metropolitan school district in the southeastern United States. The same teacher taught all four
classes. Based on the state’s Report Card data, 16% of students at the school are English Learners, 48% of
students are eligible for free and reduced lunch, and 7% qualify as students with disabilities. Furthermore, 61%
of students are White, 17% are Hispanic or Latino, 15% are Black or African American, and 7% are Asian. Of
the 92 students, 79 students returned parental consent and student assent forms. The data from these 79 students
provide the foundation for this study.
The study took place during a week (5 days) of students’ typical 45-minute science class periods. We
co-designed the lesson activities with the teacher, because research suggests that teachers influence the level and
nature of the implementation of designed curricula, and that involving teachers in the design of curricula
supports meaningful use in classrooms, particularly when implementing technology-supported curricula
(Blumenfeld, Fishman, Krajcik, Marx, & Soloway, 2000; Means, Penuel, & Padilla, 2001; Roschelle & Penuel,
2006; Talbert & McLaughlin, 2002).
The modeling project was a part of a larger unit on space science. Specifically, the project was aligned
with the following state standard: Predict the types of tides that occur when the Earth and Moon occupy various
positions. According to the teacher, before this project, most students were able to draw or recognize the shape
of tides given the position of the Moon and Sun. Students were not able to explain, however, the mechanisms
that cause tides. Therefore, the central learning goal for this study was for students to be able to explain why
tidal bulges occur and how the Sun and Moon affect the shape of tides. Specifically, we aimed to help students
understand and explain (1) the gravitational and inertial tidal bulges caused by the moon, and (2) the spring and
neap tides created by the relative roles of the sun and the moon.
Intervention design
We formed two conditions (classroom-audience and external-audience) to compare conceptual growth and
engagement as students built computational models of tides using the constructionist programming environment
StarLogo Nova (see Figure 1). We assigned two of the four 6th grade classes to each condition. We distributed
academic ability across the conditions as evenly as possible by asking the teacher to order his classes from
highest to lowest academic performance. He told us that two classes were high achieving, one was in the
middle, and one was low achieving. The middle and low achieving classes contained fewer students than the
high achieving classes by design at the school so that students in these classes could receive more support from
the teacher. Based on this information, we assigned a high-achieving and middle-achieving class to the
classroom-audience condition, and we assigned a high-achieving and low-achieving class to the external-
audience condition. Because we predicted that the external-audience condition would experience more
engagement and conceptual growth than the classroom-audience condition, we “stacked the deck” against the
external-audience condition by assigning the lower performing class to this condition. To minimize effects
related to time of day, we chose to stagger the conditions so that the teacher alternated between classroom-
audience and external-audience classes.
Content instruction, lesson materials, and activities were identical for both conditions with the only
distinction between the conditions involving framing the activity for the external-audience condition in terms of
an audience of younger students. Students learned about tides through direct instruction from the teacher on Day
1 and Day 2. The teacher used a script to ensure that each class had the same access to information about tides
from these lessons.
Figure 1. A student’s computational model of tides created with StarLogo Nova.
Day 1: framing
On the first day of the project, when the teacher introduced the modeling task to the students, he told students in
the external-audience condition that they would be designing models for 5th graders at their school to help the 5th
graders learn about tides. The classroom-audience classes were told they were designing models to learn more
about tides and show their understanding of tides. The teacher mentioned designing for 5th graders four times to
the high-achieving external-audience class and seven times to the low-achieving external-audience class on Day
1. Throughout the week, he mentioned the 5th graders less frequently (between two and three times per class
period).
Day 4: feedback
To make the 5th grade audience tangible and meaningful for students in the external-audience condition, we
created an opportunity for students to share their models with 5th graders as they were designing their models.
The external-audience classes sent five representatives from each class to field-test their models with 5th grade
students during breakfast on Day 4 (about 10 minutes). The field-testers brought feedback from the 5th graders
to their science classes, and the classes grounded their discussion of “glows” (strengths) and “grows”
(weaknesses) for their models in the feedback from the 5th graders.
The classroom-audience students also received feedback on their models. However, rather than
receiving feedback from an external audience, they evaluated their peers’ models and provided feedback to each
other. Like the external-audience classes, the classroom-audience classes also generated glows and grows for
their models. All classes spent the same amount of time reflecting on the glows and grows of their models and
were encouraged to revise their models on Day 4 based on student-identified areas of improvement.
Day 5: framing
On Day 5, all four classes created Google documents that would help others understand their models. The
directions for this assignment were the same for both classes, but the assignment had different names in the two
conditions. For the external-audience classes, the teacher called the assignment a “user guide” and told students
that their user guides would help 5th graders understand their models. For the classroom-audience classes, this
assignment was called a “report,” and students were told that their reports would help their teacher understand
their models.
Data collection and analysis
The data for our analysis come from student artifacts and video recordings. Student artifacts include pre- and
post-assessments, computational models, and the user guides/reports. During both pre- and post-assessments,
students were provided with diagrams of the Earth, Sun, and Moon. Students were asked to draw the tides and
explain their drawings. These assessments allowed us to track students’ conceptual growth from the beginning
to the end of the project. Computational models show the coding blocks that students selected to program tides
as well as the simulations of tides created by their programs. User guides/reports are students’ written
reflections about their models. They include a description of what the model is about, the agents represented in
the model, and the ways that tides are formed. Video recordings come from a camera placed at the back of the
classroom to capture whole-class activities during the project.
Our first research question compares conceptual growth between the external-audience and classroom-
audience conditions. In order to answer this question, we analyzed data from student artifacts including pre- and
post-assessments, models, and user guides/reports. These data provide markers for individual students’
conceptual growth throughout the project. Our second research question compares opportunities for domain-
specific reasoning in the two conditions. While student artifacts provide static snapshots of student knowledge,
whole class discussions provide an opportunity to track students’ goals, priorities, and reasoning as they develop
across the week. Students were more likely to justify their representational choices during discussion than they
were in writing, making video recordings critical for understanding student reasoning.
Challenges and opportunities
We present the challenges and opportunities that emerged from our intervention by research question, beginning
with an analysis of science learning outcomes in terms of the mechanisms that cause tidal bulges. Then, we
analyze how designing for an audience of 5th graders versus designing for the teacher and peers shaped the
students’ design priorities and goals and supported domain-specific reasoning during whole-class discourse.
Research question 1: differences in science learning outcomes
First, we analyzed pre-post assessments. We scored each student’s pre-assessment and post-assessment using a
9-point scale that assessed students’ understanding each of the learning goals described above. Then, we used
independent-sample t-tests to compare conceptual growth in the external-audience and classroom-audience
conditions. We present our findings from the assessments in Table 1. There was no significant difference
between the pre-assessment scores of students in the external-audience and classroom-audience conditions,
suggesting that the two conditions began the intervention with similar levels of understanding. Students in the
external-audience condition, however, experienced significantly greater pre-post gains than students in the
classroom-audience condition. These findings tell us that the students in the external-audience condition began
with a slightly lower mean score on the pre-assessment, made significantly greater gains, and surpassed the
classroom-audience students on the post-assessment.
Data from the user guides/reports was collected to triangulate data from the assessments. These
artifacts were scored for conceptual growth using the same scale as the assessments. Because the directions for
the user guides/reports were more open-ended, students were less likely to address each of the three phenomena,
resulting in slightly lower mean scores for both conditions. Even so, an independent-sample t-test shows a
significant difference in scores for the external-audience (M = 4.48, SD = 2.59) and classroom-audience (M =
3.10, SD = 2.71) conditions, t(79) = 2.33, p = .02. Like the assessment data, these data demonstrate significantly
greater conceptual growth for students in the external-audience condition than students in the classroom-
audience condition.
Table 1: Pre-assessment, post-assessment, and pre-post gains
External-Audience (n = 42) Classroom-Audience (n = 39)
M SD M SD t-test p
Pre-Assessment 2.95 1.98 2.97 2.37 -.05 .96
Post-Assessment 5.17 2.64 4.13 2.22 1.91 .06
Pre-Post Gains 2.21 2.32 1.15 2.31 2.06 .04
We also scored students’ computational models to further triangulate our findings about students’
learning. In the computational models, students did not provide written explanations of phenomena, so we could
not tell if seemingly non-normative representations were intentional or if they were bugs in the students’
programs. Therefore, we coded students’ computational models for the presence of normative representations.
Models were scored on a scale of 0 to 4. Models earned one point for each of the following normative
phenomena: the gravitational interaction between the Moon and the tides, the inertial bulge opposite the Moon,
the gravitational interaction between the Sun and the tides, and the inertial effect opposite the Sun. Models
earned a score of 0 if they did not include any of these normative representations of tides. We used an
independent-sample t-test to compare the scores of the students in the external-audience condition to the scores
of students in the classroom-audience condition. We found that students in the external-audience condition (M =
2.54, SD = 1.04) earned significantly higher scores on their models than students in the classroom-audience
condition (M = 2.03, SD = 1.08), t(71) = 2.06, p = .04.
Research question 2: priorities and domain-specific discussions
Analysis of our first question establishes that students in the external-audience condition experienced more
conceptual growth than students in the classroom-audience condition. Our second question asks how and why
designing for an external audience supports domain-specific reasoning and, subsequently, conceptual growth
from the perspective of classroom discourse. To answer this question, we analyzed video data to determine how
attention to audience affects students’ priorities, and in turn, their opportunities for domain-specific discussions.
We found that designing for an external-audience facilitated domain-specific reasoning by prompting students
to represent the mechanisms that cause tides. This perspective encouraged students in the external-audience
condition to explicitly discuss their understanding of tides, providing opportunities for conceptual growth.
Table 2: Student priorities in classroom discourse
External-Audience Classroom-Audience
(Day1 2H 2L) (Day1 3H 1M)
Gravitational Bulges 10 7
(Day4 1H 5L) (Day4 1H 2M)
(Day1 0H 0L) (Day1 0H 0M)
Inertial Bulges 1 2
Domain- (Day4 1H 0L) (Day4 1H 1M)
Specific (Day1 1H 1L) (Day1 2H 0M)
Relative Role of Sun and Moon 9 2
(Day4 5H 2L) (Day4 0H 0M)
(Day1 3H 3L) (Day1 5H 1M)
Domain-Specific Total 20 11
(Day4 7H 7L) (Day4 2H 3M)
(Day1 2H 1L) (Day1 5H 2M)
Aesthetic 8 18
(Day4 0H 5L) (Day4 7H 4M)
Total 28 29
We coded student talk in four categories (Table 2). The first three categories correspond to the
assessment categories: gravitational bulges, inertial bulges, and the relative role of the sun and moon. The last
category represented aesthetic priorities. We coded student statements as aesthetic if they referred to the
appearance of the model rather than to the mechanisms that cause tides. We found that, during class discussions,
students in the external-audience condition were significantly more likely to consider domain-specific aspects of
their models, whereas students in the classroom-audience condition were more likely to prioritize aesthetics
(X2(1,N=57)=6.44, p = .01).
Statements were classified as aesthetic when they implicitly or explicitly communicated a goal of
improving the appearance of the model rather than addressing the mechanisms that cause tides. For example,
one student described trying to make the model “look nice.” In other instances, students did not explicitly state
that their goal was to make the model “look nice,” yet their aesthetic goals were evident in problems they
prioritized and in their solutions to these problems. For example, we provide a conversation from the classroom-
high class on Day 4. The students had identified the water’s “spazziness” as a problem in their models. Gabe
provides a solution to this problem:
Gabe: Uh, so, as you can see [points at his model on the projector] some,
occasionally, the water will kind of go everywhere and, um, like it’s
probably going to do it, um, but, what it will do, is, it, um, it doesn’t look
right, it’s like, yeah, here it is now, it’s just doing something really weird
and the water’s just dancing around.
Teacher: So [makes air quotes] spazziness as some people called it.
Gabe: But, um, then I realized, if you speed it up, then it looks more like fluid, and
it looks more like a tide, because, it’s not always going to be a straight line.
Hannah: How do you speed it up? [Teacher points to the speed bar at the bottom right
of the projected screen.]
Gabe: How do you make it go faster? Um there’s that thing [points to the speed
bar] and you bring it to the bottom and that’s how you do it.
Though Gabe connects his solution to tides, his goal is not to better explain or understand tides; instead, he is
focused on making his model look nicer. This is evident in his solution for the “spazziness.” Gabe’s solution
was to speed up the simulation, not to change the underlying code for the simulation. This indicates that Gabe
was not engaging with the mechanisms that cause tides; rather, he was trying to make his model look more
“fluid” or smooth. Aesthetic priorities like Gabe’s provide limited opportunities to engage in domain-specific
reasoning about tides, because problems with the appearance of the model can often be solved without
considering the mechanisms that cause tides.
In contrast, statements were classified as domain-specific when they engaged with the mechanisms that
cause tides. Lily's statement below (prompted by the teacher asking for improvements for the models) provides
an example in terms of the relative role of the sun and moon:
Well I tried to include the sun in my diagram last night, er um, I tried to do it, like, um under
“water,” I made a um thing. Like, I copied what we did for the moon but I put sun instead of
moon. I might need to change like how many steps away it has to be, because it’s farther away
than the Moon is.
Lily is grappling with how to represent the Sun’s role in the formation of tides. She mentions “copying what we
did for the moon,” but this solution did not help her represent spring and neap tides. This is because the code for
the water checked for a Moon 15 steps away, but the Sun was 50 steps away in Lily’s model. Lily identified the
problem noting, “I might have to change how many steps away it has to be, because it’s farther away than the
moon is.” This statement shows how Lily is mapping domain-specific knowledge, like the difference in distance
between the Sun and the Moon, onto her model. Negotiating between mental models of tides and computational
models in this way can support students in developing more coherent understandings of tides.
Keaton’s statement below (prompted by the teacher asking for improvements for the models) is also
coded as domain-specific in terms of the relative role of the sun and moon because through he considers
aesthetic elements of his model, his priority is to use the appearance of this model to more clearly represent
spring and neap tides:
Ok, to help teach the kids, I think that you should, uh, like in the um, in some of the
simulations we did before I saw when it lined up uh it like showed the 90 degree angle or the
straight line, when they lined it to show the spring or neap tides, I think that, like, when the
spring and neap tide occurred, we should like I don’t know, make a neon, or like a white line
that shows like, that shows, the like the angle.
He suggests using “neon or white lines” that “show the angle” to represent spring and neap tides, because he
thinks that will help “teach the kids” (the 5th graders). He is drawing on his own experiences, “some of the
simulations we did before,” to inform his design for other students. In this way, Keaton is shifting between his
current perspective, the perspective of his model, the perspective of the simulation, and the perspective of his
audience. These shifts in perspective support him in creating a more robust and coherent understanding of tides.
Thus, data from discourse suggest that designing for 5th graders shaped students’ goals in the external-
audience condition, which, in turn, shaped their opportunities for domain-specific reasoning. Students in the
external-audience condition were more likely to engage with the mechanisms that cause tides during class
discussions. Students in the classroom-audience condition were more likely to demonstrate aesthetic priorities
during class discussion and therefore had fewer opportunities to engage with the mechanisms that cause tides.
These data suggest that students in the external-audience condition had more opportunities to engage with
domain-specific reasoning, which may have contributed to the external-audience students’ greater conceptual
growth as measured by pre-post assessments, models, and user guides/reports.
Significance
This study contributes to the goals of ICLS by helping researchers and practitioners consider supports for
technology-facilitated learning within a complex social learning environment. In this paper, we show that small
changes in the way that the teacher framed the computational modeling activity dramatically shaped students’
opportunities for science learning. We demonstrate that when engaged in computational modeling, designing for
an audience of younger students supports students’ conceptual growth about the mechanisms that cause tidal
bulges across several assessments. Our analysis of classroom discourse suggests that designing for an audience
of younger children may have facilitated opportunities for rich scientific reasoning in whole class discussions.
Thus, these findings have practical implications, suggesting that incorporating an external audience into
classroom projects can promote engagement and conceptual growth for students.
References
Blumenfeld, P., Fishman, B. J., Krajcik, J., Marx, R. W., & Soloway, E. (2000). Creating usable innovations in
systemic reform: Scaling up technology-embedded project-based science in urban schools. Educational
Psychologist, 35(3), 149-164.
Kafai, Y. B., Ching, C. C., & Marshall, S. (1997). Children as designers of educational multimedia software.
Computers & Education, 29(2), 117-126.
Means, B., Penuel, W. R., & Padilla, C. (2001). The connected school: Technology and learning in high school.
San Francisco, CA: Jossey-Bass.
Roschelle, J., & Penuel, W. R. (2006, June). Co-design of innovations with teachers: Definition and dynamics.
In Proceedings of the 7th international conference on Learning sciences (pp. 606-612). International
Society of the Learning Sciences.
Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking
with K-12 science education using agent-based computation: A theoretical framework. Education and
Information Technologies, 18(2), 351-380.
Talbert, J. E., & McLaughlin, M. W. (2002). Assessing the school environment: Embedded contexts and
bottom-up research strategies. In S. L. Friedman & T. D. Wachs (Eds.), Measuring environment across
the life span: Emerging methods and concepts (pp. 197-227). Washington, DC, US: American
Psychological Association.
Wilensky, U., & Papert, S. (2010). Restructurations: Reformulations of knowledge disciplines through new
representational forms. Constructionism 2010, Paris.
Wilensky, U., & Resnick, M. (1999). Thinking in levels: A dynamic systems approach to making sense of the
world. Journal of Science Education and Technology, 8(1), 3-19.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining
computational thinking for mathematics and science classrooms. Journal of Science Education and
Technology, 25(1), 127–147.