=Paper=
{{Paper
|id=Vol-1358/paper2
|storemode=property
|title=An Online Badging System Supporting Educators' STEM Learning
|pdfUrl=https://ceur-ws.org/Vol-1358/paper2.pdf
|volume=Vol-1358
|dblpUrl=https://dblp.org/rec/conf/lak/GamratZ15
}}
==An Online Badging System Supporting Educators' STEM Learning==
https://ceur-ws.org/Vol-1358/paper2.pdf
An Online Badging System Supporting Educators’ STEM
Learning
Chris Gamrat Heather Toomey Zimmerman
Penn State University Penn State University
Learning, Design, and Technology Learning, Design, and Technology
University Park, PA 16802 University Park, PA 16802
gamrat@psu.edu heather@psu.edu
ABSTRACT badges can be housed as online representations within
In this paper, we investigate how a digital badging system was private badging systems, they can also be shown on other
used as part of an informal, not-for-credit professional websites to serve as visual representation of learners’
development project. Teacher Learning Journeys was designed experiences to outside audiences through Open Badges
for personalized science learning for educators in K-12 standards, an initiative of the Mozilla Foundation. When
schools, museums, universities, and teaching colleges through badges are considered as a communicative tool, researchers
employing two levels of micro-credentials: lower achievement have been renaming digital badges as Open Badges [13].
digital stamps and higher achievement digital badges. When shown to external audiences, digital badges can
Researchers conducted a qualitative collective case study contain metadata visible by others (i.e., the issuer,
centered on 36 teachers; the primary data were records from certifying agency, activity description, expertise the badge
learners’ interactions within the digital badge system; signifies, and evaluation criteria). Thus, badges offer
secondary data came from a survey at the end of the transparency to the assessment of individuals’
experience and two interviews with 11 focal teachers. Findings achievements and are available for scrutiny [10]. In this
suggest the following design principles: (a) two levels of
way, digital (or open) badges with their associated
assessment can support personalized learning, (b) mastery of
metadata are part of people’s professional portfolios that
learning can be demonstrated and assessed through reflective
logs, (c) collaboration during and after badging activities can
document learning, skills, and experiences in an easy-to-
provide value to the learners, and (d) establishment of share format. Open Badges and digital badges are terms
relevance of badging experiences can support the application used throughout the technology-enhanced learning field.
of content outside the badging system. Our system (described below) was designed to be Open-
Badge compatible, but this functionality was not available
Categories and Subject Descriptors for learners during the study period, so we use the term
K.3.1 [Computers and Education]: Computer uses in digital badges in this paper.
Education – collaborative learning Digital badges have been used in social media and online
games in five key areas identified by Antin and Churchill
General Terms [3]: (a) setting goals and providing feedback on goal
Design, Human Factors achievement, (b) providing instruction about what activities
are possible, (c) building a user’s reputation based on
Keywords interests, (d) serving as a status symbol and documenting
Digital badges, Open Badges, Science Learning, STEM achievements, and (e) showing affiliation with a
Education, Professional Development, Lifelong Learning, community. As digital badges have been used increasingly
Learning Across Settings in education, three of these five gaming categorizations
1. DIGITAL BADGES IN EDUCATION from above have been conceptualized as important for use
Digital badges have been proposed as tools that learners in learning environments: (a) rewarding and motivating
can use to support short-term through lifelong learning achievements, (b) credentialing or recognizing learning
within and across various educational institutions [10], (with badges serving as micro-assessments), and (c) acting
[11]. We define digital badges as online representations of as markers that learning has occurred for learners
learning experiences, which serve as micro-credentials that themselves, the learners’ teachers or mentors, and the
document learners’ expertise and skills. While digital learners’ peers [2], [13].
1.1 Research Findings on Badges within
Copyright © by the paper’s authors. Copying permitted for
private and academic purposes. Educational Systems
In: D. Hickey, J. Jovanovic, S. Lonn, J.E. Willis, III (eds.): While many types of educational institutions are adopting
Proceedings of the Open Badges in Education (OBIE 2015) digital badges, empirical research studies (qualitative,
Workshop, Poughkeepsie, New York, USA, 16-Mar-2015, quantitative, or design-based investigations) into how
published at http://ceur-ws.org digital badges (and now Open Badges) support learning
have only recently been published. Because the research is badges to support self-regulation in an online STEM
new, themes on the role of digital and Open Badges in course, Cucchiara and colleagues [8] developed interrelated
education are emerging. badges, in what they referred to as a “badge ecosystem,” to
incorporate technical and interpersonal (i.e., soft) skills.
Through an analysis of 30 badging projects funded by the
These nontechnical badges were important aspects of the
Badges for Badges for Lifelong Learning Competition
holistic badge offering for this project.
(funded by the MacArthur Foundation) Hickey et al. [11]
offer numerous design principles for recognizing, 1.2 Digital Badges as Pedagogical Tools
motivating, assessing, and studying learning with digital Within the broader landscape of technology-enhanced
badges and within badging systems. Six of these design learning research, our research interests are aligned to
principles are related to our inquiry: (1) support learners to badges as pedagogical tools. Our perspective is that badges
set goals related to the badging activities, (2) map learning can support learning and act as markers that learning has
trajectory through badge display pages and badging happened. The recognition that learning has happened is
activity logs, (3) promote collaboration through the important not only for learners, but also for their teachers
discussion of personal backgrounds and experience in the and mentors, and their peers in the same learning
subject matter, (4) use hierarchy of badges or leveled badge environment [2], [13]. In this way, we consider badges as
systems, which suggests learners pick interlocked content pedagogical tools used by learners and their teachers and
and decide their level of assessment, (5) use mastery mentors to support informal learning.
learning to focus on awarding badges to learners who
reflected on their activity and described how they might Hickey et al. [11] have categorized the extant badging
implement it in the future—outside the badging system, research, into various types. Given the Hickey et al.
and (6) enhance the badges’ validity with expert judgment, typology of badging research, our analysis falls into
which asserts that master practitioners or other experts can research with badges and research of badges [11]. This
be used to review the work of learners and to act as means that we are considering how to improve the impact
mentors. of badges in the learners’ lives (i.e., research of badges),
while we are also attending to the metadata within the
In research with 51 middle school pupils (seventh and badging system to improve the badging experiences
eighth grade) who were learning science, technology, (research with badges) [11]. Our research group has created
engineering, and math (STEM), Abramovich, Schunn, and a series of projects to investigate digital badges as a means
Higashi [1] investigated the interaction of badges with for increasing learners’ choices and decision making within
learners’ prior knowledge and motivation. To accomplish and across informal, formal, and professional development
their study, the team designed two types of digital badges: activities. Research with badges and for badges refers to
one was awarded for participation (without regard for our intent to use badges to offer improvements for systemic
learning outcomes) and the other was awarded for impact such as offering a mechanism to support
demonstrated mastery of STEM skills (as an assessment). professional development needs.
Abramovich et al. found that learners were motivated Our project’s overarching research goal is to examine
differently by the badges. Participation-only badges were learners’ badge-earning pathways in order to refine
earned eventually by every student and had little to no learning theory about how the things that learners learn in
impact on learning and motivation. In the case of the skill- one setting can be applied to another consequential setting
oriented badges, the more skill badges earned, the more via computer tools. For example, our team’s prior work has
those learners expected to be successful in mathematics. examined the role of: digital photography in supporting
The authors concluded that learners who earned the least learning across settings [20], mobile computers in
number of skill-oriented badges also had less expectation supporting meaningful conversations about science where
for STEM content learning success than those who earned families’ prior experiences are made relevant to new
more skill-oriented badges. experiences in informal institutions, [21] and
From their study, Abramovich et al. make two technologically-enhanced web 2.0 tools in supporting
recommendations to the design of badging systems. First, connections from community to school [22]. Our project
they suggest that to maintain high levels of learner here builds on our team’s prior work of designing for
motivation badging system designers include few learning across settings. In addition, within this badging
participation-only badges and have many skill-oriented project, we seek to develop design principles applicable to
digital badges available. Second, based on an analysis of improving the educational programming related to badges
survey data, Abramovich et al. suggest the provision of and the badging system designs of badge issuers.
instructions on the criteria needed to earn a badge will also In this analysis, we focus on investigating the following
assist to keep motivation high. research question: How do learners interact with elements
While much of the research has occurred within STEM of a digital badging system including the goal statements,
learning environments, not all badges are focused on logs, materials submitted to earn a stamp or badge, and
STEM content. For instance, from their case study of their mentor? We answer this question through a tiered case
study analysis of 36 schoolteachers and with the in-depth A teacher started their learning journey by creating a TLJ
analysis of 11 of the 36 teachers. Of these 11 focal account and writing a goal statement. While optional, the
teachers, we present a thick description of two teachers’ goal statement was used to meet goal-setting needs by
experiences—one teacher who represented a typical badge helping the teacher to begin to articulate a learning pathway
system user and one teacher who represented an intense [11]. Next, the teacher could review all 54 PD badging
user of badging. activities or search through them by topic or grade range
and decide which to add initially to their itinerary. Only the
2. TEACHER LEARNING JOURNEYS teacher and the research team could access each itinerary.
(TLJ) BADGING SYSTEM In this section, the teacher could review the badging
TLJ was created to support teacher professional activities they wanted to accomplish or remove badging
development (PD) in STEM subjects for K-12 education in activities they no longer wanted to complete.
the United States. Teachers accessed the TLJ system to
engage in STEM learning at their own pace and to meet The teacher then could initiate any PD badging activity
their own identified needs and a level of depth that from their itinerary by attending a webinar or engaging in
reflected the needs of the pupils in their classrooms. other educational activities. In TLJ, no credit was available
just for participation; the ideas had to be applied to the
2.1 TLJ’s Goal to Support STEM Learning teacher’s educational setting. The teacher then chose to
In the first year of the TLJ system’s operation that is submit their materials to earn a stamp or a badge, as
presented here, educational badging options included 54 described in Section 2.3. A National Aeronautics and Space
activities related to three areas of science and engineering: Administration (NASA) education expert was assigned as a
solar system (19 badging activities), weather and climate mentor to the teacher based on the content area of the
(20 badging activities), and engineering (16 badging badge. The mentor provided written feedback to the
activities). The teachers earned badges by reflecting on teacher, which resulted in the (a) earning of a badge or a
activities that they participated in; these badging activities stamp or (b) additional activities to be completed. The
covered a wide-range of PD activities. PD activities used in feedback was stored in the system as part of the metadata
our badging program included synchronous webinars, associated with their engagement with a particular activity
asynchronous discussion boards, educational media and and could be accessed by the teacher, the mentor, and the
texts, archived online presentations by scientists, engineers, researchers. Then, the teacher’s itinerary would change to
and education experts. Figure 1 shows one of the 20 show that the badge was earned.
badging activities in the Weather & Climate library. After earning a badge or stamp, the teacher would have the
option to anonymously fill out a survey on the activity,
which gave the research and development team valuable
feedback. The teacher would also be able to view the status
of their badges and stamps in portfolio pages for the earned
achievements. Additionally, any badges or stamps that
were being reviewed by mentors were also displayed with
the text pending superimposed on the badge image. The
Figure 1. The Remote Sensing badging activity within teacher could also review their work in the log section. This
the weather and climate topic area. feature of TLJ acted as a blog-style page that captured each
entry the teacher made for the activities completed as well
2.2 Features of the Badging System as any additional evidence they uploaded for review by a
Teachers were introduced to the system through an mentor or that they wanted to be included in their badging
instructional video created by the principal investigator of portfolio.
the project. This video elaborated on badges and each of
the functions available. Also, teachers were invited to use 2.3 Features of the TLJ Stamps and Badges
as many of the available badging PD resources as they An important feature of the TLJ badges was that they were
wanted. leveled [11]. Once teachers selected the PD badging
activities from TLJ, they controlled their level of
The TLJ badging system was developed utilizing a travel engagement with the topic by adjusting the type of micro-
metaphor. TLJ helped teachers to set their unique learning credential sought for learning —
a TLJ stamp or TLJ badge.
goals for their badging journey, find relevant badging
A TLJ stamp reflects the teacher’s need for a more cursory
activities to support their goals, compose a log of their
engagement with and consequent assessment of an activity.
experience for the activity that could be submitted to earn
The stamp is the lowest level badge. A TLJ badge is earned
badges, receive feedback from an expert as they continued
for a higher level of mastery and shows a more in-depth
their journey, and store all of their badges
engagement with not only the STEM concept but also with
(accomplishments) in display pages styled to be similar to
applying the STEM concepts to the teachers’ professional
that of the stamp inside of a passport.
settings. The TLJ badge is the highest-level badge, and for
the sake of clarity in the paper, we will only refer to the
highest level of badges as badges; the lower level will be cultural tools to mediate STEM meaning-making for
referred to as stamps. teachers along the dimensions of collaboration,
authenticity, and personalization, as described below.
To illustrate the difference between badges and stamps, the
PD badging activity, Scale Models: The Earth-Moon 3.2 Dimensions of Mobile Learning
System in the Solar System is a good example. This Scale Environments
Models activity has three options for teachers: (1) the Our TLJ research and development was guided by a
entry-level activity is earning a TLJ stamp (Figure 2 left); framework for mobile learning [8] because TLJ was
teachers attend the webinar and write one brief reflective intended for use by learners as they moved across social
post, (2) a higher level of mastery means earning a TLJ and technological settings—in their schools, homes, or
badge (Figure 2 right), teachers also attend the webinar but communities accessed by desktops, smartphones, tablets,
they write a full lesson plan on incorporating the content and other computers. While our prior work focused only on
into classroom activity, or (3), teachers can earn both the the personalization dimension of mobile learning [9], this
TLJ stamp and badge for this Scale Models activity by analysis focuses equally on authenticity (also referred to as
meeting both the stamp and badge criteria. To make the relevance), collaboration, and personalization because
stamp and badges visually distinct, the stamp was given a when related to teacher PD, these three learning dimensions
design akin to a passport office rubber stamp and the badge address the prior critiques of ineffective PD [5] and show
an embroidered patch as shown in Figure 2. what is possible through digital badging systems.
3.2.1 Authenticity
In the Kearney et al. model [14], authenticity is a mobile
learning dimension comprised of two subcomponents:
contextualization and situatedness. Contextualization is
high when a learner is exposed to learning situations that
are perceived to be realistic or relevant [4] to their personal
or professional life. By making use of TLJ, teachers focus
on the specific PD badging activities that are relevant to the
topics and grade levels that they are teaching. This helps
teachers create an authentic context for the badges available
through the TLJ system. Situatedness is high when a
Figure 2. A TLJ stamp (left) and a TLJ badge (right). badging learning experience is related to or within a
3. THEORETICAL FRAMEWORK community of practice [19]. This can be a full participation,
To support our analytical work, our theoretical framework such as engaging in conversations with the fellow STEM
considers bringing together the concept of cultural tools teachers, or the engagement can be a peripheral activity,
from sociocultural theory [18] with three dimensions of such as observing discussions of the high-engagement
mobile learning [14] (i.e., authenticity, collaboration, and webinar participants. This can also relate to the value of
personalization). As such, we examine how badges and the badge as status to the community of practitioners within
badging system acted as cultural tools to mediate the which the teachers are engaged [2], [3], [13].
teachers’ learning in regard to authenticity to relevant 3.2.2 Collaboration
STEM teaching and learning in K-12 schools, collaboration With the emphasis of socio-cultural considerations,
with other teachers and mentors, and personalization to cooperative, communicative, and collaborative efforts to
their educational setting. support meaning-making are included as a dimension in the
3.1 Cultural Tools to Mediate Learning Kearney et al. [14] framework for mobile learning.
Vygotsky’s [18] work in sociocultural theory maintains that Collaboration is included in this framework to permit
learning requires people to use cultural tools with the help learners multiple opportunities to articulate their
of experienced others. Cultural tools are artifacts such as understandings as they engage with others in shared
physical objects, conceptual symbols (including language), meaning-making experiences. Conversation, both verbal
and representations that people use within goal-directed and written, provides for a valuable ongoing exchange of
actions [13]. For example, within studies of technology- ideas that support the development of STEM teaching
enhanced learning, Cole [7] describes three kinds of practices. In addition to shared meaning-making,
artifacts that serve as cultural tools: (a) physical objects collaboration is meant to include the exchange of
(e.g., a computer, calculator, or hammer), (b) information between learners and between learners and
representations and ways of acting, and (c) imagined mentors. Through providing collaboration, cooperation, and
artifacts (e.g., games and art). In this way, cultural tools are communication opportunities, badge system designers can
the set of resources available to members of a cultural encourage the learners to become more connected with
group to mediate meaning-making. In our work, we each other and share experiences. In the case of TLJ, this
consider how the badges and the badging system served as sharing goes beyond STEM content to also include
information about student learning, needed resources (e.g., in each category of 0–5 years, 6–15 years, and more than
favorite web links), and even lesson plans that work within 15 years of work experience. The research participants
local contexts. came from the United States but with the majority in the
eastern and central US regions.
3.2.3 Personalization
Personalization is a dimension of mobile learning that 4.2 Data Collection
allows for learner-centered education because learners can Four different types of data were collected during year one:
customize many factors related to their learning. Kearney et (a) the 36 participants’ stamp and badge logs and goal
al. [8] refer to personalization as a concept that supports the setting statements (n=183 user-generated documents,
“just enough, just-in-time, just-for-me” aspect of mobile including stamp and badge logs and goal statements), (b)
learning which can allow learners to create their own online year one evaluation survey (n=29, 80.5% of total),
pathway of learning. Agency [15], [17], a subcomponent of (c) online activity evaluation surveys for each complete
personalization, is high when a learner is able to negotiate badge or stamp activity (n=76), and (d) pre- and post-
and make decisions about their own education, such as interviews with 11 preselected teachers. These data types
what subject matter to take and what goals to set. The are described in-depth below.
dimension of customization furthers personalization by
allowing the learner to select what they need to meet their 4.2.1 System Logs and User Generated Documents
personal and professional goals. TLJ offers teachers the The system logs captured all user-generated documents.
ability to both set their learning goals and to meet these The system logs also captured when a teacher added a
goals by navigating through and choosing from available badging activity to their itinerary and the recognition
PD activities. Each activity is a smaller part of what might (stamp, badge, or both) that they received for it.
normally make up a larger workshop or course. This allows 4.2.2 Summative Survey and Post-badging Surveys
teachers to customize what they spend time on to meet both A 13 question electronic survey was sent out as a
their personal needs and the needs of their workplace (and summative evaluation at the end of year one to the 36
pupils). While the field of educational technology might participants. The response rate was 80.5%. The survey
also use the term personalization to refer to personalization captured opinions on the overall experience with TLJ as
offered through data mining patterns generated by an well as possible future improvements. The activity
intelligent computer system, here, we use the term evaluation surveys were made available upon completion
personalization to refer to providing teachers with the of any badging activities. The activity surveys were
agency and the ability to customize. intended to understand the perceptions of value and quality
4. METHODOLOGY of the individual activities and to determine if any activities
The purpose of this paper is to understand the learners’ needed significant revision for year two of TLJ.
experiences with and through TLJ. We seek to use learner 4.2.3 Pre- and Post-Interviews with Focal Teachers
badging artifacts and interviews to understand how the TLJ For the interviews, 11 teachers were strategically selected
teachers interacted with elements of a digital badging by the categories shown in Table 1. The pre- and post-
system including the goal statements, logs, materials interviews were conducted over the phone. Each interview
submitted to earn a stamp or badge, and their mentor. lasted 45 minutes to one hour. The questions were focused
Hickey and team [11] have categorized the funded research on perceptions of TLJ and other experiences with features
(both completed and in progress) into various types; our of the badging environment and included: “Did you find
research falls into the category of ‘research of badges’ value in setting a purpose statement for your learning
where researchers “study the badge system’s impact and journey?” “Did you find the itinerary an easy way to keep
integration into learners’ lives” (p. 57). Often, research of track of your professional development activities? and “Did
badges use includes interviews, surveys, and data mining you find value in the feedback you received from the
[11], and as such, our research follows those norms. education specialists that reviewed your badging logs?”
4.1 Research Participants 4.3 Case Study Structure
Teachers were recruited from partnering organizations, The research team designed a strategic case study [16] at
with the assistance of the National Science Teachers three levels. At the first level of the case, the badging
Association and NASA, via listservs and emails. Teachers records of all 36 teachers from year one were considered.
at all levels in all kinds of educational institutions were This level of analysis was investigated through the data
invited to participate in this professional development collected by the activity surveys, the post-TLJ survey, and
experience. Teachers were not compensated for the system logs. Second, a more fine-grained analysis
participating. All teachers who expressed interest were examined 11 teachers who completed pre- and post-
accepted in the study, but only the 36 teachers who earned interviews for year one of TLJ (shown in Table 1). These
at least one stamp or badge in year one were included in interviews with teachers were used to ensure that out-of-
this analysis. The majority of research participants were at badging data were used to enrich the analysis. To
the elementary or middle school levels—with nearly a third understand learner intention and learner background, these
teachers serve to help contextualize the TLJ badging badging system—in their classrooms, in their schools, and
system in greater depth. in their lives.
Table 1. Matrix sampling for 11 pre- and post-TLJ The analytical process began by iteratively reading the
interviews records of 11 teachers who participated in the interviews.
We made this choice because we had data from outside the
Work Elementary Middle School
TLJ system about their interests in badging, the STEM
experience Teacher Teachers
content and their workplace conditions. The interviews
2 teachers 2 teachers focused on the teachers’ experiences earning digital badges
0–5 Years (Lucy, (Deborah, and stamps, using the online badging system to support
Annmarie) Cindy) learning, and how the overall TLJ experience connected to
their educational institution.
1 teacher 2 teachers
6–15 Years As we read the 11 teachers’ goal setting statements, badge
(Nancy) (Lily, Zoe)
activity logs, and interview transcripts, we applied codes
for personalization, collaboration, and authenticity. First,
2 teachers 2 teachers
the coded excerpts were examined within each case (e.g.,
15+ Years (Erin, Hazel) (Barbara, Sally) one teacher’s experiences). Second, the coded segments
were compared and contrasted within an Excel spreadsheet
to understand patterns within the badging experience of the
Third, at the most fine-grained level, we sought to elucidate 11 focal teachers. We used the analysis of the 11 teachers
learners’ complex interactions with the badges, stamps, and to strengthen our coding book and to develop preliminary
the badging system within TLJ. To conduct this in-depth analytical insights before moving to the third step of the
analysis, two teachers were selected — one as an example analysis. In the third analysis step, we examined the logs
of average engagement in TLJ and the other as an example from earning badges and stamps and goal statements from
of high-level engagement in TLJ. These two teachers were all participating 36 teachers. By looking at the full dataset,
strategically sampled as the typical user and the extreme we were then able to find examples and counter examples
user. The first participant, Barbara, received a total number of our themes. We then turned to the activity surveys and
of stamps and badges similar to the year one average. The summative surveys to compare the trends from the
second participant, Sally, completed more than double the teachers’ logs and reflections to survey responses.
average teacher. In this way, our examination includes both Throughout the analysis process, we relied most heavily on
a typical learner (Barbara) and a heavily involved learner the badging activity logs and the goal setting reflections.
(Sally, see Figure 3). Both Barbara and Sally agreed to However, the interview data and survey responses allowed
participate in year one as part of their summer professional us to triangulate our findings and to ensure we understood
learning and expressed enthusiasm when getting started. the teachers’ experiences from their perspectives.
They were selected in order to focus on two people with
similar backgrounds, yet they experienced different 5. FINDINGS
learning journeys given their interests, expertise, and In this section, we answer our research question about the
personal goals as described in Section 4.4. kinds of interactions learners had with the elements of the
TLJ badging system. Overall, the learners, who in this
study are teachers, reported an estimated average of 13.4
hours (range 2 to 40 hours) of engagement with TLJ
activities during the summer. We first describe the way that
the full group of 36 teachers used TLJ’s various
components, with explanatory details from the 11 interview
participants, especially the focal case study educators:
Barbara and Sally.
Figure 3. Case study participant Sally’s stamps and 5.1 Learning Organized by the Itinerary
badges. Each participant’s journey began with writing a personal
learning goal statement, which allowed support for teachers
in their personalized learning experience. In particular, it
4.4 Data Analysis helped in identifying the badged activities they would want
The team conducted an analysis of the TLJ experiences of to complete over the summer and add them to a personal
the 36 teachers that participated within the year one itinerary. As Table 2 shows, learners identified over half of
implementation, which was run from June to August 2012. the activities that they wanted to complete at the start of the
Our analysis focuses on understanding learners’ experience, yet completed very few. Across the 36
experiences within the TLJ badging system and how these teachers, 221 activities were picked during the first few
experiences could be useful to interactions outside the weeks of the program, but only 11 were completed. Time at
the beginning was spent exploring and planning for the PD they needed in order to meet their personal goals. By
options within the badging system, rather than earning examining Barbara and Sally’s goals, selections, and
stamps or badges. This supports the motivating learning completed PD, we found that their focus was narrow when
principle to set goals described by Hickey and colleagues compared to other teachers that completed at least one
[11]. During year one, teachers were able to consider their stamp. From these findings, we can tell that support may be
goals through drafting a focused statement and then needed to help teachers in identifying focused and
creating a learning trajectory [11] for themselves by adding manageable learning goals.
activities to their itinerary. Table 2 also demonstrates
fluctuations in the engagement in PD over the time of the 5.2 Choosing a Stamp or a Badge
first year of TLJ. Personalization was also clear when analyzing the topics
that learners selected. Our earlier work found that teachers
Table 2. Stamps and badges completed versus the most often sought the entry-level stamp as micro-
potential activities the teachers selected credential, rather than a badge [9]: the 36 participating
teachers earned 133 stamps (86.4%) and 21 badges (13.6%)
June July August Total across the three STEM content areas. We posit that by
Selected 221 176 29 426 providing teachers the ability to make decisions about their
All 36
assessment (high or low levels of mastery) provided them
Teachers Completed 11 109 34 154 the flexibility to personalize their PD to align with their
existing and desired expertise teachers needed for
Selected 4 3 0 7 successful teaching. The total in Table 2 is compared to
Sally Barbara and Sally’s experiences with TLJ and demonstrates
Completed 1 8 0 9 a wide discrepancy in selected activities to do and those
completed. While Barbara and Sally completed nearly as
many activities as they selected for their itinerary, this is
Selected 5 2 0 7
Barbara not the case with the overall 36 teachers. The full group
Completed 2 3 0 5 demonstrated that the teachers selected many more than
they were able to accomplish in the summer of year one.
This may provide additional reasoning for creating leveled
As evidenced by Sally and Barbara, setting more specific [11] badges. By offering lower-stakes or smaller
and attainable goals may support teacher focus while granularity badges, learners may be able to select and
exploring, selecting, and completing PD activities. Barbara complete more of the badges that they’ve identified as
focused her goals on engaging her specific class and grade, relevant to their learning.
stating that she wanted “to learn new and innovative
approaches to teaching the solar system to my sixth grade Personalization was not just in the topic area but in the
students which will allow them to become more interested approach to PD. Barbara and Sally each worked
and excited.” The PD that Barbara completed for stamp- exclusively on one topic. Barbara worked on solar system
level recognition was mostly centered on using technology activities and began by exploring PD related to a general
in the classroom. In her initial interview, Barbara’s understanding of planets, next focused specifically on
emphasis on finding innovative teaching tools was Mercury for three activities, and then returned to more
synonymous with using technology in the classroom. In the general solar system PD again. Sally chose only weather
end, Barbara completed five Solar System stamps. and climate activities. Her selection was similar to
Barbara’s where she would vary her content focus. Sally
Sally wanted to focus on increasing her personal chose PD activities with a fairly consistent alternating
knowledge to “assist my students in understanding of how pattern between weather and climate content. Sally reported
the cycling of matter in and out of the atmosphere relates.” she was initially not enthusiastic about teaching weather
Many other educators, like Sally, used their goal statements and climate science in class, but she felt strongly about
to demonstrate an interest in helping their pupils to finding activities within the TLJ that would energize her
improve. However, Sally’s goal was more specific with and support her to feel more prepared to work with her
regard to content and improving her knowledge. It is clear middle school classes. Barbara also expressed interest in
to see that Sally’s learning goal was exactly what she finding solar system PD activities in order to help her
pursued in TLJ content. All of the PD that she received prepare for her lessons in the upcoming school year.
recognition for was in the category of weather and climate,
which aligned with her goals. At the end of year one, Sally 5.3 Activity Logs
completed six Weather & Climate stamps and three To earn a badge or a stamp, teachers had to write and
Weather & Climate badges. submit an activity log. The teachers’ activity logs consisted
of their explanations of how, when, and where the TLJ
During their post-interview, both Sally and Barbara badging STEM materials will be used to enhance pupils’
commented that they saw value in setting purpose learning in their university, school, or museum. For these
statements because it helped them think critically about the
teachers, the TLJ content not only filled a gap in STEM provide opportunities for her pupils to discuss in class other
resources as discussed above, but TLJ activities were used topics, such as climate change and land formations.
to transform current teaching practices. Teachers discussed
how the TLJ material could be used to meet specific needs 5.4 Mentor Feedback on Activity Logs
of their pupils. Nancy earned six Solar System stamps and In order for a teacher to receive recognition for completing
one Weather & Climate stamp. In her activity logs, she a PD activity, a reflective log was submitted for review.
wrote about the unique learning needs of individual pupils These required logs for completed PD also provided an
and used TLJ resources to help those diverse pupils enjoy opportunity to make professional learning an authentic
interactive activities. For example, Nancy commented, learning experience. Sally made use of her reflections as
“[TLJ] helped me absorb the information….If it was useful write up for other colleagues in order to share the resources
for me, then it would be useful for learners in classroom she found. Barbara reiterated the value of the reflections as
environments.” Nancy, like many other teachers in our a reminder for what worked and what did not for her class.
dataset, was not only looking for quality content in the Both Sally and Barbara made use of the logs to make direct
badging activities for her own learning, but in her activity applications to their own teaching.
log, she carefully considered how the delivery and The NASA education experts within TLJ that acted as
activities of the STEM content would work in the diversity mentors frequently provided feedback to learners related to
of her class. the submitted logs for the badging activities. This feedback
Erin was one of the most robust TLJ users; she completed often included suggestions related to the topic, common
eight Engineering stamps; four Engineering badges, two STEM misconceptions they or their pupils may face, and
Weather & Climate stamps, one Weather & Climate badge, pedagogical approaches appropriate to the STEM content
and one Solar System stamp. In her activity logs, Erin presented. Barbara described her experience with the
described how she could supplement the TLJ materials to feedback as offering “really great suggestions.” This
bridge the gap for pupils who had little exposure to the dedicated support from these expert educators supports
content or to challenge more advanced pupils. To earn her TLJ’s effort to enhance the credibility of the badging
seven stamps, Nancy wrote in her activity logs how she activities with experts employed [11] with respect to the
would adapt the TLJ content for the younger elementary evaluation of logs and the awarding of badges and stamps.
pupils in her classroom. Specifically, Nancy developed 5.5 Collaboration during PD Activities and
lesson plans that incorporated the detailed NASA images
and simulations in her teaching. Nancy also reflected on after PD Activities were Completed
how she would use the materials from the badging system Discussions were supported through the use of TLJ
to engage her pupils more fully in the STEM content. She webinars, which allowed the teachers to communicate with
wrote, “students who are not highly motivated in science or other teachers and with teacher educators. This
mathematics and students who are not challenged enough communication was short-duration (1–1.5 hours) and done
through the use of engineering processes would enjoy the over an internal chat program that was available to all
interactive activities if they are implemented and facilitated participants. However, it still provided benefits for teachers
effectively.” that did not know each other prior to the webinar. For
example, in her post-interview Barbara describes her
Through the TLJ activity logs, the teachers often wrote interest in having the opportunity to discuss what might
about the topic of inclusion and of a student-centered work in their classrooms as evidenced by forming teacher
approach. Zoe described herself as a learner who enjoys cohorts in order to provide an on-going channel for the
hands-on and visually rich instruction in her logs. Zoe exchange of dialogue and artifacts such as lesson plans,
completed two Solar System stamps. In her log, she wrote student collected data, and feedback. Cohorts within TLJ
that she liked “how all the activities presented are hands-on could provide teachers with long-term opportunities for
and visual to reach a variety of learners. If the students are discussion and collaboration. Long-term benefits for
able to see it and manipulate it they are more likely to learn teachers could include a chance for sharing successes,
it.” Barbara wrote that an activity was just “okay” for her, identifying ways to improve, and developing shared lesson
but then after trying to “put myself in my 6th grade plans. Short discussions provided benefits for teachers as
students’ shoes and see how they would respond,” she well, such as learning how each teacher approached a
concluded that with the help of some supplemental the lesson differently, but could also lead into a longer term
activity would be “a great listening activity.” engagement. For example, in one of her logs Erin writes
Lily used her logs to reflect on how she would use projects about attending an engineering webinar, describing how
and classroom discussions in her upcoming school year. each teacher took a turn in discussing the activity, and then
Lily wrote logs to earn one Solar System stamp, two how she later connected with some of these teachers
Engineering stamps, three Weather & Climate stamps, and through another online resource for STEM educators.
one Weather & Climate badge. She intended to use the While the full TLJ analysis of the 36 teachers showed
information that she learned within the badging system to evidence that all the teachers valued the collaboration.
When surveyed about future social and collaborative
functionality for TLJ, 67% of the survey participants —the 5.6 Relevance of Badging Activities to Life
second most popular response—agreed with the statement: Outside the Badging System
“I would like to be part of a cohort of teachers similar to The PD that TLJ provided allowed for personalization that
me that I could work within online workshops and other resulted in reports of relevance and authentic practice.
activities.” More specifically, the cases of Barbara and Teachers were given the opportunity to submit feedback for
Sally illustrate the nuance in the existing opportunities to each completed activity and among the 76 responses, 96%
collaborate via synchronous or asynchronous conversations either agreed or strongly agreed to the statement “this
with other teachers. For example, Sally stated that while activity is relevant to my teaching.” Across all the teachers’
she did participate in discussions, she was equally happy responses to the post-survey, 93% agreed that the TLJ
being an observer of the other teachers’ exchanges. When experience influenced their teaching for the upcoming
asked about future collaborative opportunities, both school year. In addition to the PD content, the activities
teachers wanted more opportunities to engage in were also structured to require a reflection where teachers
discussions like those in the webinars. Barbara further in some way consider how they might apply what they have
elaborated that she would like to continue the PD activity learned back to their classroom.
discussion later with others that completed the activity in
order to share what worked and did not work when trying Within the case study analysis, Sally explained she was
to implement the lesson. While the analysis of data only able to find relevance with TLJ by being able to “hunt and
demonstrates relatively short duration collaboration, future choose” among all the PD activities to find what best meets
iterations of TLJ would benefit from badges designed to her needs as a middle school teacher. While Barbara shared
support opportunities for peer modeling similar to that the resources she found with her partner teacher to make
made available in the work of Ching and Hursh [6]. use of them in their lesson plans. Since starting to share
these resources, Barbara moved closer to making her TLJ
The collaborative aspects of TLJ reached into the teachers’ experiences situated within her district’s teaching
work with their pupils. The teachers reported incorporating, community. In addition to engaging smaller professional
or the intention to incorporate, materials from the TLJ communities, such as the school and district level, the
badging activities in their classroom to support the teachers in our study contributed to an educator online
development of collaborative skills in their pupils or with learning community that was supported by TLJ and can
their peers. These collaborative aspects were reported to be help to promote exchange between teachers from all across
included within pupils’ group projects, classroom the country. Through itineraries and logs, teachers are able
discussions, and on the teachers’ websites for use with to map their own learning trajectories [11] of past and
pupils and parents. Erin, as a leader in her own STEM future learning which may help learners to visualize their
school, needed enhanced PD that covered content beyond badge experiences and learning over time.
the standard science curriculum, given the special emphasis
on STEM in her workplace. Erin wrote in her logs how the For the teachers in our study, the earning of the actual
TLJ materials were already successfully incorporated into stamps and badges was secondary to the learning of the
training that she facilitated for other teachers during the STEM content area. For example, Barbara and Sally
summer that emphasized collaborative skills. In addition to described the stamps and badges as “nice” but “not
Erin, other teachers wrote in the activity logs that they necessary” respectively. These two teachers were interested
would share the resources from the TLJ activities with their in the content for their own personal goals, as well as
colleagues. professional goals. Barbara wrote: “the big value is in the
activities themselves.” This finding demonstrates that the
While designed to support both asynchronous, independent value of these awards of completion is currently not evident
learners, as well as groups of learners, these findings to the general educational community. Both collegial and
further support Hickey and colleagues’ recommendation to competitive benefits can be gained from improving a
promote collaboration [11]. Collaboration was an important teacher’s ability to showcase their accomplishments and in
theme among the teachers’ logs. Teachers wrote about the future, researchers and educators working on badging
collaborating with local universities (Lily and Erin), a need to make the value of badging clearer to school
principal (Erin), and other teachers (Anne Marie). Teachers districts, employers, policymakers and other opinion
appreciated the feedback provided in TLJ by their mentor, leaders. However, the overall opinion of badges and stamps
and with this support, teachers felt confident enough to take was favorable as evidenced in the post-survey (Figure 4).
on larger projects like science fairs (Lily), science When asked to select as many of the following forms or
academies, and leadership roles relating to their own recognition you would like to receive for recognition of
schools PD (Erin). As teachers engaged in a community of your work in TLJ, teachers ranked badges and stamps as
learners, it was most often through professional learning highly as continuing education credits (CEUs) and below
opportunities arranged with peers in their district or school printable portfolios and letters of recognition. In this
building, as exemplified by Erin’s experience. regard, future redesigns of TLJ may include a portfolio
with stamps, badges, and details describing the learning
goals of the PD. Portfolios like this can then be shared with
colleagues, administrators, parents of the children in their learning. Many wrote or spoke in their interviews that they
classroom, or even friends and family who are interested in found the direct application of TLJ materials to their
STEM topics. classrooms and customization of the activities through TLJ
logs meaningful parts of the badging experience. The
importance of making connections from the badging
materials to their own learning and that of their pupils was
especially compelling in the excerpts from Sally’s and
Barbara’s interviews and logs. Collaboration was a key
aspect of the TLJ badging system because the teachers
shared experiences and education resources with other TLJ
teachers, teachers in their educational institutions, and even
the pupils and parents that they served. The logs and
interview data showed that TLJ provided opportunities for
teacher dialog that was relevant to their work
environments. Importantly, our data showed that the
Figure 4. The types of recognition preferred by the 36 badging activities allowed the TLJ teachers to share lesson
TLJ research participants. plans in relation to how the engineering, solar system, and
weather and climate content could be made relevant to the
6. DISCUSSION lives of youth.
This analysis focused on the experiences of 36 teachers in
TLJ in regard to how TLJ badges and badging system 6.2 Methodological Implication for Studying
served as a cultural tool to support STEM learning in Badging Systems with Metadata
relation to dimensions of personalization, collaboration, As part of our research with badges [10], we developed
and authenticity (i.e., relevance). Our project was oriented techniques for studying the impact of the badging through
to understanding learners’ badge-earning trajectories as a the data and metadata contained in the badging system
lens into (a) the refinement of mobile learning theory that itself. Given that digital badging is an emerging technology
aims to support learners’ reflective activities and self- without many empirical guides to studying badging
expressed goals and (b) the development of technology- impacts, an important outcome of our work for the field of
enhanced learning practice through the distillation of technology-enhanced learning is to consider which forms
design principles applicable to badge issuers in improving of data were useful to our analysis. Most importantly, we
their educational programming and instructional design found that no one data source served our research needs.
decisions. To reflect our goal, this final section is divided The ability to triangulate the badging experience of learners
into three parts. The first subsection reflects on how the in TLJ with the multiple data sources (i.e., pre- and post-
activities supported or did not support learners’ reflection interviews, activity surveys, summative survey, goal-setting
and personal or professional goals obtainment. The second statements, and badging logs) helped to support how
subsection considers methodological implications for teachers used badges for STEM learning and PD. We also
studying learning with and of badges. The final section found that collecting data at the start, during, and at the end
returns to our orienting design principles [2], [3], [11], [13] of year one helped elucidate the learning journeys as
to advance the field of technology-enhanced learning teachers’ experiences with badging grew over time.
understandings of badges as cultural tools to mediate Collecting activity logs and activity surveys immediately
STEM learning. after each activity was completed was especially important
6.1 TLJ Activities in Support of Learners’ to attending to any redesign work that was needed for year
two and beyond.
Personal and Professional Goals
From our analysis of interviews, summative interviews, and We also found the following data was missing that would
badging activity logs, TLJ teachers found themselves in have helped our work. During year one of TLJ, we did not
decision-maker roles to personalize their STEM learning. collect the feedback from the mentors. This could have
TLJ teachers first selected their learning goals and then helped the research team to better understand how and if
they identified personally applicable badging activities that teachers were taking this feedback into account in badges
they believed best met their own goals. By providing completed afterward. Moreover, this study would also have
options to the teachers, the TLJ educators explored STEM benefited from the collection of data related to
content at depths and breadths that matched their collaborative engagement between the learners and the
individualized expertise, teaching experience, and experts and among learners while engaging in more
classroom needs. For our interviews and activity logs, we synchronous webinar badge activities, including capturing
know that this is important to not only their professional chat logs and other forms of communication between
learning but to their personal satisfaction with the TLJ teachers. This would have helped the research team to
badging program. All the teachers wrote that they found explore how learners supported the learning of their peers
benefits in making choices to personalize their STEM
through a more in-depth focus on collaboration and the role in the PD activities. Given the success and the popularity of
of conversations to support learning. these prompts, this structure could be repeated by TLJ in
other aspects of the design. For instance, specific questions
6.3 Implications for Future Design Work could be asked as part of the reflection logs that were
While we had a relatively small dataset of 36 teachers, our required to be submitted to earn a stamp. More clearly
work is suggestive of several design principles for digital incorporating specific prompts and supporting framework
badges that can be refined through future data collection into the structure of each badge or stamp activity, we posit,
and analysis. First, the teachers in our full sample (and 11 could help to support teachers to reflect on certain aspects
focal teachers) personalized their badging experience in of their learning or future use of what they have learned
two key ways: selecting specific badging activities and then thus making their experience more authentic. Being able to
choosing an appropriate level of assessment (stamps, focus on a few parts of their learning may help to provide
badges, or both) for their own expertise, goals, and an even more favorable experience with required
classroom needs. This aspect of the design was highly reflections by all teachers.
valued by our research participants across multiple data
sources. In order to support additional personalization in 7. CONCLUSION
future iterations, we will continue to use goal-setting In this paper, we explored the utility of combining
purpose statements in the initial goal-setting section of our perspectives on technology-enhanced learning, cultural
work as well as at least two levels of assessment. Our tools, and mobile learning frameworks to design and study
findings from some teachers suggest that goal setting could digital badges with K-12 teachers. From our analysis of the
be improved by providing examples to help guide the initial experiences of the teachers that participated in the project’s
creation of goals that are attainable, specific, and focused first year, we suggest that future badging systems collect
and possibly by encouraging teachers to rework their goal multiple forms of feedback (or data) from participants to
statement after a certain number of badges were earned. As support meaningful understanding of the badging system.
shown in Table 2, the teachers selected many more We also found that including choices in the types of
activities than they could have completed in the time that content, the types of assessment, and the pathway sought
they allotted for themselves over the summer. Also, the was important to our participants in terms of
teachers selected many more activities than they could have acknowledging their expertise, interests, and classroom
completed in the time that they allotted for themselves over needs. These finding support the following design
the summer. To address this finding, rather than asking principles: (a) two levels of assessment (i.e., stamps and
learners to rework their goal-setting statements after a badges) can support personalized learning, (b) mastery of
certain number of badges are obtained, the system could be learning can be demonstrated and assessed through
designed to automatically send out reminders asking reflective logs completed to earn a badge or stamp, (c)
learners to revisit their set goals at specific periods. collaboration during and after badging activities (with
colleagues within the badging system and outside of it) can
Given that many teachers, including Sally and Barbara,
provide value to the learners, and (d) establishment of
expressed interest in collaborating with others as they
relevance of badging experiences to professional practice
earned badges, more long-term collaborative opportunities
can support the application of content outside the badging
can be designed into the content in our next design
system after the recognitions are earned.
iteration. For instance, creating cohorts or other groupings
could provide people with the ability to discuss their 8. ACKNOWLEDGMENTS
badging learning experiences. This cohort model would The development of the TLJ system and this research
allow learners to exchange information such as data project were partially supported by the NASA Aerospace
collected by their classroom and pedagogical approaches Education Services Project and Penn State University’s
that have worked in a particular lesson. These cohorts could Center for Online Innovation in Learning. The authors
be set at the start of the experience or after so many badges express appreciation to the participating teachers for
are earned to create groupings of people with similar participating in our research project. We acknowledge team
STEM content experiences. These collaborations might members Kyle Peck and Jaclyn Dudek for their
also be supported by using the TLJ system to allow contribution to the development of the TLJ system and for
teachers to create their own groupings or to match their scholarly conversations about badges as pedagogical
geographically located teachers together. tools.
Finally, the logs were identified by most teachers as helpful 9. REFERENCES
to connect the STEM learning within the badging system to [1] Abramovich, S., Schunn, C. and Higashi, R. M. 2013.
their educational institution. This suggests there may be Are badges useful in education?: It depends upon the
benefits from incorporating even more supportive type of badge and expertise of learner. Educational
structures into the badging system related to the activity Technology Research and Development, (Mar. 2013)
logging required to earn a stamp or badge. For example, 61, 2, 217–232.
within the collaborative PD webinars, reflective questions
were posed to all the teachers to support their engagement
[2] Ahn, J., Pellicone, A., and Butler, B. S. 2014. Open [12] Jakobsson, A., & Davidsson, E. 2012. Using
badges for education: what are the implications at the sociocultural frameworks to understand the
intersection of open systems and badging?. (2014) significance of interactions at science and technology
Research in Learning Technology, 22, 23563 - centers and museums. In (E. Davidsson & A.
http://dx.doi.org/10.3402/rlt.v22.23563 Jakobsson, Eds.) Understanding Interactions at
Science Centers and Museums (pp. 3-21). Rotterdam,
[3] Antin, J., and Churchill, E. F. 2011. Badges in social
NL: SensePublishers.
media: A social psychological perspective. CHI 2011
Gamification Workshop Proceedings. [13] Jovanovic, J. and Devedzic, V. 2014. Open badges:
http://uxscientist.com/?sort=post_date&page=7 Challenges and opportunities. Advances in Web-Based
Learning–ICWL 2014. Springer International
[4] Barron, L. et al. 1991. Technology and the design of
Publishing, 56-65.
generative learning environments. Educational
Technology. (1991) 31, 5, 34–40. [14] Kearney, M., Schuck, S., Burden, K., and Aubusson, P.
2012. Viewing mobile learning from a pedagogical
[5] Borko, H. 2004. Professional development and teacher
perspective. Research in Learning Technology. (Feb.
learning: Mapping the terrain. Educational Researcher.
2012), 20, 1063519, 1–17.
33, 8, 3–15.
[15] Pachler, N., Bachmair, B., and Cook, J. 2010. Mobile
[6] Ching, C. C. and Hursh, A. W. 2014. Peer modeling
devices as resources for learning: Adoption trends,
and innovation adoption among teachers in online
characteristics, constraints and challenges. Mobile
professional development. Computers and Education.
Learning. N. Pachler et al., eds. Springer US. 73–93.
73, 72–82.
[16] Stake, R.E. 1995. The Art of Case Study Research.
[7] Cole, M. 1998. Cultural Psychology: A Once and
Sage Publications.
Future Discipline. Harvard University Press.
[17] Traxler, J. 2009. Learning in a mobile age.
[8] Cucchiara, S., Giglio, A., Persico, D., and Raffaghelli,
International Journal. 1, (March 2009), 1–12.
J. E. 2014. Supporting self-regulated learning through
digital badges: A case study. New Horizons in Web [18] Vygotsky, L.S. 1978. Mind in Society: The
Based Learning (pp. 133-142). Springer International Development of Higher Psychological Processes.
Publishing. Harvard University Press.
[9] Gamrat, C., Zimmerman, H. T., Dudek, J., and Peck, [19] Wenger, E. 1998. Communities of Practice: Learning,
K. 2014. Personalized workplace learning: An Meaning and Identity. Cambridge University Press.
exploratory study on digital badging within a teacher
[20] Zimmerman, H. T., Gamrat, C., and Hooper, S. 2014.
professional development program. British Journal of
Digital postcards from summer camp. TechTrends. 58,
Educational Technology. (Nov. 2014), 45, 6, 1136–
2 (March/Apr. 2014), 87–92.
1148.
[21] Zimmerman, H.T. Land, S. M., McClain, L. R.,
[10] Halavais, A. M. C. 2012. A geneaology of badges.
Mohney, M. R., Choi, G. W., and Salman, F. H. 2013.
Information, Communication & Society. (Apr. 2012),
Tree Investigators : Supporting families’ scientific talk
15, 3, 354–373.
in an arboretum with mobile computers. International
[11] Hickey, D. T., Otto, N., Itow, R., Schenke, K., Tran, C. Journal of Science Education, Part B. 5, 1 (Oct. 2013),
and Chow, C. 2014. Research design principles for 44–67.
studying learning with digital badges. (2014). Badges
[22] Zimmerman, H. T. and Weible, J. L. In press. Learning
Design Principles Documentation Project. Indiana
in and about rural places: Connections and tensions
University Center for Research on Learning and
between students’ everyday experiences and
Technology.
environmental quality issues in their community.
Cultural Studies of Science Education.