The workshop preserved the general OBIE focus on opportunities and challenges associated with Open Badges, and the overall objective of connecting educators, researchers, and entrepreneurs in discourses on teaching, learning, assessment, digital credentials, and digital education in general. Still, considering the context where it took place, namely the major international conference on learning analytics, this 2nd installment in the OBIE series was primarily intended for those interested in the intersection of Open Badges and Learning Analytics. This intersection includes collection, integration, and analysis of data and resources associated with Open Badges, with the ultimate objective of supporting educators, learners and other stakeholders in the ever-increasing Open Badges ecosystem through the provision of informative and relevant feedback, predictive capabilities, and eventually a better understanding of the factors underlying educational processes and outcomes.
The workshop accepted position papers and demonstration abstracts, short papers, and longer technical papers. Each submission was reviewed by at least two (often three) members of the Program Committee. After the peer review process, five out of seven submissions were accepted for presentation at the OBIE 2015 workshop and inclusion in the workshop proceedings.
The Program Committee consisted of well-recognized researchers in the areas of Open Badges and Learning Analytics, as well as associated areas of Alternative Assessment, Participatory Learning, Networked Learning, and Instructional Scaffolding. In particular, we were 1 https://sites.google.com/site/obie2015ws/ 2 http://lak15.solaresearch.org/ 3 https://sites.google.com/site/obie2014ws/ pleased to have the following researchers as members of the Program Committee: • • • • • • • • • • • • • •
The OBIE 2015 workshop was organized as a half-day event. It started with a keynote talk entitled Whom Will Digital Badges Empower? Sociological Perspectives on Digital Badges [1], by Dr. Michael Olneck, Professor Emeritus of Educational Policy Studies and Sociology, University of Wisconsin-Madison. After a lively discussion that ensued from Prof. Olneck’s talk, the workshop continued with presentations of accepted papers. This session started with papers focusing on the experiences and lessons learned from individual badging projects, and moved towards papers that were broader in scope and offered more general view on Open Badges, badge system design, and design principles for studying learning with digital badges. The final part of the event was a discussion session during which the workshop attendees were joined, via an online communication channel, by Carla Casilli4, one of the leading experts in the Open Badges community and recently-appointed Interim Executive Director of IMS Digital Credentialing at IMS Global.5
Why should a learner be motivated to earn an Open Badge? This question was an underlying theme in many of the OBIE papers and discussions. Intrinsic motivation in learning has been linked to greater conceptual learning and increased future interest in a given topic or skill. Despite
being an important element in understanding learner intentions, there have not been consistent metrics for motivation in prior learning analytics literature. Open Badges research also struggles with such measures, particularly as the badge itself may change the nature of the learners' motivation from intrinsic (internal interest) to extrinsic (external rewards). During the OBIE 2015 workshop, Michael Olneck [1] first raised this issue within the context of scarcity of credentials: Is the value of a credential inherently tied to its scarcity? For example, are medical degrees valuable because they are scarce, or because there is an intrinsically motivated demand? Olneck countered that competency-based education, enriched with digital badges, has the potential to democratize this model by making the skills and knowledge areas that comprise today's credentials more explicit. For example, instead of having a single medical degree, the learner could be presented with a series of anatomy, surgical, psychological, and other components that eventually culminate in a degree that is completely transparent to the learner, issuer, and external parties. Once the components of a credential are made explicit, the learner is empowered to take control of their own learning through the mechanism of Open Badges, according to Olneck. Willis, Quick, & Hickey [6] note, however, that if the control is superseded by unintended uses such as aggressive marketing or exploitation, then learners will be less motivated to share their Open Badges publicly. Overall, the notion of empowering learners through the mechanism of Open Badges has several potential implications for motivation in the future. The issue of whether Open Badges are motivating in and of themselves was another common theme throughout the workshop. Gamrat & Zimmerman [2], for example, found that Open Badges were not a strong motivator for teacher professional development and that the participants struggled to identify the relevance for Open Badges in the "real world," external to the learning environment. The motivational implications to the inherent value proposition are deeply intertwined. If the value of the Open Badge, an external motivator, is unclear to the learner, then he/she is potentially less motivated to pursue the activities associated with that micro-credential, even if the learner was intrinsically motivated to pursue that learning opportunity without the prospect of earning an Open Badge. Devedzic et al. [3] also mention the skepticism related to the value of Open Badges which was more pronounced in the master's level student population.
Looking across Open Badge research designs, McDaniel & Fanfarelli [4] note that both the context of the Open Badge issuer and the function of the badge itself can have implications for motivation. Paired with engagement and performance, motivation can have a significant impact on the number and quality of badge applications and awards. Hickey & Willis [5] confirm this approach, noting that motivating learners is one of four common functions of Open Badges (along with recognizing, assessing, and studying). Open Badge implementations are generally more successful when the value propositions are clear and implicit for the learners. Open Badges have the potential for dramatically increasing learners' motivation – understanding and assessing that motivation will be one of the key challenges for the Open Badges and learning analytics communities to address in future work. One of the original intents of holding the second iteration of this workshop as part of the LAK conference was to gain insight at the intersection of learning analytics and Open Badges from leading researchers. The relative infancy of the Open Badges ecosystem, however, is a limiting factor for comprehensive findings at the junction of these two areas of interest. Despite this limitation, the workshop did highlight several potential avenues for future investigation, as well as proofs of concept.
Workshop attendee Simon Buckingham Shum6, Vice President of the Society for Learning Analytics Research (SOLAR)7, stated that he thought Open Badges presented an opportunity to experiment with thresholds for learning outcomes and the badge could contain relevant evidence. For example, the badge could contain information about how many different kinds of assignments and learning outcomes the badge represents, as well as linking to the learners' final assignment. Badges awarded or semiawarded based on analytics (e.g., the recipient is a central node in network X) is another possible avenue for researchers to investigate.
Analytics on badge archives and repositories, similar to analytics on digital learning object repositories, was also a potential area for future study mentioned by Shum. Willis et al. [6], however, point out that analytics tied to Open Badges expose a range of ethical and privacy questions that warrant consideration. Because of the inherent depth and detail of the data stored within Open Badges, learners are potentially more at risk for nefarious uses of their public data than previously available records of educational data. Because of the power in the badge designer to require particular types of learning evidence (e.g., a reflection), the learner could be exposed to targeted marketing, threats to job security, or other negative outcomes depending on when, where, and how the badge is shared on the Internet. Such issues are counterbalanced against the tremendous potential for unpacking the learning processes and related outcomes supported by the Open Badge standard. How learners process, weigh, and navigate these issues will be an important area of research as the Open Badge ecosystem grows, and will likely be applicable to the broader conversations related to ethics and learning analytics. In terms of relevant findings from Open Badge projects related to learning analytics, Gamrat & Zimmerman [2] 6 http://simon.buckinghamshum.net/ 7 http://solaresearch.org/ note that the utility of badges served different reporting purposes. Administrators used badges to understand how many pre-service teachers were participating, and what specific activities they were engaging in. Learners, however, used the badges as a tracking mechanism as well as a venue for them to store notes for further reflection and practice. As tools are developed to aggregate, summarize, and distill Open Badges, it will be an important design consideration to understand how best to utilize the analytics deeply embedded within these digital assets. Could an Open Badges dashboard, for example, highlight skills and competencies along a learners' personal pathway and help scaffold key insights into other learners' progressions and opportunities that are relevant for individual goals and interests?
Research on Open Badges and findings related to learning analytics are still at an early stage. The OBIE workshop presented a breadth of perspectives, both narrow and broad, that served to provide a cogent context upon which future researchers can further explore, apply existing learning theories to these new contexts, and develop analyticspowered navigation displays to support learners' growth and exploration. The evidence and supporting metadata included in the Open Badges standard present an opportunity for deeply understanding learners' learning processes and progressions across a multiplicity of contexts. As demonstrated by the papers, keynote, and conversation throughout the OBIE workshop, this opportunity space deserves critical investigation and carefully designed experiments to more fully understand how to realize this potential.
The proceedings consist of five papers accepted for the workshop through the peer-review process, and an invited paper prepared by Prof. Michael Olneck based on the keynote talk he gave during the event. In what follows, we give a brief overview of the OBIE 2015 papers.
The invited paper Whom will Digital Badges Empower? Sociological Perspectives on Digital Badges [1] relies on the sociological theory and research to identify and discuss constraints for the realization of the claimed benefits of digital badges. In particular, Olneck recognizes and provides argumentation for several sources of constraints, including the way credentials operate in labor markets and in the organization of work, the enduring power of conventional education forms, and the contradictory position of companies in the education field. By pointing out the challenges and enabling a comprehension of their origins and nature, this work provides the necessary grounds for contemplating actions to be taken to overcome the identified obstacles and realize the full potential of digital badges.
The next two papers report on the practical application of digital badges by presenting the authors’ experiences in using badges to motivate, grade, and recognize learning. In their paper An Online Badging System Supporting Educators’ STEM Learning [2], Gamrat and Zimmerman investigate how a digital badging system was used as part of an informal, not-for-credit professional development project. The authors used learners’ badging artifacts and interviews to study learners’ interaction with elements of the digital badging system including the goal statements, logs, materials submitted to earn a badge, and their mentor. The reported study findings led to some design principles to be considered when designing personalized learning supported by digital badges. In the second paper on practical experiences with a badging system, Grading Soft Skills with Open Badges [3], Devedzic et al. report on the use of badges for acknowledging, grading, awarding and recognizing learners' efforts and achievements in developing their soft skills (e.g., collaboration, skilled communication, problem-solving and innovation). The authors propose a model, called SAGRADA, as the conceptual framework for supporting and awarding learners with badges. Particular attention is given to the core of this model, namely a set of metrics for measuring and tracking learners’ efforts in developing their soft skills. The paper also presents a specific application case based on the SAGRADA model, including the preliminary results of the first observational study, and plans for applying learning analytics methods and techniques to further support learners in the development of their soft skills.
The next three papers provide a more general and comprehensive view on Open Badges, badge system design, and research design for studying learning with digital badges. The first in this group is the paper, How to Design Experimental Research Studies around Digital Badges [4], by McDaniel and Fanfarelli. Aiming to assist researchers who are eager to contribute, but new to the field of Open Badges, the authors outline an approach to designing research studies around digital badges. They offer suggestions for all-important elements of the research process, including literature review, research questions, study design, data analysis, and the like. They also point to the challenges that are specific to this particular area of research, and illustrate them through multiple examples. The next paper in this group, Research Designs for Studying Individual and Collaborative Learning with Digital Badges [5], presents a model for studying learning through digital badges. Hickey and Willis introduce six research designs used to study learning in badges development projects funded in 2012. The principles discussed distinguish between summative, formative, and transformative research, and between using conventional forms of evidence and using the evidence contained in digital badges. In the final paper, Digital Badges and Ethics: The Uses of Individual Learning Data in Social Contexts [6], Willis, Quick, and Hickey argue that publiclyavailable learning data in digital badges poses ethical questions that ought to be considered. They further explore the intersection of how learning analytics processes and digital badge data suggest three separate, yet interrelated, ethical questions for consideration. These questions address the ethical principles of human autonomy, freedom, and determinism.
The efforts put in the organization of this workshop and other associated activities were partially supported by grants from the MacArthur Foundation’s Digital Media and Learning Initiative. Partial support also came from the GRASS project (ref. num.: 543029-LLP-1-2013-1-RSKA3-KA3MP) funded by the European Commission. We would like to thank to the LAK’15 Chairs and Program Committee for hosting this event. Last, but not the least important, our special thanks go to the members of the workshop Program Committee, as well as all the members of the Open Badges community who supported this event.