Continuously attracting contributors is key to the health of open source software projects. The appearance of badges in online collaborative development platforms a ords maintainers the opportunity to advertise the quality of their project to potential contributors. In this preliminary research, we analyse 14,592 GitHub package repositories for Cargo and 203,029 repositories for Packagist. We measure how prevalent badges are in those repositories, which badges are used, when and how they are introduced, and which combinations of badges co-occur. We nd that the most widespread badges convey static information or relay information about the build status of a project. Those badges are typically added early in projects and prior to or at the same time as other badges.
The rise of distributed collaborative development platforms, such as GitHub, BitBucket and GitLab, allowed thousands of people to remotely work together on the same projects. These platforms provide additional features on top of their underlying version control system to further support distributed collaborative development. Examples of such features are issue tracking, code review, integration with external tools, etc. These features are usually provided through a centralised web-based graphical interface that acts as a portal to showcase a project. Given the variety and quantity of information that can be communicated through such interfaces, it is not surprising that project maintainers have sought a simpler, faster and more concise way to communicate essential information or advertise speci c aspects of a software project.
Badges are small images conveying one speci c information to the reader at a glance. We found evidence of their use in GitHub dating back to 2011. They typically appear at the top of a project's README le, which is displayed by GitHub on the project repository homepage. Badges can advertise various aspects of a project, e.g., its license , the code coverage of its test suite , the adopted code style , etc. Some badges act as an incentive to maintain excellence in the qualities they display, lest a bad signal be sent to the project's users and potential contributors. For instance, a code coverage badge creates an incentive to maintain a high code coverage, as otherwise, potential contributors can easily see that the project is poorly tested and, therefore, prone to have hard to detect bugs. Similarly, a badge that measures the quality of the code provides an incentive to maintain a high code quality, to not display on the project's homepage that the code base is of poor quality.
Trockman et al [
As contributors in open source projects are
volunteers, they typically have little time to devote to the
projects they contribute to, so they must select such
projects with care as they can ill a ord to contribute to
projects of low quality, which are harder to contribute
to and more likely to fail [2{4]. On their part, project
integrators need to maintain an in ux of contributions
in order to keep their projects evolving and growing, as
it has long been known that software performing
realworld activities must continually grow and continually
change to adapt to the environment it evolves in [
It is known that the number of stars, the time taken
to merge pull requests and number of programming
languages are the factors most likely to attract
contributors [
Our overall research goal is to investigate the
relation between the presence of badges and the in ux of
new contributors and new contributions in a project,
as well as on the health of projects. It is known, for
instance, that the use of continuous integration tools
helps catch bugs more e ciently and integrate pull
requests faster [
To conduct this study, we need a large dataset of
candidate repositories hosted on online collaborative
development platforms. As we are interested in
studying the e ect of badges on contributions, the dataset
should exclude git repositories created merely for
experimental or personal reasons, or that only show
sporadic traces of commit activity [
Online package registries of open source libraries for popular software programming languages constitute good candidates, since they contain a lot of projects, many of them being publicly available on GitHub. We arbitrarily selected two such package registries because we knew from previous work that many of their packages have an associated git repository publicly available on GitHub. These registries are Cargo for Rust libraries, and Packagist for PHP libraries.
We collected a list of 15,625 packages on Cargo and 216,613 packages on Packagist using their respective o cial API. We downloaded package metadata for those packages and extracted the link to their associated git repositories. We ltered packages (1) without an associated git repository; (2) whose git repository is no longer available; and (3) corresponding to \spam" packages1. Since GitHub hosts nearly all the git repositories of remaining packages (> 94%), and since it is far easier to deal with only a single collaborative development platform, we excluded repositories that were not available on GitHub. The nal dataset contains 14,592 package repositories for Cargo and 203,029 package repositories for Packagist.
We cloned all these repositories in July 2019 to
extract badge-related historical information. To
identify badges, we focused on images contained in the
projects' README les. We extracted all images from
these README les, taking into account the various
supported markup languages (HTML, Markdown and
ReST). We manually identi ed those corresponding to
badges following the iterative approach proposed by
Trockman et al. [
RQ0: How prevalent are badges? With this rst question, we aim to determine the extent of badge usage in project repositories. We identied for each of the two considered datasets the badges used by their projects. We found 21,884 instances of 109 distinct badges in Cargo projects, and 239,529 instances of 366 distinct badges in Packagist. While there are more badges than projects in both datasets, that does not necessarily imply that all projects use badges. Figure 1 shows the evolution of the number 1We manually identi ed more than 200 packages on Packagist that are not related to software projects, e.g., iphonex-giveaway, captain-marvel-pelicula-completa-uncut, etc. These spam packages are usually quickly removed from Packagist by the maintainers of the registry. and proportion of projects using at least one badge in Cargo and Packagist. For Cargo, we do not report before 2014 as this only concerns 90 projects out of which only 19 had a badge.
While there are far more projects with badges in Packagist than in Cargo, we observe a markedly higher badge penetration within Cargo (topping o at 57%) than within Packagist (the highest observed proportion is 36%). For both datasets, the adoption rate eventually reaches a plateau: even though the number of projects using badges keeps increasing more than linearly, it does not supersede the rate of creation of new projects. In both datasets, the most frequent badge is the one reporting the build status of the Travis continuous integration tool (30% and 20% of all badges used, respectively).
RQ1: What are the most frequent badge categories? RQ0 revealed that badges are widely used. However, not all projects use the same badges, and projects may use badges for a variety of di erent purposes. Furthermore, many badges ful l a similar role (e.g., several badges can be used to relay the build status of a project, based on di erent providers such as Travis and Appveyor).
Therefore, we grouped these badges into 7
categories, following the approach of Trockman et al [
In Cargo, the most frequent combination by far is the one containing Build and Info badges,
With RQ2 we found that most co-occurring badge categories correspond to badge instances introduced on the same day in a project. We now focus on when those badges were introduced in a project. For each badge category, we computed the proportion of projects with at least one badge making use of a badge of this category. Figure 3 shows the evolution of these proportions for both datasets. For Cargo, we observe a fast and somewhat massive adoption of Build badges: the proportion of projects using such badges went from 2% (September 2013) to 77% (August 2014). A similar observation can be made for Info badges to a lesser extent, going from 2% (January 2015) to 59% two years later. The situation is di erent for Packagist where many projects already existed before badges were available. Indeed, around 5% of projects using badges in Packagist were created before the availability of such badges. This proportion is only 2% for Cargo, which is not surprising given that Rust appeared in 2010. The adoption of badges in Packagist is therefore more gradual, going from practically 0% (June 2013) to 35-40% in two years; with the notable exception of Build badges, whose adoption occurred much faster. By August 2012, these badges had been adopted by 60% of the projects, a probable consequence of the introduction of Travis-CI in March 2011.
cargo
Since we found a non-trivial amount of co-occurring badge categories, in a second step, we examine how frequently badges belonging to di erent categories were added on the same calendar day in a project. Whenever a category is represented by several badges within a same project, the introduction date of the oldest badge is considered. Table 2 shows the proportion of co-occurring badge categories that were introduced simultaneously. We observe for Cargo that most co-occurring badge categories are adopted simultaneously. For Packagist, it mainly depends on the considered combination. For instance, a large majority of the combinations involving Build+Pop, Build+Info or Pop+Info corresponds to badges introduced on the same day. On the other hand, we observe that 62% (=100-38) of the combinations with Build+QA are not adopted simultaneously but in subsequent events. s1.0 t jce0.8 ro0.6 .foopp00..42 BQuAild IPnofpo r p0.0 2012 2013 2014 2015
packagis2t016 2017 2018 2019 s1.0 t jce0.8 ro0.6 p fo0.4 .op0.2 r p0.0 2012 2013 2014 2015 2016 2017 2018 2019
For each project and badge category, we measured the elapsed time before the rst introduction of a badge of a given category in a given project. Since some projects predate the availability of (services advertised by) badges, we measure this time with respect to the date of the rst opportunity to introduce those badges. So, if the project was created prior to the rst occurrence of a category in our dataset, then the date of this rst occurrence is used as a baseline. Otherwise, we relied on the creation date of the project as a baseline. Table 3 reports on the median and mean of these durations, aggregated by badge category. The left part of the table expresses these durations in days since the date of rst opportunity, while the right part expresses them proportionally to the opportunity window (i.e., time between the date of the rst opportunity and the last known commit of a project). The huge di erence between median and mean values suggests skewed distributions: while a majority of badges are quickly introduced in projects, there are some outliers taking a while to introduce badges. This is especially visible in Packagist: its median values are lower than the ones for Cargo but its mean values are much higher. We also observe that, on average, quality assurance badges were added much later in Cargo projects (median is 10 days vs. 5 for Packagist). In both datasets, Build badges are introduced earlier than other badges. 4
We carried out an empirical analysis of the usage of badges in GitHub repositories, with the ultimate goal of determining their impact on contributions to open source projects. As a preliminary step, we sought to determine whether badges were widely used in projects for two popular programming language library registries: Packagist and Cargo.
We found that they are used in more than a third
of Packagist projects and more than half of Cargo
projects, and that more and more projects tend to use
them. Still, badge adoption rates lag behind the rate
of appearance of new projects. Then, we categorised
badges in seven categories, according to the type of
information being relayed by each badge, and measured
the relative prevalence of each category. We observed
that Packagist projects use a more diverse set of badges
than Cargo, the latter mostly sticking to Build and
Info badges. We examined the frequency at which the
most common categories co-occurred within the same
projects, nding that Build badges and Info badges are
usually found together and that the other categories
of badges were rarely found without a corresponding
Build or Info badge. We also found that co-occurring
badges were frequently adopted simultaneously. We
next examined the temporal aspect of badge adoption
and found that the adoption rates of badge categories
were either increasing or stable. We also showed that
badges were usually added early on, within the rst 5%
of a project's lifetime, but still a signi cant amount of
projects adopt badges much later. The results we
obtained are in line with those of Trockman et al [
As future work, we intend to investigate the impact of badge adoption on contributions. In doing so, an aspect to take into account will be the comparative effort required to maintain some badges over others. We also will quantify the phenomenon of badge removal, determine the reasons why it occurs and what is the impact on contributions.