=Paper= {{Paper |id=None |storemode=property |title=Evidence for the Pareto principle in Open Source Software Activity |pdfUrl=https://ceur-ws.org/Vol-708/sqm2011-goeminne-mens-11-pareto.pdf |volume=Vol-708 }} ==Evidence for the Pareto principle in Open Source Software Activity== https://ceur-ws.org/Vol-708/sqm2011-goeminne-mens-11-pareto.pdf

Evidence for the Pareto principle
in Open Source Software Activity
Mathieu Goeminne and Tom Mens
Institut d’Informatique, Faculté des Sciences
Université de Mons – UMONS
Mons, Belgium
{ mathieu.goeminne | tom.mens }@umons.ac.be

Abstract—Numerous empirical studies analyse evolving open the metrics that we rely upon. Section III presents the ex-
source software (OSS) projects, and try to estimate the activity perimental setup of our empirical study that we have carried
and effort in these projects. Most of these studies, however, only out. Section IV presents the results of our analysis of activity
focus on a limited set of artefacts, being source code and defect
data. In our research, we extend the analysis by also taking into distribution in three OSS projects. Section V discusses the
account mailing list information. The main goal of this article evidence we found for the Pareto principle. Section VI presents
is to find evidence for the Pareto principle in this context, by related work, and Section VII concludes.
studying how the activity of developers and users involved in
OSS projects is distributed: it appears that most of the activity II. M ETHODOLOGY
is carried out by a small group of people. Following the GQM
paradigm, we provide evidence for this principle. We selected A. GQM paradigm
a range of metrics used in economy to measure inequality in To gain a deeper understanding of how OSS projects evolve,
distribution of wealth, and adapted these metrics to assess how we follow the well-known Goal-Question-Metric (GQM)
OSS project activity is distributed. Regardless of whether we
analyse version repositories, bug trackers, or mailing lists, and paradigm. Our main research Goal is to understand how ac-
for all three projects we studied, it turns out that the distribution tivity is distributed over the different stakeholders (developers
of activity is highly imbalanced. and users) involved in OSS projects. Once we have gained
Index Terms—software evolution, activity, software project, deeper insight in this issue, we will be able to exploit it to
data mining, empirical study, open source software, GQM, Pareto provide dedicated tool support to the OSS community, e.g.,
by helping newcomers to understand how the community is
I. I NTRODUCTION structured, by improving the way in which the community
members communicate and collaborate, by trying to reduce
Numerous empirical studies aim to understand and model the potential risk of the so-called bus factor1 , and so on.
how open source software (OSS) evolves over time [1]. In To reach the aforementioned research goal, we raise the
order to gain a deeper understanding of this evolution, it following research Questions:
is essential to study not only the software artefacts that
1) Is there a core group of OSS project members (develop-
evolve (e.g. source code, bug reports, and so on), but also
ers and/or users) that are significantly more active than
their interplay with the different project members (mainly
the other members?
developers and users) that communicate (e.g., via mailing lists)
2) How does the distribution of activity within an OSS
and collaborate in order to construct and evolve the software.
community evolve over time?
In this article, we wish to understand how activity is spread
3) Is there an overlap between the different types of activity
over the different members of an OSS project, and how this
(e.g., committing, mailing, submitting and changing bug
activity distribution evolves over time. Our hypothesis is that
reports) the community members contribute to?
the distribution of activity follows the Pareto principle, in the
4) How does the distribution of activity vary across differ-
sense that there is a small group of key persons that carry
ent OSS projects?
out most of the activity, regardless of the type of considered
activity. To verify this hypothesis, we carry out an empirical As a third step, we need to select appropriate Metrics that
study based on the GQM paradigm [2]. We rely on concepts will enable us to provide a satisfactory answer to each of the
borrowed from econometrics (the use of measurement in above research questions. For our empirical study, we will
economy), and apply them to the field of OSS evolution. make use of basic metrics to compute the activity of OSS
In particular, we apply indices that have been introduced project members, and aggregate metrics that allow us to com-
for measuring distribution (and inequality) of wealth, and pare these basic metric values across members (to understand
use them to measure the distribution of activity in software how activity is distributed), over time (to understand how they
development. 1 The bus factor refers to the total number of key persons (involved in the
The remainder of this paper is structured as follows. Sec- project) that would, if they were to be hit by a bus, lead the project into
tion II explains the methodology we followed and defines serious problems
evolve), and across projects (to compare the situation between The Hoover index, defined in Equation 3, is one of the
different OSS projects). simplest ways to assess inequality of wealth or income. Its
value is the ratio of incomes to take up from the richest
B. Basic metrics
part of the population to redistribute to the poorest one so
To obtain the basic metrics of OSS activity, we will extract that the incomes become perfectly equal. A Hoover index of
information from three different types of data sources we have 0 represents perfect equality, while a value of 1 represents
at our disposal: version repositories, mailing lists, and bug perfect inequality.
trackers. For each of these data sources, we can define metrics
that extract and reflect a particular type of activity: n
1 X xi 1
• Development activity: the activity of developers com-
H(X) = − (3)
2 i=1 xtotal n
mitting source code to a version repository, measured as
number of commits. The Gini index, defined in Equation 4, provides a more
• Mailing activity: the activity of project members posting complex (but also more representative) way to assess inequal-
messages to a mailing list, measured as number of mails. ity of income. The cumulative function of income distribution
• Bug tracker activity: the activity of persons interacting is represented by a perfect diagonal if all entities in the
with a bug tracker, measured in three different ways: population would have the same income, and by a curve under
number of new bug report submissions, number of com- the perfect line otherwise. The Gini index is the value of the
ments added to existing bug reports, number of changes surface area between the perfect line and the curve, divided
to existing bug reports. by the surface area below the perfect line.
Since we are not only interested in a static view of a  n 
particular snapshot of an OSS project at a particular moment X
ixi 
in time, we will extract each of the above activity metrics 2

i=1
 
during the entire life of the considered OSS projects. G(X) = 1 − · n− n
  (4)
n−1  X 
xi
 
C. Aggregate metrics
i=1
Since several of the research questions require a comparison
of the basic metrics (across persons, across projects, and Yet another index to assess inequality of income or wealth
over time), we need aggregate metrics that combine the basic is the Theil index. It is based on the Shannon entropy [5],
metrics. This is valuable, in particular, if we want to reason and is defined in Equation 5. Because the Theil index is not
about the distribution of activity across OSS project members. bounded a priori, one cannot easily compare it with the two
To study such distribution, we borrow ideas from econo- aforementioned indices. To normalize the Theil index so that
metrics. This discipline uses statistics and metrics to analyse it always returns a value between 0 and 1, we can apply the
economic data. As an example, various aggregation measures normalisation function N described in Equation 6 to it.
of statistical dispersion have been proposed (e.g., the Hoover, n
Gini, and Theil indices) and applied to assess the inequality of 1 X xi x
i
T (X) = · ln (5)
the wealth distribution among people, regions, countries, and n i=1 x̄ x̄
so on.
Recently, some of these aggregation measures have been
N : t → 1 − e−t (6)
used for analysing evolving software systems. Vasa et al. [3]
proposed to use the Gini index as an alternative to traditional III. E XPERIMENTAL SETUP
software metrics. Serebrenik et al. [4] proposed to use the
Theil index instead. Following this emerging trend, we will A. Implementation
use three different aggregation measures to study OSS activity In order to obtain replicable and verifiable results, we do
distribution. Below we provide the definitions of the three not only need a good methodology. At least as important is
aggregation measures we selected: the Hoover index, the Gini automated tool support that enables us to extract data from
index, and the Theil index. These definitions rely on two the different data sources, compute the basic and aggregate
auxiliary definitions. metrics based on this data, statistically analyse the obtained
Let X = {x1 , . . . , xn } be a set of values indexed in metrics, and visually confirm the results.
ascending order (∀i ∈ 1 . . . n − 1, xi ≤ xi+1 ). The sum of To this extent, we use and extend our generic framework for
all these values will be called xtotal (Equation 1). The mean analysing open source software projects. This framework, pre-
of all values will be called x̄ (Equation 2). sented in [6], has been developed in a modular and extensible
n
X way, facilitating the implementation of new modules meeting
xtotal = xi (1) specific needs. Support for automatic extraction of data from
i=1 version repositories, mailing lists and bug tracking systems
xtotal was already built-in. The framework also supports generation
x̄ = (2) and visualisation of various types of software (project) metrics.
n
OSS project Brasero Evince Wine
For the specific purposes of this article, we added a new
main programming language C C/C++ C
module to compute activity distribution (i.e. the relative activ- versioning system git svn git
ity of each involved person), and different variants of activity age (in years) 8 11 11
are supported. In particular, we implemented the three types size (in KLOC) 107 580 2001
# of commits 4100 4000 74500
of activity defined in Section II-B. We also added a module # of mails 460 1800 14000
for computing aggregation indices. Among others, the Hoover, # bug reports 250 950 3300
Gini, and Theil index are currently supported. The existing # commiters 206 204 1229
# mailers 102 610 6879
statistic analysis and visualisation modules can directly exploit # bug reporters 386 961 2676
the information computed by the activity distribution module
and the aggregation index module to produce statistical output TABLE I
representing the inequality indices. M AIN CHARACTERISTICS OF SELECTED OSS PROJECTS . T HE REPORTED
VALUES HAVE BEEN COMPUTED FOR THE LAST VERSION , N OVEMBER
2010.
B. Pareto principle
Many types of distributions in which people are involved
correspond to the so-called Pareto principle: roughly 80% of
1
the effects stem from approximately 20% of the causes [7].
0.9
This principle and the associated law have been observed
repeatedly in a variety of domains, including software evo- 0.8

lution [8], [9]. 0.7

Answering the first research question of Section II-A boils 0.6

down to finding empirical evidence for the Pareto principle 0.5
in OSS project activity distribution. One should note the
0.4
difference between the Pareto principle and the related notion
0.3
of Pareto distribution [10]. While a Pareto distribution satisfies
0.2
the Pareto principle, the inverse is not true: a statistical
distribution may satisfy the Pareto principle without being 0.1

a Pareto distribution. In fact, many types of power law 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
probability distributions have been observed when analysing
human activity, and OSS project activity in particular, and the
Fig. 1. Cumulative view of distribution of activity for Brasero (November
Pareto distribution is only one them [7], [11]. Many power 2010). The x-axis shows the cumulative percentage of active persons, ordered
law distributions satisfy the Pareto principle without being a from most to least active; the y-axis shows the cumulative percentage of
Pareto distribution. activity. The upper distribution (blue circles) corresponds to the commit
activity . The distribution with red squares corresponds to the mail activity.
The second research question of Section II-A corresponds to The distribution with green triangles corresponds to the bug report change
determining whether the Pareto principle is present throughout activity.
the entire life of the project, and whether it emerges, stabilises
or disappears over time.
These distributions illustrate that there always is a small
C. Selected projects core team of persons that account for most of the activity. For
We will analyse the distribution and evolution of activ- the commit activity, 3 out of 193 persons carry out about 70%
ity on the following OSS projects: Brasero (projects.gnome. of the total number of commits. Even more striking is the fact
org/brasero), Wine (www.winehq.org), and Evince (projects. that a single developer accounts for 60% of the total number
gnome.org/evince). They have been selected based on a variety of commits. For the mail activity, 7 out of 92 persons sent
of factors: popularity, age size, availability of the necessary about 60% of all the mails. For the bug report change activity,
data sources for analysis, and so on. Some of the characteris- 5 out of 253 persons carry out about 40% of all bug report
tics of the three selected projects are presented in Table I. changes.
While a detailed statistical analysis of the exact type of
IV. E MPIRICAL STUDY
distribution is left for future work, we do find clear support for
A. Brasero the Pareto principle: for the commit activity, 20% of the most
Figure 1 shows the cumulative activity distribution in the active committers contribute to about 85% of all commits. For
Brasero community, for three types of activity: commits made the mail (resp. bug report change) activity, 20% of the most
to the version repository, mails sent to the mailing list, active committers contribute with about 75% of all mails (resp.
and changes made to bug reports in the bug tracker. These bug report changes).
distributions are shown for the last version of Brasero we Figure 2 compares the activity distribution between different
analysed, namely the one available on November 2010. For the types of bug report activities available in the bug tracker:
previous versions, with the exception of the earliest versions, submitting new bug reports, changing existing bug reports,
we get similar results. and commenting on existing bug reports. It provides similar
1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Fig. 3. Overlaps of activity for the top 20 most active individuals of the
Brasero community for the three considered categories of activity (November
Fig. 2. Cumulative view of distribution of activity for Brasero bug reports 2010).
(November 2010). The x-axis shows the cumulative percentage of active bug
reporters, ordered from most to least active; the y-axis shows the cumulative
percentage of bug report activity. The upper distribution (green triangles)
corresponds to bug report changes, the middle one (red squares) to bug report time, we used the econometric aggregation measures intro-
submissions, and the lower one (blue circles) to bug report comments. duced in Section II-C. Figure 4 displays the evolution of three
indices (Hoover, Gini and Theil) for the commits in Brasero.
Each data point in this figure corresponds to a different
distributions as those found in Figure 1. distribution such as the ones shown in Figure 1. We observe
To answer research question 3 of Section II-A, we deter- that, regardless of the index used, the values do not fluctuate a
mined the overlap between different categories of activities lot, and tend to stabilise over time. For Gini, for example, we
(committing, mailing, changing bug reports), by analysing see that the index remains most of the time between 0.8 and
those persons that were involved in different activities. Fig- 0.9, indicating a very unequal distribution of commit activity
ure 3 presents the results of this analysis. The triplet notation for all observed versions. This corroborates what we already
(c%, m%, b%) used in each of the intersections corresponds observed before: a low number of individuals contribute most
to the percentage of activity of a particular individual that of the commits.
contributed to more than one activity category. For example,
there is a person with (61%, 11%, 20%), indicating that he 1

contributed to 61% of the total activity of the top 20 most 0.9
active committers, to 11% of the mail activity of the top 20 0.8
most active mailers, and to 20% of the bug report change
0.7
activity of the top 20 most active bug report changers.
0.6
As expected, we find a clear overlap of activity. The two Gini
0.5
most active committers (out of the top 20) are also very active Hoover
0.4
mailers and bug report changers. In fact, the same two persons Theil (normalised)
0.3
account for 67% of the top 20 commit activity, 34% of the
0.2
top 20 mail activity and 27% of the top 20 bug report change
0.1
activity in Brasero. We also observe that three of the 20 most
active mailers are also active as top 20 bug report changers. 0
Oct-06 Feb-07 Jun-07 Oct-07 Feb-08 Jun-08 Oct-08 Feb-09 Jun-09 Oct-09 Feb-10 Jun-10 Oct-10
Note that the analysis process for obtaining the results in
Figure 3 was manual, which explains the restriction to the
top 20 most active individuals only. The reason is that it Fig. 4. Comparison of three aggregation indices, Gini (blue straight line),
is quite challenging to automate a reliable identification of Theil (green dotted line) and Hoover (red dashed line), applied to the evolution
identities (logins, e-mail addresses, names) that correspond to of commit activity for Brasero since October 2006.
the same person. As can be seen in Figure 3, the total sum
of all contributors per activity category is 19 instead of 20 Figure 5 shows how the Gini index differs across the
for committers and mailers. The reason for this is that, during different activity categories we analysed over time for Brasero:
the manual analysis, we observed that two different identities commits, mails and bug report changes. Again, the results
actually corresponded to the same individual (because he has correspond to what we observed in the distributions of Fig-
used two different e-mail addresses or logins over time). ure 1. In all cases, the activity is unequally distributed across
Therefore, we merged the corresponding data into a single individuals. This is especially the case for the commits (with
entity. a single committer accounting for 60% of the total number
To find out how the distribution of activity evolves over of commits), explaining the high value of the Gini index. For
$"
!#," 1
!#+"
0.9
!#*"
0.8
!#)"
!#(" 0.7

!#'" .4556/7" 0.6
!#&" 58697" 0.5
!#%" :;"0".<8=>?7" 0.4
!#$"
0.3
!"
-./0!)" 1230!*" -./0!*" 1230!+" -./0!+" 1230!," -./0!," 1230$!" -./0$!" 0.2

0.1

Fig. 5. Comparison of Gini indices over the evolution of Brasero commits 0
(blue straight line) and bug report changes (green dotted line) since October 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
2006, and mails (red dashed line) since September 2007.
Fig. 6. Cumulative distribution of activity for Evince (November 2010).
The distribution with blue circles corresponds to the commit activity. The
mail activity and bug report change activity, there is also an distribution with red squares corresponds to the mail activity. The distribution
with green triangles corresponds to the bug report change activity.
unequal distribution, but less flagrant than for the commits.
This explains why their Gini index curve is below the one for
the commit activity.

B. Evince
A study of the evolution of the Evince community provides
similar results. Perhaps an important difference is that the
development activity is a bit more equally distributed than
for Brasero (where we found a single person responsible for
60% of the total commit activity).
Figure 6 shows the cumulative activity distribution for three
types of activity for Evince: commits to the version repository,
mails sent to the mailing list, and changes made to bug
reports in the bug tracker. Evince has two top committers,
each accounting for 15% of the total commit activity. The
Pareto principle is also clearly present in Figure 6: 20% of
all committers contribute to 80% of the total commit activity,
20% of all mailers contribute to 70% of the total mail activity, Fig. 7. Overlaps of activity for the top 20 individuals contributing to the
and 20% of all bug report changers contribute to 88% of the considered Evince activity categories (November 2010).
total bug report change activity.
Figure 7 provides a different view on the same data,
obtained by manually analysing the top 20 of most active 1

persons for each activity category. As for Brasero, the 4 most 0.9

active persons of the Evince community contribute to each of
0.8
the three activity categories. Together, they account for 35%
0.7
of all top 20 commits, 26% of all top 20 mails, and 36% of
0.6
all top 20 bug report changes.
0.5
Figure 8 shows the evolution of the three econometric ag-
Gini
gregation indices on the evolution of Evince’s commit activity. 0.4

Hoover
After a steep startup phase, we see that the indices start to 0.3
Theil (normalised)

stabilise rapidly to a more or less stable value. This value 0.2

is lower than for Brasero, since the distribution of commit 0.1

activity is a bit more equally distributed for Evince. 0
Apr-99 Dec-99 Aug-00 Apr-01 Dec-01 Aug-02 Apr-03 Dec-03 Aug-04 Apr-05 Dec-05 Aug-06 Apr-07 Dec-07 Aug-08
Since Figure 8 reveals a similar pattern for all three indices,
in Figure 9 we restricted ourselves to the Gini index only.
We used it to compare the evolution of the commit, mail and Fig. 8. Comparison of three aggregation indices applied to the evolution of
bug report activity of Evince. For each of these activities, we commit activity for Evince since April 1999.
observe that the Gini index stabilises rapidly to a more or less
constant value, indicating that the way in which the community Figure 10 also provides evidence for the Pareto principle.
is structured is fairly stable. For commits and mails, the high 11% of all committers account for a total of 90% of commits.
Gini index indicates that the activity is not equally distributed Similarly, a total of 13% of bug report changers account for
over the community members. For mails, the Gini index is 80% of all changes.
lower so this activity is more spread over different persons. As a side note, we observed that the use of logins and
accounts in Wine was poorly structured. For example, many
$"
committers have 4 or 5 email addresses that are rarely used.
!#,"
!#+" This may be explained by the fact that the Wine community
!#*" is very open: it is very easy for new persons to become active
!#)" in this community.
!#("
!#'" 1233456" Figure 11 shows the overlap of activity of the top 20 most
!#&" 37486" active persons in each activity category for Wine. Again,
!#%" 9:;"/<.2/5"1=7>;<6" we needed to merge two different identities corresponding
!#$"
!"
to the same individual into a single identity (explaining a
-./0,," -./0!!" -./0!$" -./0!%" -./0!&" -./0!'" -./0!(" -./0!)" -./0!*" -./0!+" -./0!," -./0$!" total sum of 19 instead of 20 for committers). In contrast
to the previously analysed projects, none of the 20 most
Fig. 9. Comparison of the evolution of Gini indices for Evince, since April active persons contributed to three different activity categories.
1999 for commits (blue straight line), since January 2005 for mails (red dashed Another major difference was that the core group of active
line), and since August 2004 for bug report changes (green dotted line). persons for Wine was significantly bigger, explaining the
smaller percentages we obtained for the most active persons
C. Wine involved in a particular activity.
A study of the Wine community reveals that the number of
persons involved is much bigger than for the other two systems
studied. This was already apparent from Table I.

Fig. 11. Overlaps of activity of top 20 individuals contributing to the
considered Wine activity categories (November 2010).

Fig. 10. Cumulative distribution of activity for Wine (November 2010). Figure 12 compares the evolution of Wine’s commit activity
The distribution with blue circles corresponds to the commit activity. The using three different aggregation indices. We observe that the
distribution with red squares corresponds to the mail activity. The distribution
with green triangles corresponds to the bug report change activity.
Gini and Theil indices are very high and continue to increase
over time, indicating a very unequal distribution of activity,
Figure 10 shows the cumulative activity distribution for with a large group of inactive persons and a small group of
Wine committers, mailers and bug report changers. The most active persons.
active committer accounts for 13% of the total commit activity, Figure 13 displays the evolution of the three types of activity
the two most active bug report changers each account for 13% for Wine over time, using the Gini index as aggregation
and 11% of the total change activity, respectively. measure. For mail activity, the Gini index is initially much
Concerning the mail activity of Wine, we observed that lower than for the commit activity, but after a while its
there is one huge mailer wineforum-user@winehq.org value starts to increase to comparable values. This high value
accounting for 48% of the total project mail activity. It turned is largely explained by the presence of a single artificial
out that this mailer was in fact an automated transcription of mailer (corresponding to the wineforum). For the bug report
the discussion forum in the mail system. After excluding this change activity, we also observe an increasing growth of
outlier from the mail activity data set, we found that the most the Gini index, revealing an increasing inequality of activity
active mailer only accounts for 4% of the total mail activity. distribution over time.
1 indices of statistical dispersion, the Hoover index, Theil index
0.9 and Gini index. They all gave similar results, i.e., they tend to
0.8
evolve in the same way for a given OSS project’s history.
0.7
This probably implies that one can freely choose any of
these indices to assess the evolution of activity distribution.
0.6
This corresponds to the findings of Vasilescu et al. [13] that
Gini
0.5
Hoover compared different aggregation measures applied to software
0.4 Theil (normalised) metrics and observed a strong correlation between them.
0.3 For all three OSS projects we studied, the distribution of
0.2 activity was initially more equally distributed, but over time
0.1
the activity tends to become concentrated in a core group of
persons that is significantly more active than the others. This
0
Nov-98 Nov-99 Nov-00 Nov-01 Nov-02 Nov-03 Nov-04 Nov-05 Nov-06 Nov-07 Nov-08 Nov-09 Nov-10 knowledge is quite important, as the sudden disappearance
of some members of the core group may have an important
impact on the future of the software project. In other ways,
Fig. 12. Comparison of three aggregation indices applied to the evolution we found empirical evidence of the so-called bus factor, the
of commit activity for Wine since November 1998.
total number of key persons that would, if they were to be
hit by a bus, lead the project into serious problems. Note that
$"
!#,"
this was less the case for Wine, by far the biggest of the three
!#+" projects, where the activity was more equally distributed over
!#*" the most active committers than for the other two projects.
!#)" 2.44536"
For all types of activities, we found a long tail of persons
!#(" 47586"
!#'"
whose activity rate can be largely neglected: during the entire
9:;"<=>.<3"2?7@;=6"
!#&" lifetime of the project, they contributed once or twice to one of
!#%" the considered project activity categories (commits, mails and
!#$"
bug reporting). We also observed that, except for Wine, the
!"
-./0,+" -./0,," 1230!!" 1230!$" 1230!%" 1230!&" 1230!'" 1230!(" 1230!)" 1230!*" 1230!+" 1230!," 1230$!"
most active project members take part in all these activities.
As a potential threat to validity, during our experiments,
Fig. 13. Comparison of the evolution of Gini indices for Wine, since
we encountered some problems to determine which persons
November 1998 for commits (blue straight line), since September 2005 for contribute to different activity categories (Figures 3, 7, 11).
mails (red dashed line), and since November 2005 for bug report changes This was mainly a manual and error-prone process that took a
(green dotted line).
lot of effort. In the future we intend to automate this process.
To achieve this, an efficient identity merging algorithm is
needed that allows one to identify matches between entities
V. D ISCUSSION
participating in different data sources. But for poorly struc-
The results shown in the previous section provide strong tured projects such as Wine this may still be quite problematic
evidence for the Pareto principle. The activity of contributors and difficult to automate. We have studied this topic in more
to open source projects is not equally distributed: in all three detail in [14].
studied projects, a core group of persons appears to carry
out the majority of the work. This kind of behaviour may VI. R ELATED W ORK
be related to the way in which open source developer com- Some researchers focus on the study of core teams [15], [16]
munities are structured. According to [12], a typical structure and the inequality of distribution in software development [13],
is the so-called onion model. It is followed, among others, [3], [4]. Our work distinguishes itself from that of most other
by the community in charge of the Linux kernel. In such a researchers involved in mining software repositories [17], [18],
layered model, there is a single responsible of the project, [19], [20], [21], who tend to focus on the analysis of the
surrounded by a small core team of software developers, software development artefacts (e.g. source code, bug reports,
around which there is a bigger layer of active developers, and so on) and the dependencies between those. Instead, our
followed an even bigger layer of occasional developers. The main interest goes to the individuals involved in creating
last layer constitutes those users of the project that do not and modifying those artefacts, as well as the interaction and
contribute anything themselves. The more to the center of the communication between those individuals.
onion, the more active a developer, and conversely. Although Because the data related to software development activity
many variants of this layered model exist, the general idea is dispersed and incomplete, and due to the heterogeneity of
behind it remains the same. data formats used in software development, an important effort
Our empirical analysis showed the usefulness of applying must be made in order to collect and represent all available
results from econometry to the analysis of the activity in data in a uniform way [22], [23]. In previous work [14], we
software ecosystems. We used three different aggregation discuss the need to merge identities (logins, e-mail addresses,
names) across different data sources, and compare existing identify the cause of overlaps between activities, because our
identity merging algorithms that try to achieve this. definitions for measuring activity need to be refined further.
Numerous studies have found evidence for the Pareto While Brasero and Evince show a similar evolution of
principle (or, more generally, a power law distribution) in activity distribution and similar overlaps between most active
human-related networks [24]. For instance, evidence for a persons’ activity categories, the Wine software project appears
power law distribution has been found in the number of to have a different behaviour. It has a significantly bigger
citations in papers [25], the number of sexual partners in community, there is significantly less overlap between activity
human societies [26], and many more. In software evolution, categories, and we observed a higher inequality in the activity
Herraiz founds a double Pareto distribution in software size [8] distribution. We can only speculate as to the causes of this.
(using different measures of size). Mitzenmacher generalized
this observation for file system distributions [27]. Hunt and VIII. F UTURE WORK
Johnson demonstrated [28] that most of the data available in While the work presented in this article is very promising,
Sourceforge, a software forge for free/open source software2 , many challenges lay still ahead of us.
follows a Pareto distribution. We intend to carry out a detailed statistical study of the
activity distributions we presented in this article. In particular,
VII. C ONCLUSION we would like to find out which kind of statistical distribution
In this article, we studied and compared the evolution of they represent. Can we find evidence for power laws, Pareto
OSS project activity. Following the GQM paradigm, our main distributions or other types of statistical distributions?
research goal was to understand how activity is distributed We also intend to analyse the activity distribution for
in OSS projects over time. We considered three categories of subgroups and subprojects, and study how the structure and
activity: committing data and code, sending mail and changing size of the project community and the software itself affects
bug reports. We extracted and analysed such activity based on the type of activity distribution. Another aspect worthy of
three different types of data sources: version repositories, mail- further study is the evolution of the activity of individuals
ing lists, and bug tracking data. We carried out an empirical involved in OSS projects. Can we find evidence for a learning
analysis over three different long-lived OSS projects for which curve for newcomers in a project? Does the activity of core
this data was available: Brasero, Evince and Wine. members increase or decrease over time? Can we identify
For all three studied projects and for all considered activity certain typical evolution patterns of activity?
categories, we found evidence for the Pareto principle. The Within an OSS project we would like to study the statistical
activity distributions showed a strong inequality in the activity correlation between different characteristics such as software
of different persons involved in an OSS project: there is a small quality, stakeholder activity, and software size. We also want
group of very active members, and a much bigger group of to study how each of these characteristics correlate across
largely inactive members. For two of the three studied projects, different projects.
the core group of most active members takes part in more than The notions of activity studied in this article, and the metrics
one activity category. In Wine this was much less the case. used for computing them, are perhaps too high-level. We could
In order to gain understanding in how OSS projects evolve, define and use more fine-grained and more specific definitions,
we studied this inequality of distribution over time. To do in order to get a more detailed picture of how project members
so we relied on statistical techniques borrowed from econo- interact, and in order to come up with new effort estimation
metrics. We applied three economic aggregation measures and effort prediction models based on the activity of project
(the Hoover, Gini and Theil index). The evolution of activity members.
distribution appeared to follow two kinds of behaviour. The It remains an open question whether the type of activity
first one is typical of a totally new project: at the beginning, distribution we observed in our empirical study is specific to
the activity is more or less equally distributed, but quickly open source projects. Our hypothesis is that this is indeed the
we observe a tendency towards a more unequal distribution case. To verify this hypothesis, we would like to repeat our
where the activities become more concentrated in a small experiment on proprietary, closed source software projects.
core team. In the second type of observed behaviour, the The main challenge here it to get access to such data, in
activity distribution is already imbalanced since the beginning particular, the evolutionary data related to the activities of the
of the project, and this imbalance continues to become more developers involved in these projects.
pronounced over time. Finally, the long term goal is to provide assistance or
Studying who are the most active persons involved in each guidance, through interactive or automated tool support, for
type of activity, we discovered that these persons are often very all stakeholders involved in an (open source) software project.
active in different activity categories. For Brasero and Evince, Users may rely on information such as the bus factor to decide
the two projects in which we observed this behaviour, the on using a particular open source software product. Developers
project is led by a small group of very active members wearing may use it to identify the core developers and influential
several hats at the same time. We have not yet been able to persons. Project managers may wish to control the activity
distribution, to estimate or predict the effort, or to reduce the
2 http://sourceforge.net/ bus factor risk through a better distribution of activity.
ACKNOWLEDGMENT [19] R. Abreu and R. Premraj, “How developer communication frequency
relates to bug introducing changes,” in IWPSE-Evol ’09: Proceedings of
The research is partially supported by (i) F.R.S.-FNRS the joint international and annual ERCIM workshops on Principles of
software evolution (IWPSE) and software evolution (Evol) workshops.
FRFC project 2.4515.09 “Research Center on Software Adapt- New York, NY, USA: ACM, 2009, pp. 153–158.
ability”; (ii) research project AUWB- 08/12-UMH “Model- [20] D. M. German, “Mining cvs repositories, the softchange experience,”
Driven Software Evolution”, an Action de Recherche Con- in Proceedings of the First International Workshop on Mining Software
Repositories, Edinburg, Scotland, UK, 2004, pp. 17–21.
certée financed by the Ministère de la Communauté française [21] W. Poncin, A. Serebrenik, and M. van den Brand, “Process mining
- Direction générale de l’Enseignement non obligatoire et de software repositories,” in CSMR ’11: Proceedings of the European
la Recherche scientifique, Belgium. Conference on Software Maintenance and Reengineering., 2011.
[22] I. Herraiz, G. Robles, and J. M. Gonzalez-Barahona, “Research friendly
software repositories,” in IWPSE-Evol ’09: Proceedings of the joint
R EFERENCES international and annual ERCIM workshops on Principles of software
evolution (IWPSE) and software evolution (Evol) workshops. New
[1] J. Fernandez-Ramil, A. Lozano, M. Wermelinger, and A. Capiluppi, York, NY, USA: ACM, 2009, pp. 19–24.
“Empirical studies of open source evolution,” in Software Evolution, [23] I. Herraiz, D. Izquierdo-Cortazar, and F. Rivas-Hernández, “Flossmet-
T. Mens and S. Demeyer, Eds. Springer, 2008, pp. 263–288. rics: Free/libre/open source software metrics,” in CSMR ’09: Proceed-
ings of the 2009 European Conference on Software Maintenance and
[2] V. R. Basili, “Software modeling and measurement: the
Reengineering. Washington, DC, USA: IEEE Computer Society, 2009,
goal/question/metric paradigm,” College Park, MD, USA, Tech.
pp. 281–284.
Rep., 1992.
[24] M. E. J. Newman, “The Structure and Function of Complex Networks,”
[3] R. Vasa, M. Lumpe, P. Branch, and O. Nierstrasz, “Comparative analysis
SIAM Review, vol. 45, no. 2, pp. 167–256, 2003.
of evolving software systems using the Gini coefficient,” in Proc. Int’l
[25] S. Redner, “How popular is your paper? An empirical study of the
Conf. Software Maintenance, 2009, pp. 179–188.
citation distribution,” The European Physical Journal B, vol. 4, p. 131,
[4] A. Serebrenik and M. van den Brand, “Theil index for aggregation of 1998.
software metrics values,” in IEEE International Conference on Software [26] F. Lilijeros, C. Edling, L. Amaral, E. Stanley, and Y. åberg, “The web
Maintenance. Los Alamitos, CA, USA: IEEE Computer Society, 2010, of human sexual contacts,” Nature, vol. 411, pp. 907–908, 2001.
pp. 1–9. [27] M. Mitzenmacher, “Dynamic models for file sizes and double Pareto
[5] H. Theil, Economics and information theory. Center Math. Stud. distributions,” Internet Mathematics, vol. 1, pp. 305–333, 2002.
Business Econ., Univ. Chicago, 1967. [28] F. Hunt and P. Johnson, “On the Pareto distribution of Open Source
[6] M. Goeminne and T. Mens, “A framework for analysing and visualising projects,” in Proceedings of Open Source Software Development Work-
open source software ecosystems,” in Proceedings of the Joint ERCIM shop, Newcastle, UK, 2002.
Workshop on Software Evolution (EVOL) and International Workshop
on Principles of Software Evolution (IWPSE), ser. IWPSE-EVOL ’10.
New York, NY, USA: ACM, 2010, pp. 42–47.
[7] M. Newman, “Power laws, Pareto distributions and Zipf’s law,” Con-
temporary Physics, vol. 46, no. 5, pp. 323–351, 2005.
[8] I. Herraiz, “A statistical examination of the evolution and properties of
libre software,” Ph.D. dissertation, Universidad Rey Juan Carlos, 2008.
[9] C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A. Swaminathan, “Min-
ing email social networks,” in MSR ’06: Proceedings of the 3rd IEEE
International Working Conference on Mining Software Repositories,
A. Press, Ed., Shanghai, China, May 2006.
[10] M. Hardy, “Pareto’s law,” The Mathematical Intelligencer, vol. 32, pp.
38–43, 2010, 10.1007/s00283-010-9159-2.
[11] A. Clauset, C. R. Shalizi, and M. E. J. Newman, “Power-law distri-
butions in empirical data,” SIAM Review, vol. 51, no. 4, pp. 661–703,
2009.
[12] M. Antikainen, T. Aaltonen, and J. Vaisanen, “The role of trust in
OSS communities - case Linux Kernel community,” in Open Source
Development, Adoption and Innovation. Springer, Jun. 2007, pp. 223–
228.
[13] B. Vasilescu, A. Serebrenik, and M. van den Brand, “Comparative
study of software metrics aggregation techniques,” in BENEVOL 2010,
December 2010.
[14] M. Goeminne and T. Mens, “A comparison of identity merging algo-
rtihms for open source software ecosystems,” Journal on Systems and
Software. [Submitted], 2011.
[15] G. Robles, J. M. Gonzalez-Barahona, and I. Herraiz, “Evolution of
the core team of developers in libre software projects,” in MSR ’09:
Proceedings of the 6th IEEE International Working Conference on
Mining Software Repositories. Washington, DC, USA: IEEE Computer
Society, 2009, pp. 167–170.
[16] S. Minto and G. C. Murphy, “Recommending emergent teams,” in
Proceedings of the Fourth International Workshop on Mining Software
Repositories. Washington, DC, USA: IEEE Computer Society, 2007,
pp. 5–.
[17] M. D’Ambros, H. Gall, M. Lanza, and M. Pinzger, “Analysing software
repositories to understand software evolution,” in Software Evolution,
T. Mens and S. Demeyer, Eds. Springer, 2008, pp. 37–67.
[18] S. Diehl, H. C. Gall, and A. E. Hassan, Eds., Special Issue on Mining
Software Repositories, ser. Empirical Software Engineering, vol. 14,
no. 3, Jun. 2010.