Evidence for the Pareto principle in Open Source Software Activity Mathieu Goeminne and Tom Mens Institut d’Informatique, Faculté des Sciences Université de Mons – UMONS Mons, Belgium { mathieu.goeminne | tom.mens }@umons.ac.be Abstract—Numerous empirical studies analyse evolving open the metrics that we rely upon. Section III presents the ex- source software (OSS) projects, and try to estimate the activity perimental setup of our empirical study that we have carried and effort in these projects. Most of these studies, however, only out. Section IV presents the results of our analysis of activity focus on a limited set of artefacts, being source code and defect data. In our research, we extend the analysis by also taking into distribution in three OSS projects. Section V discusses the account mailing list information. The main goal of this article evidence we found for the Pareto principle. Section VI presents is to find evidence for the Pareto principle in this context, by related work, and Section VII concludes. studying how the activity of developers and users involved in OSS projects is distributed: it appears that most of the activity II. M ETHODOLOGY is carried out by a small group of people. Following the GQM paradigm, we provide evidence for this principle. We selected A. GQM paradigm a range of metrics used in economy to measure inequality in To gain a deeper understanding of how OSS projects evolve, distribution of wealth, and adapted these metrics to assess how we follow the well-known Goal-Question-Metric (GQM) OSS project activity is distributed. Regardless of whether we analyse version repositories, bug trackers, or mailing lists, and paradigm. Our main research Goal is to understand how ac- for all three projects we studied, it turns out that the distribution tivity is distributed over the different stakeholders (developers of activity is highly imbalanced. and users) involved in OSS projects. Once we have gained Index Terms—software evolution, activity, software project, deeper insight in this issue, we will be able to exploit it to data mining, empirical study, open source software, GQM, Pareto provide dedicated tool support to the OSS community, e.g., by helping newcomers to understand how the community is I. I NTRODUCTION structured, by improving the way in which the community members communicate and collaborate, by trying to reduce Numerous empirical studies aim to understand and model the potential risk of the so-called bus factor1 , and so on. how open source software (OSS) evolves over time [1]. In To reach the aforementioned research goal, we raise the order to gain a deeper understanding of this evolution, it following research Questions: is essential to study not only the software artefacts that 1) Is there a core group of OSS project members (develop- evolve (e.g. source code, bug reports, and so on), but also ers and/or users) that are significantly more active than their interplay with the different project members (mainly the other members? developers and users) that communicate (e.g., via mailing lists) 2) How does the distribution of activity within an OSS and collaborate in order to construct and evolve the software. community evolve over time? In this article, we wish to understand how activity is spread 3) Is there an overlap between the different types of activity over the different members of an OSS project, and how this (e.g., committing, mailing, submitting and changing bug activity distribution evolves over time. Our hypothesis is that reports) the community members contribute to? the distribution of activity follows the Pareto principle, in the 4) How does the distribution of activity vary across differ- sense that there is a small group of key persons that carry ent OSS projects? out most of the activity, regardless of the type of considered activity. To verify this hypothesis, we carry out an empirical As a third step, we need to select appropriate Metrics that study based on the GQM paradigm [2]. We rely on concepts will enable us to provide a satisfactory answer to each of the borrowed from econometrics (the use of measurement in above research questions. For our empirical study, we will economy), and apply them to the field of OSS evolution. make use of basic metrics to compute the activity of OSS In particular, we apply indices that have been introduced project members, and aggregate metrics that allow us to com- for measuring distribution (and inequality) of wealth, and pare these basic metric values across members (to understand use them to measure the distribution of activity in software how activity is distributed), over time (to understand how they development. 1 The bus factor refers to the total number of key persons (involved in the The remainder of this paper is structured as follows. Sec- project) that would, if they were to be hit by a bus, lead the project into tion II explains the methodology we followed and defines serious problems evolve), and across projects (to compare the situation between The Hoover index, defined in Equation 3, is one of the different OSS projects). simplest ways to assess inequality of wealth or income. Its value is the ratio of incomes to take up from the richest B. Basic metrics part of the population to redistribute to the poorest one so To obtain the basic metrics of OSS activity, we will extract that the incomes become perfectly equal. A Hoover index of information from three different types of data sources we have 0 represents perfect equality, while a value of 1 represents at our disposal: version repositories, mailing lists, and bug perfect inequality. trackers. For each of these data sources, we can define metrics that extract and reflect a particular type of activity: n 1 X xi 1 • Development activity: the activity of developers com- H(X) = − (3) 2 i=1 xtotal n mitting source code to a version repository, measured as number of commits. The Gini index, defined in Equation 4, provides a more • Mailing activity: the activity of project members posting complex (but also more representative) way to assess inequal- messages to a mailing list, measured as number of mails. ity of income. The cumulative function of income distribution • Bug tracker activity: the activity of persons interacting is represented by a perfect diagonal if all entities in the with a bug tracker, measured in three different ways: population would have the same income, and by a curve under number of new bug report submissions, number of com- the perfect line otherwise. The Gini index is the value of the ments added to existing bug reports, number of changes surface area between the perfect line and the curve, divided to existing bug reports. by the surface area below the perfect line. Since we are not only interested in a static view of a  n  particular snapshot of an OSS project at a particular moment X ixi  in time, we will extract each of the above activity metrics 2  i=1   during the entire life of the considered OSS projects. G(X) = 1 − · n− n   (4) n−1  X  xi   C. Aggregate metrics i=1 Since several of the research questions require a comparison of the basic metrics (across persons, across projects, and Yet another index to assess inequality of income or wealth over time), we need aggregate metrics that combine the basic is the Theil index. It is based on the Shannon entropy [5], metrics. This is valuable, in particular, if we want to reason and is defined in Equation 5. Because the Theil index is not about the distribution of activity across OSS project members. bounded a priori, one cannot easily compare it with the two To study such distribution, we borrow ideas from econo- aforementioned indices. To normalize the Theil index so that metrics. This discipline uses statistics and metrics to analyse it always returns a value between 0 and 1, we can apply the economic data. As an example, various aggregation measures normalisation function N described in Equation 6 to it. of statistical dispersion have been proposed (e.g., the Hoover, n Gini, and Theil indices) and applied to assess the inequality of 1 X  xi  x  i T (X) = · ln (5) the wealth distribution among people, regions, countries, and n i=1 x̄ x̄ so on. Recently, some of these aggregation measures have been N : t → 1 − e−t (6) used for analysing evolving software systems. Vasa et al. [3] proposed to use the Gini index as an alternative to traditional III. E XPERIMENTAL SETUP software metrics. Serebrenik et al. [4] proposed to use the Theil index instead. Following this emerging trend, we will A. Implementation use three different aggregation measures to study OSS activity In order to obtain replicable and verifiable results, we do distribution. Below we provide the definitions of the three not only need a good methodology. At least as important is aggregation measures we selected: the Hoover index, the Gini automated tool support that enables us to extract data from index, and the Theil index. These definitions rely on two the different data sources, compute the basic and aggregate auxiliary definitions. metrics based on this data, statistically analyse the obtained Let X = {x1 , . . . , xn } be a set of values indexed in metrics, and visually confirm the results. ascending order (∀i ∈ 1 . . . n − 1, xi ≤ xi+1 ). The sum of To this extent, we use and extend our generic framework for all these values will be called xtotal (Equation 1). The mean analysing open source software projects. This framework, pre- of all values will be called x̄ (Equation 2). sented in [6], has been developed in a modular and extensible n X way, facilitating the implementation of new modules meeting xtotal = xi (1) specific needs. Support for automatic extraction of data from i=1 version repositories, mailing lists and bug tracking systems xtotal was already built-in. The framework also supports generation x̄ = (2) and visualisation of various types of software (project) metrics. n OSS project Brasero Evince Wine For the specific purposes of this article, we added a new main programming language C C/C++ C module to compute activity distribution (i.e. the relative activ- versioning system git svn git ity of each involved person), and different variants of activity age (in years) 8 11 11 are supported. In particular, we implemented the three types size (in KLOC) 107 580 2001 # of commits 4100 4000 74500 of activity defined in Section II-B. We also added a module # of mails 460 1800 14000 for computing aggregation indices. Among others, the Hoover, # bug reports 250 950 3300 Gini, and Theil index are currently supported. The existing # commiters 206 204 1229 # mailers 102 610 6879 statistic analysis and visualisation modules can directly exploit # bug reporters 386 961 2676 the information computed by the activity distribution module and the aggregation index module to produce statistical output TABLE I representing the inequality indices. M AIN CHARACTERISTICS OF SELECTED OSS PROJECTS . T HE REPORTED VALUES HAVE BEEN COMPUTED FOR THE LAST VERSION , N OVEMBER 2010. B. Pareto principle Many types of distributions in which people are involved correspond to the so-called Pareto principle: roughly 80% of 1 the effects stem from approximately 20% of the causes [7]. 0.9 This principle and the associated law have been observed repeatedly in a variety of domains, including software evo- 0.8 lution [8], [9]. 0.7 Answering the first research question of Section II-A boils 0.6 down to finding empirical evidence for the Pareto principle 0.5 in OSS project activity distribution. One should note the 0.4 difference between the Pareto principle and the related notion 0.3 of Pareto distribution [10]. While a Pareto distribution satisfies 0.2 the Pareto principle, the inverse is not true: a statistical distribution may satisfy the Pareto principle without being 0.1 a Pareto distribution. In fact, many types of power law 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 probability distributions have been observed when analysing human activity, and OSS project activity in particular, and the Fig. 1. Cumulative view of distribution of activity for Brasero (November Pareto distribution is only one them [7], [11]. Many power 2010). The x-axis shows the cumulative percentage of active persons, ordered law distributions satisfy the Pareto principle without being a from most to least active; the y-axis shows the cumulative percentage of Pareto distribution. activity. The upper distribution (blue circles) corresponds to the commit activity . The distribution with red squares corresponds to the mail activity. The second research question of Section II-A corresponds to The distribution with green triangles corresponds to the bug report change determining whether the Pareto principle is present throughout activity. the entire life of the project, and whether it emerges, stabilises or disappears over time. These distributions illustrate that there always is a small C. Selected projects core team of persons that account for most of the activity. For We will analyse the distribution and evolution of activ- the commit activity, 3 out of 193 persons carry out about 70% ity on the following OSS projects: Brasero (projects.gnome. of the total number of commits. Even more striking is the fact org/brasero), Wine (www.winehq.org), and Evince (projects. that a single developer accounts for 60% of the total number gnome.org/evince). They have been selected based on a variety of commits. For the mail activity, 7 out of 92 persons sent of factors: popularity, age size, availability of the necessary about 60% of all the mails. For the bug report change activity, data sources for analysis, and so on. Some of the characteris- 5 out of 253 persons carry out about 40% of all bug report tics of the three selected projects are presented in Table I. changes. While a detailed statistical analysis of the exact type of IV. E MPIRICAL STUDY distribution is left for future work, we do find clear support for A. Brasero the Pareto principle: for the commit activity, 20% of the most Figure 1 shows the cumulative activity distribution in the active committers contribute to about 85% of all commits. For Brasero community, for three types of activity: commits made the mail (resp. bug report change) activity, 20% of the most to the version repository, mails sent to the mailing list, active committers contribute with about 75% of all mails (resp. and changes made to bug reports in the bug tracker. These bug report changes). distributions are shown for the last version of Brasero we Figure 2 compares the activity distribution between different analysed, namely the one available on November 2010. For the types of bug report activities available in the bug tracker: previous versions, with the exception of the earliest versions, submitting new bug reports, changing existing bug reports, we get similar results. and commenting on existing bug reports. It provides similar 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Fig. 3. Overlaps of activity for the top 20 most active individuals of the Brasero community for the three considered categories of activity (November Fig. 2. Cumulative view of distribution of activity for Brasero bug reports 2010). (November 2010). The x-axis shows the cumulative percentage of active bug reporters, ordered from most to least active; the y-axis shows the cumulative percentage of bug report activity. The upper distribution (green triangles) corresponds to bug report changes, the middle one (red squares) to bug report time, we used the econometric aggregation measures intro- submissions, and the lower one (blue circles) to bug report comments. duced in Section II-C. Figure 4 displays the evolution of three indices (Hoover, Gini and Theil) for the commits in Brasero. Each data point in this figure corresponds to a different distributions as those found in Figure 1. distribution such as the ones shown in Figure 1. We observe To answer research question 3 of Section II-A, we deter- that, regardless of the index used, the values do not fluctuate a mined the overlap between different categories of activities lot, and tend to stabilise over time. For Gini, for example, we (committing, mailing, changing bug reports), by analysing see that the index remains most of the time between 0.8 and those persons that were involved in different activities. Fig- 0.9, indicating a very unequal distribution of commit activity ure 3 presents the results of this analysis. The triplet notation for all observed versions. This corroborates what we already (c%, m%, b%) used in each of the intersections corresponds observed before: a low number of individuals contribute most to the percentage of activity of a particular individual that of the commits. contributed to more than one activity category. For example, there is a person with (61%, 11%, 20%), indicating that he 1 contributed to 61% of the total activity of the top 20 most 0.9 active committers, to 11% of the mail activity of the top 20 0.8 most active mailers, and to 20% of the bug report change 0.7 activity of the top 20 most active bug report changers. 0.6 As expected, we find a clear overlap of activity. The two Gini 0.5 most active committers (out of the top 20) are also very active Hoover 0.4 mailers and bug report changers. In fact, the same two persons Theil (normalised) 0.3 account for 67% of the top 20 commit activity, 34% of the 0.2 top 20 mail activity and 27% of the top 20 bug report change 0.1 activity in Brasero. We also observe that three of the 20 most active mailers are also active as top 20 bug report changers. 0 Oct-06 Feb-07 Jun-07 Oct-07 Feb-08 Jun-08 Oct-08 Feb-09 Jun-09 Oct-09 Feb-10 Jun-10 Oct-10 Note that the analysis process for obtaining the results in Figure 3 was manual, which explains the restriction to the top 20 most active individuals only. The reason is that it Fig. 4. Comparison of three aggregation indices, Gini (blue straight line), is quite challenging to automate a reliable identification of Theil (green dotted line) and Hoover (red dashed line), applied to the evolution identities (logins, e-mail addresses, names) that correspond to of commit activity for Brasero since October 2006. the same person. As can be seen in Figure 3, the total sum of all contributors per activity category is 19 instead of 20 Figure 5 shows how the Gini index differs across the for committers and mailers. The reason for this is that, during different activity categories we analysed over time for Brasero: the manual analysis, we observed that two different identities commits, mails and bug report changes. Again, the results actually corresponded to the same individual (because he has correspond to what we observed in the distributions of Fig- used two different e-mail addresses or logins over time). ure 1. In all cases, the activity is unequally distributed across Therefore, we merged the corresponding data into a single individuals. This is especially the case for the commits (with entity. a single committer accounting for 60% of the total number To find out how the distribution of activity evolves over of commits), explaining the high value of the Gini index. For $" !#," 1 !#+" 0.9 !#*" 0.8 !#)" !#(" 0.7 !#'" .4556/7" 0.6 !#&" 58697" 0.5 !#%" :;"0".<8=>?7" 0.4 !#$" 0.3 !" -./0!)" 1230!*" -./0!*" 1230!+" -./0!+" 1230!," -./0!," 1230$!" -./0$!" 0.2 0.1 Fig. 5. Comparison of Gini indices over the evolution of Brasero commits 0 (blue straight line) and bug report changes (green dotted line) since October 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2006, and mails (red dashed line) since September 2007. Fig. 6. Cumulative distribution of activity for Evince (November 2010). The distribution with blue circles corresponds to the commit activity. The mail activity and bug report change activity, there is also an distribution with red squares corresponds to the mail activity. The distribution with green triangles corresponds to the bug report change activity. unequal distribution, but less flagrant than for the commits. This explains why their Gini index curve is below the one for the commit activity. B. Evince A study of the evolution of the Evince community provides similar results. Perhaps an important difference is that the development activity is a bit more equally distributed than for Brasero (where we found a single person responsible for 60% of the total commit activity). Figure 6 shows the cumulative activity distribution for three types of activity for Evince: commits to the version repository, mails sent to the mailing list, and changes made to bug reports in the bug tracker. Evince has two top committers, each accounting for 15% of the total commit activity. The Pareto principle is also clearly present in Figure 6: 20% of all committers contribute to 80% of the total commit activity, 20% of all mailers contribute to 70% of the total mail activity, Fig. 7. Overlaps of activity for the top 20 individuals contributing to the and 20% of all bug report changers contribute to 88% of the considered Evince activity categories (November 2010). total bug report change activity. Figure 7 provides a different view on the same data, obtained by manually analysing the top 20 of most active 1 persons for each activity category. As for Brasero, the 4 most 0.9 active persons of the Evince community contribute to each of 0.8 the three activity categories. Together, they account for 35% 0.7 of all top 20 commits, 26% of all top 20 mails, and 36% of 0.6 all top 20 bug report changes. 0.5 Figure 8 shows the evolution of the three econometric ag- Gini gregation indices on the evolution of Evince’s commit activity. 0.4 Hoover After a steep startup phase, we see that the indices start to 0.3 Theil (normalised) stabilise rapidly to a more or less stable value. This value 0.2 is lower than for Brasero, since the distribution of commit 0.1 activity is a bit more equally distributed for Evince. 0 Apr-99 Dec-99 Aug-00 Apr-01 Dec-01 Aug-02 Apr-03 Dec-03 Aug-04 Apr-05 Dec-05 Aug-06 Apr-07 Dec-07 Aug-08 Since Figure 8 reveals a similar pattern for all three indices, in Figure 9 we restricted ourselves to the Gini index only. We used it to compare the evolution of the commit, mail and Fig. 8. Comparison of three aggregation indices applied to the evolution of bug report activity of Evince. For each of these activities, we commit activity for Evince since April 1999. observe that the Gini index stabilises rapidly to a more or less constant value, indicating that the way in which the community Figure 10 also provides evidence for the Pareto principle. is structured is fairly stable. For commits and mails, the high 11% of all committers account for a total of 90% of commits. Gini index indicates that the activity is not equally distributed Similarly, a total of 13% of bug report changers account for over the community members. For mails, the Gini index is 80% of all changes. lower so this activity is more spread over different persons. As a side note, we observed that the use of logins and accounts in Wine was poorly structured. For example, many $" committers have 4 or 5 email addresses that are rarely used. !#," !#+" This may be explained by the fact that the Wine community !#*" is very open: it is very easy for new persons to become active !#)" in this community. !#(" !#'" 1233456" Figure 11 shows the overlap of activity of the top 20 most !#&" 37486" active persons in each activity category for Wine. Again, !#%" 9:;"/<.2/5"1=7>;<6" we needed to merge two different identities corresponding !#$" !" to the same individual into a single identity (explaining a -./0,," -./0!!" -./0!$" -./0!%" -./0!&" -./0!'" -./0!(" -./0!)" -./0!*" -./0!+" -./0!," -./0$!" total sum of 19 instead of 20 for committers). In contrast to the previously analysed projects, none of the 20 most Fig. 9. Comparison of the evolution of Gini indices for Evince, since April active persons contributed to three different activity categories. 1999 for commits (blue straight line), since January 2005 for mails (red dashed Another major difference was that the core group of active line), and since August 2004 for bug report changes (green dotted line). persons for Wine was significantly bigger, explaining the smaller percentages we obtained for the most active persons C. Wine involved in a particular activity. A study of the Wine community reveals that the number of persons involved is much bigger than for the other two systems studied. This was already apparent from Table I. Fig. 11. Overlaps of activity of top 20 individuals contributing to the considered Wine activity categories (November 2010). Fig. 10. Cumulative distribution of activity for Wine (November 2010). Figure 12 compares the evolution of Wine’s commit activity The distribution with blue circles corresponds to the commit activity. The using three different aggregation indices. We observe that the distribution with red squares corresponds to the mail activity. The distribution with green triangles corresponds to the bug report change activity. Gini and Theil indices are very high and continue to increase over time, indicating a very unequal distribution of activity, Figure 10 shows the cumulative activity distribution for with a large group of inactive persons and a small group of Wine committers, mailers and bug report changers. The most active persons. active committer accounts for 13% of the total commit activity, Figure 13 displays the evolution of the three types of activity the two most active bug report changers each account for 13% for Wine over time, using the Gini index as aggregation and 11% of the total change activity, respectively. measure. For mail activity, the Gini index is initially much Concerning the mail activity of Wine, we observed that lower than for the commit activity, but after a while its there is one huge mailer wineforum-user@winehq.org value starts to increase to comparable values. This high value accounting for 48% of the total project mail activity. It turned is largely explained by the presence of a single artificial out that this mailer was in fact an automated transcription of mailer (corresponding to the wineforum). For the bug report the discussion forum in the mail system. After excluding this change activity, we also observe an increasing growth of outlier from the mail activity data set, we found that the most the Gini index, revealing an increasing inequality of activity active mailer only accounts for 4% of the total mail activity. distribution over time. 1 indices of statistical dispersion, the Hoover index, Theil index 0.9 and Gini index. They all gave similar results, i.e., they tend to 0.8 evolve in the same way for a given OSS project’s history. 0.7 This probably implies that one can freely choose any of these indices to assess the evolution of activity distribution. 0.6 This corresponds to the findings of Vasilescu et al. [13] that Gini 0.5 Hoover compared different aggregation measures applied to software 0.4 Theil (normalised) metrics and observed a strong correlation between them. 0.3 For all three OSS projects we studied, the distribution of 0.2 activity was initially more equally distributed, but over time 0.1 the activity tends to become concentrated in a core group of persons that is significantly more active than the others. This 0 Nov-98 Nov-99 Nov-00 Nov-01 Nov-02 Nov-03 Nov-04 Nov-05 Nov-06 Nov-07 Nov-08 Nov-09 Nov-10 knowledge is quite important, as the sudden disappearance of some members of the core group may have an important impact on the future of the software project. In other ways, Fig. 12. Comparison of three aggregation indices applied to the evolution we found empirical evidence of the so-called bus factor, the of commit activity for Wine since November 1998. total number of key persons that would, if they were to be hit by a bus, lead the project into serious problems. Note that $" !#," this was less the case for Wine, by far the biggest of the three !#+" projects, where the activity was more equally distributed over !#*" the most active committers than for the other two projects. !#)" 2.44536" For all types of activities, we found a long tail of persons !#(" 47586" !#'" whose activity rate can be largely neglected: during the entire 9:;"<=>.<3"2?7@;=6" !#&" lifetime of the project, they contributed once or twice to one of !#%" the considered project activity categories (commits, mails and !#$" bug reporting). We also observed that, except for Wine, the !" -./0,+" -./0,," 1230!!" 1230!$" 1230!%" 1230!&" 1230!'" 1230!(" 1230!)" 1230!*" 1230!+" 1230!," 1230$!" most active project members take part in all these activities. As a potential threat to validity, during our experiments, Fig. 13. Comparison of the evolution of Gini indices for Wine, since we encountered some problems to determine which persons November 1998 for commits (blue straight line), since September 2005 for contribute to different activity categories (Figures 3, 7, 11). mails (red dashed line), and since November 2005 for bug report changes This was mainly a manual and error-prone process that took a (green dotted line). lot of effort. In the future we intend to automate this process. To achieve this, an efficient identity merging algorithm is needed that allows one to identify matches between entities V. D ISCUSSION participating in different data sources. But for poorly struc- The results shown in the previous section provide strong tured projects such as Wine this may still be quite problematic evidence for the Pareto principle. The activity of contributors and difficult to automate. We have studied this topic in more to open source projects is not equally distributed: in all three detail in [14]. studied projects, a core group of persons appears to carry out the majority of the work. This kind of behaviour may VI. R ELATED W ORK be related to the way in which open source developer com- Some researchers focus on the study of core teams [15], [16] munities are structured. According to [12], a typical structure and the inequality of distribution in software development [13], is the so-called onion model. It is followed, among others, [3], [4]. Our work distinguishes itself from that of most other by the community in charge of the Linux kernel. In such a researchers involved in mining software repositories [17], [18], layered model, there is a single responsible of the project, [19], [20], [21], who tend to focus on the analysis of the surrounded by a small core team of software developers, software development artefacts (e.g. source code, bug reports, around which there is a bigger layer of active developers, and so on) and the dependencies between those. Instead, our followed an even bigger layer of occasional developers. The main interest goes to the individuals involved in creating last layer constitutes those users of the project that do not and modifying those artefacts, as well as the interaction and contribute anything themselves. The more to the center of the communication between those individuals. onion, the more active a developer, and conversely. Although Because the data related to software development activity many variants of this layered model exist, the general idea is dispersed and incomplete, and due to the heterogeneity of behind it remains the same. data formats used in software development, an important effort Our empirical analysis showed the usefulness of applying must be made in order to collect and represent all available results from econometry to the analysis of the activity in data in a uniform way [22], [23]. In previous work [14], we software ecosystems. We used three different aggregation discuss the need to merge identities (logins, e-mail addresses, names) across different data sources, and compare existing identify the cause of overlaps between activities, because our identity merging algorithms that try to achieve this. definitions for measuring activity need to be refined further. Numerous studies have found evidence for the Pareto While Brasero and Evince show a similar evolution of principle (or, more generally, a power law distribution) in activity distribution and similar overlaps between most active human-related networks [24]. For instance, evidence for a persons’ activity categories, the Wine software project appears power law distribution has been found in the number of to have a different behaviour. It has a significantly bigger citations in papers [25], the number of sexual partners in community, there is significantly less overlap between activity human societies [26], and many more. In software evolution, categories, and we observed a higher inequality in the activity Herraiz founds a double Pareto distribution in software size [8] distribution. We can only speculate as to the causes of this. (using different measures of size). Mitzenmacher generalized this observation for file system distributions [27]. Hunt and VIII. F UTURE WORK Johnson demonstrated [28] that most of the data available in While the work presented in this article is very promising, Sourceforge, a software forge for free/open source software2 , many challenges lay still ahead of us. follows a Pareto distribution. We intend to carry out a detailed statistical study of the activity distributions we presented in this article. In particular, VII. C ONCLUSION we would like to find out which kind of statistical distribution In this article, we studied and compared the evolution of they represent. Can we find evidence for power laws, Pareto OSS project activity. Following the GQM paradigm, our main distributions or other types of statistical distributions? research goal was to understand how activity is distributed We also intend to analyse the activity distribution for in OSS projects over time. We considered three categories of subgroups and subprojects, and study how the structure and activity: committing data and code, sending mail and changing size of the project community and the software itself affects bug reports. We extracted and analysed such activity based on the type of activity distribution. Another aspect worthy of three different types of data sources: version repositories, mail- further study is the evolution of the activity of individuals ing lists, and bug tracking data. We carried out an empirical involved in OSS projects. Can we find evidence for a learning analysis over three different long-lived OSS projects for which curve for newcomers in a project? Does the activity of core this data was available: Brasero, Evince and Wine. members increase or decrease over time? Can we identify For all three studied projects and for all considered activity certain typical evolution patterns of activity? categories, we found evidence for the Pareto principle. The Within an OSS project we would like to study the statistical activity distributions showed a strong inequality in the activity correlation between different characteristics such as software of different persons involved in an OSS project: there is a small quality, stakeholder activity, and software size. We also want group of very active members, and a much bigger group of to study how each of these characteristics correlate across largely inactive members. For two of the three studied projects, different projects. the core group of most active members takes part in more than The notions of activity studied in this article, and the metrics one activity category. In Wine this was much less the case. used for computing them, are perhaps too high-level. We could In order to gain understanding in how OSS projects evolve, define and use more fine-grained and more specific definitions, we studied this inequality of distribution over time. To do in order to get a more detailed picture of how project members so we relied on statistical techniques borrowed from econo- interact, and in order to come up with new effort estimation metrics. We applied three economic aggregation measures and effort prediction models based on the activity of project (the Hoover, Gini and Theil index). The evolution of activity members. distribution appeared to follow two kinds of behaviour. The It remains an open question whether the type of activity first one is typical of a totally new project: at the beginning, distribution we observed in our empirical study is specific to the activity is more or less equally distributed, but quickly open source projects. Our hypothesis is that this is indeed the we observe a tendency towards a more unequal distribution case. To verify this hypothesis, we would like to repeat our where the activities become more concentrated in a small experiment on proprietary, closed source software projects. core team. In the second type of observed behaviour, the The main challenge here it to get access to such data, in activity distribution is already imbalanced since the beginning particular, the evolutionary data related to the activities of the of the project, and this imbalance continues to become more developers involved in these projects. pronounced over time. Finally, the long term goal is to provide assistance or Studying who are the most active persons involved in each guidance, through interactive or automated tool support, for type of activity, we discovered that these persons are often very all stakeholders involved in an (open source) software project. active in different activity categories. For Brasero and Evince, Users may rely on information such as the bus factor to decide the two projects in which we observed this behaviour, the on using a particular open source software product. Developers project is led by a small group of very active members wearing may use it to identify the core developers and influential several hats at the same time. We have not yet been able to persons. Project managers may wish to control the activity distribution, to estimate or predict the effort, or to reduce the 2 http://sourceforge.net/ bus factor risk through a better distribution of activity. ACKNOWLEDGMENT [19] R. Abreu and R. Premraj, “How developer communication frequency relates to bug introducing changes,” in IWPSE-Evol ’09: Proceedings of The research is partially supported by (i) F.R.S.-FNRS the joint international and annual ERCIM workshops on Principles of software evolution (IWPSE) and software evolution (Evol) workshops. FRFC project 2.4515.09 “Research Center on Software Adapt- New York, NY, USA: ACM, 2009, pp. 153–158. ability”; (ii) research project AUWB- 08/12-UMH “Model- [20] D. M. German, “Mining cvs repositories, the softchange experience,” Driven Software Evolution”, an Action de Recherche Con- in Proceedings of the First International Workshop on Mining Software Repositories, Edinburg, Scotland, UK, 2004, pp. 17–21. certée financed by the Ministère de la Communauté française [21] W. Poncin, A. Serebrenik, and M. van den Brand, “Process mining - Direction générale de l’Enseignement non obligatoire et de software repositories,” in CSMR ’11: Proceedings of the European la Recherche scientifique, Belgium. Conference on Software Maintenance and Reengineering., 2011. [22] I. Herraiz, G. Robles, and J. M. Gonzalez-Barahona, “Research friendly software repositories,” in IWPSE-Evol ’09: Proceedings of the joint R EFERENCES international and annual ERCIM workshops on Principles of software evolution (IWPSE) and software evolution (Evol) workshops. New [1] J. Fernandez-Ramil, A. Lozano, M. Wermelinger, and A. Capiluppi, York, NY, USA: ACM, 2009, pp. 19–24. “Empirical studies of open source evolution,” in Software Evolution, [23] I. Herraiz, D. Izquierdo-Cortazar, and F. Rivas-Hernández, “Flossmet- T. Mens and S. Demeyer, Eds. Springer, 2008, pp. 263–288. rics: Free/libre/open source software metrics,” in CSMR ’09: Proceed- ings of the 2009 European Conference on Software Maintenance and [2] V. R. Basili, “Software modeling and measurement: the Reengineering. Washington, DC, USA: IEEE Computer Society, 2009, goal/question/metric paradigm,” College Park, MD, USA, Tech. pp. 281–284. Rep., 1992. [24] M. E. J. Newman, “The Structure and Function of Complex Networks,” [3] R. Vasa, M. Lumpe, P. Branch, and O. Nierstrasz, “Comparative analysis SIAM Review, vol. 45, no. 2, pp. 167–256, 2003. of evolving software systems using the Gini coefficient,” in Proc. Int’l [25] S. Redner, “How popular is your paper? An empirical study of the Conf. Software Maintenance, 2009, pp. 179–188. citation distribution,” The European Physical Journal B, vol. 4, p. 131, [4] A. Serebrenik and M. van den Brand, “Theil index for aggregation of 1998. software metrics values,” in IEEE International Conference on Software [26] F. Lilijeros, C. Edling, L. Amaral, E. Stanley, and Y. åberg, “The web Maintenance. Los Alamitos, CA, USA: IEEE Computer Society, 2010, of human sexual contacts,” Nature, vol. 411, pp. 907–908, 2001. pp. 1–9. [27] M. Mitzenmacher, “Dynamic models for file sizes and double Pareto [5] H. Theil, Economics and information theory. Center Math. Stud. distributions,” Internet Mathematics, vol. 1, pp. 305–333, 2002. Business Econ., Univ. Chicago, 1967. [28] F. Hunt and P. Johnson, “On the Pareto distribution of Open Source [6] M. Goeminne and T. Mens, “A framework for analysing and visualising projects,” in Proceedings of Open Source Software Development Work- open source software ecosystems,” in Proceedings of the Joint ERCIM shop, Newcastle, UK, 2002. Workshop on Software Evolution (EVOL) and International Workshop on Principles of Software Evolution (IWPSE), ser. IWPSE-EVOL ’10. New York, NY, USA: ACM, 2010, pp. 42–47. [7] M. Newman, “Power laws, Pareto distributions and Zipf’s law,” Con- temporary Physics, vol. 46, no. 5, pp. 323–351, 2005. [8] I. Herraiz, “A statistical examination of the evolution and properties of libre software,” Ph.D. dissertation, Universidad Rey Juan Carlos, 2008. [9] C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A. Swaminathan, “Min- ing email social networks,” in MSR ’06: Proceedings of the 3rd IEEE International Working Conference on Mining Software Repositories, A. Press, Ed., Shanghai, China, May 2006. [10] M. Hardy, “Pareto’s law,” The Mathematical Intelligencer, vol. 32, pp. 38–43, 2010, 10.1007/s00283-010-9159-2. [11] A. Clauset, C. R. Shalizi, and M. E. J. Newman, “Power-law distri- butions in empirical data,” SIAM Review, vol. 51, no. 4, pp. 661–703, 2009. [12] M. Antikainen, T. Aaltonen, and J. Vaisanen, “The role of trust in OSS communities - case Linux Kernel community,” in Open Source Development, Adoption and Innovation. Springer, Jun. 2007, pp. 223– 228. [13] B. Vasilescu, A. Serebrenik, and M. van den Brand, “Comparative study of software metrics aggregation techniques,” in BENEVOL 2010, December 2010. [14] M. Goeminne and T. Mens, “A comparison of identity merging algo- rtihms for open source software ecosystems,” Journal on Systems and Software. [Submitted], 2011. [15] G. Robles, J. M. Gonzalez-Barahona, and I. Herraiz, “Evolution of the core team of developers in libre software projects,” in MSR ’09: Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories. Washington, DC, USA: IEEE Computer Society, 2009, pp. 167–170. [16] S. Minto and G. C. Murphy, “Recommending emergent teams,” in Proceedings of the Fourth International Workshop on Mining Software Repositories. Washington, DC, USA: IEEE Computer Society, 2007, pp. 5–. [17] M. D’Ambros, H. Gall, M. Lanza, and M. Pinzger, “Analysing software repositories to understand software evolution,” in Software Evolution, T. Mens and S. Demeyer, Eds. Springer, 2008, pp. 37–67. [18] S. Diehl, H. C. Gall, and A. E. Hassan, Eds., Special Issue on Mining Software Repositories, ser. Empirical Software Engineering, vol. 14, no. 3, Jun. 2010.