Skill-based Team Formation in Software
Ecosystems
Daniel Schall 1
Abstract. This paper introduces novel techniques for the A key challenge in software ecosystems is to manage quality
discovery and formation of teams in software ecosystems. For- of software [8, 9, 15] and addressing nonfunctional require-
mation techniques have a wide range of applications including ments (NFRs) in general [4]. Software vendors may require
the assembly of expert teams in open development ecosys- their plug-in developers to maintain certain quality levels to
tems, finding optimal teams for ad-hoc tasks in large enter- deserve an approved status. As shown in earlier research, indi-
prises, or working on complex tasks in crowdsourcing environ- vidual as well as development team skill has a significant effect
ments. Software development performance and software qual- on the quality of a software product [5, 7, 12]. The approach in
ity are affected by the skills and application domain experi- this work takes a socio-technical view on software ecosystems
ences that the team members bring to the project. Team for- wherein ecosystems are understood as the interplay between
mation in software ecosystems poses new challenges because the social system and the technical system [11, 13]. Software
development activities are no longer coordinated by a sin- development teams composed of members with prior joint
gle organization but rather evolve much more flexibly within project experience may be more effective in coordinating pro-
communities. A suitable approach for finding optimal teams grammers’ distributed expertise because they have developed
must consider expertise, user load, social distance and collab- knowledge of who knows what [12]. This paper addresses the
oration cost of team members. We have designed this model problem of team formation in open, dynamic software ecosys-
specifically for the analysis of large-scale software ecosystems tems. In open source development teams are more often than
wherein users perform development activities. We have stud- not formed spontaneously (i.e., they ‘emerge’) based on peo-
ied our approach by analysing the R ecosystem and find that ple’s availability, willingness to collaborate and to contribute
our approach is well suited for the team discovery in software to a certain task.
ecosystems. Potential tasks for expert teams in software ecosystems and
open source development include:
1 Introduction • Come up with a software design and/or implementation of
a complex component or subsystem.
Establishing a software ecosystem becomes increasingly im- • Perform software architecture review of an existing imple-
portant for a companies’ collaboration strategy with other mentation or provide expert opinion about an emerging
companies, open source developers and end users. The idea technology.
behind software ecosystems differs from traditional outsourc- To give a concrete example of a potential high-level task, a
ing techniques [19, 22]. The initiating actor does not necessar- complex design or implementation may involve the analysis of
ily own the software produced by the contributing actors nor timeseries data including data extraction from a source sys-
are contributing actors hired by the initiator (e.g., a firm). tem, transformation of the data, storage, processing, and visu-
All actors as well as software artefacts, however, coexist in an alization. Clearly, this task typically requires multiple people
interdependent way. For example, actors jointly develop ap- with distinct skills such as data modelling, statistical knowl-
plications and thus there is a relationship among the actors. edge (uni-/multivariate data analysis), data persistence man-
Software components may depend on each and thus there is a agement, and data visualization using various technologies
relationship among the components. This is a parallel to nat- and toolkits. Indeed, the high-level task needs to be further
ural ecosystems where the different members of the ecosys- decomposed into smaller task. The goals of this work is to
tems (e.g., the plants, animals, or insects) are part of a food find a team of experts given a set of high level skills. Once
network where the existence of one species depends on the the team has been discovered, detailed task decomposition
rest. In contrast to natural ecosystems, some software ecosys- and work planning can be performed, which is however not in
tems may be mainly top-down controlled, with most changes the focus of this work.
driven by change requests and bug reports coming from other We provide the following key contributions:
actors [20]. Other software ecosystems may be controlled in
a bottom-up manner, primarily driven by input from its core • A novel approach supporting team formation in software
developers [21]. ecosystems based on user expertise, load, social distance
among team members, and collaboration cost.
1 Siemens Corporate Technology, Siemensstrasse 90, 1211 Vienna, • Support the discovery of potential mediators if social con-
Austria, email: daniel.schall@siemens.com nectedness in teams is low.
• Analysis of user expertise to recommend the most suitable 3 Formation in Ecosystems
team members.
• Evaluation of the concepts using data collected from the Here we present the overall team formation algorithm. We
Comprehensive R Archive Network (CRAN) - the largest employ a genetic algorithm that attempts to find the best
public repository of R packages. team. Genetic algorithms (GAs) mimic Darwinian forces of
natural selection to find optimal values of some function [23].
For each team a single number denoting the team’s fitness
The remainder of this paper is organized as follows. In Sec- is calculated (where larger values are better).
tion 2 we overview related work and concepts. In Section 3
we introduce our team formation approach. Experiments are
detailed in Section 4. The paper is concluded in Section 5. 3.1 Genetic Algorithm Outline
There are multiple objectives that need to be optimized. The
objectives are to:
2 Related Work • maximize the average expertise score for given skills
• minimize the average cost
• minimize the average distance
The success of a project depends not only on the expertise The team with the best trade-off among these objectives
of the people who are involved, but also on how effectively shall obtain the highest fitness and will be recommended as
they collaborate, communicate and work together as a team the best fitting team. The required team skills are stated by
[17, 26]. On the one hand, a team must contain the right set customers who wish to assign a specific complex task to a
of expertise, but on the other hand one should determine a team of experts — be it within a corporation or outsourcing
staffing level that, while comprising all the needed expertise, a specific task to the crowd. Our assumption is that such com-
minimizes the cost and contributes to meeting the project plex tasks demand for the expertise of multiple team mem-
deadline [10]. The most critical resource for knowledge teams bers. Within our formation approach, additional constraints
is expertise and specialized skills in using and handling tools. can be considered such as one person shall only cover one skill
But the mere presence of expertise in a team is insufficient and not multiple ones. Indeed, in practice people are familiar
to produce high-quality work. A team must collaborate in an with multiple topics thereby covering multiple skills. Factors
effective manner. It has been found that prior collaborative such as matching user load with complexity, effort, and dead-
ties have a profound effect on developers’ project joining de- line of a task are not in focus of this work. The reader may
cisions [12]. refer to [25] for further information on these topics.
In software engineering, team formation is often needed to The main computational steps of our formation approach
perform a development or maintenance activity. A general are introduced and elaborated in detail in Algorithm 1. The
trend is the growing number of large scale software projects, relevant lines in Algorithm 1 are specified in parenthesis.
software development and maintenance activities demanding
for the participation of larger groups [6, 14]. In [3], the authors 1. Based on the set S = {s1 , s2 , . . . , sn } of demanded skills,
proposed assignment of experts to handle bug reports based prepare a mapping structure that holds skills, users U , and
on previous activity of the expert. Social collaboration on expertise ranking scores (lines 6-9). In this step, current
GitHub including team formation has been addressed in [18]. user load is evaluated (based on previously assigned tasks)
The authors [2] study the problem of online team formation. and users with high load are filtered out.
In their work, a setting in which people possess different skills 2. Initialize a population of individuals. An individual is a
and compatibility among potential team members is modelled team consisting of n team members where n can be config-
by a social network. It has to be noted that team formation in ured. The parameter n is given by the size of the skill set
social networks is an NP -hard problem. Thus, optimization S if each team member has to provision exactly one skill
algorithms such as genetic algorithms [29] should be consid- (line 12).
ered in solving the team composition problem [31]. 3. Loop until max iterations have been reached and compute
Expertise identification is one of the key challanges and suc- the main portion of the genetic algorithm based search
cess factors for team work and collaboration [16]. The discov- heuristic. Finally, after this loop select the team with the
ery of experts is becoming critical to ease the communication highest fitness (lines 14-48).
between developers in case of global software development or 4. Depending on the ecosystems community structure and the
to better know members of large software communities [30]. demanded set of skills, teams may have good or poor con-
Network analysis techniques offer a rich set of theories and nectivity in terms of social links among team members.
tools to analyse the topological features and human behaviour Therefore, construct a subgraph of the social collaboration
in online communities. We have extensively studied the au- graph GS containing only the nodes from the best team
tomatic extraction of expertise profiles in our prior research and their edges between each other. Analyse the connec-
(see [24, 25, 27]) and build upon our social network based tivity of this subgraph by computing the average number
expertise mining framework. of neighbours (lines 50-51).
With regards to team formation in open source commu- 5. If connectivity is low, try to find a dedicated coordinator
nities as well as software ecosystems, there is still a gap in who is ideally connected to all team members through so-
related work and to our best knowledge there is no existing cial links. The role of the coordinator is to mediate com-
approach that supports formation based on mined expertise munication among members and strengthen team cohesion
profiles. (line 53-56).
Algorithm 1 Formation algorithm. 3.2.1 Rank Expertise by Skills
1: input: skills S, population size p size, Expertise profiles are not created in a predefined, static man-
2: elitism elitism k, max iterations max iter ner. Expertise is calculated based on actual community contri-
3: output: best individual - team with the highest fitness
butions. However, we do not attempt to analyse detailed user
4: # init mappings of skills, users, scores
contributions in terms of software versioning control systems
5: M ← ∅
6: for Skill s ∈ S do but rather focus on ‘high-level’ package metadata thereby fol-
7: m ← getRankingScoreM apping(s) lowing a less privacy intrusive approach. The method used for
8: M [s] ← m # add mapping determining the expertise scores is not within the focus of this
9: end for work due to space limits. The interested reader may refer to
10: # initialize population of [27, 28] for information on the basic approach.
11: # randomly composed individuals
12: P ← createP opulation(S, M, p size)
13: iter ← 0 # iteration counter 3.2.2 Basic Selection Strategies
14: while iter++ < max iter do An initial set of candidate solutions are created and their cor-
15: # new population
responding fitness values are calculated. This set of solutions
16: P 0 ← createEmptyP opulation(p size)
17: # obtain a ranked list is referred to as a population and each solution as an indi-
18: R ← rankP opulationByF itness(P) vidual (i.e., the team composed of team members). The indi-
19: # elitism - copy small part of the fittest viduals with the best fitness values are selected and combined
20: for i = 0 . . . elitism k − 1 do randomly to produce offsprings, which make up the next pop-
21: # add to population ulation. The approach of selecting a subset of individuals with
22: P 0 [i] ← getIndividualByRankIndex(R, i) the best fitness values is called elitism. Elitism is realized by
23: end for copying the fittest individuals to the next population.
24: # build new population To maintain a demanded population of individuals, individ-
25: i ← elitism k
uals are selected and undergo crossover (mimicking genetic
26: while i < p size do
27: # stochastically select from P reproduction). For the team formation approach this means
28: I[] ← rouletteW heelSelection(P) that team members are swapped between two teams. The ba-
29: # crossover sic part of the selection process is to stochastically select from
30: if randomN umber < crossover rate then one generation to create the basis of the next generation (see
31: # assign new offspring rouletteW heelSelection in line 28). The requirement is that
32: I[0] ← crossover(I[0], I[1]) the fittest individuals have a greater chance of survival than
33: end if weaker ones. This replicates nature in that fitter individuals
34: # mutation will tend to have a better probability of survival and will go
35: if randomN umber < mutation rate then
forward. Weaker individuals are not without a chance. In na-
36: # assign mutated individual
37: I[0] ← mutate(I[0]) ture such individuals may have genetic coding that may prove
38: end if useful to future generations [1].
39: # add new offspring Individuals are also subject to random mutations. However,
40: P 0 [i++] ← I[0] the probability of mutation is low because otherwise the ge-
41: end while netic algorithm would be just a random search procedure. In
42: # assign the new population our work we apply a ‘smart’ approach to mutation and do
43: P ← P0 not select a replacement team member at random. Rather, a
44: # evaluate population - total fitness new team member is predicted and voting is performed by
45: evaluate(P)
the existing team members.
46: end while
47: # this is the best team
48: I ← f indBestIndividual(P) 3.2.3 Relationship-driven Mutation
49: # construct a graph GI based on I
50: GI ← extractSubGraph(GS , I) In traditional GAs, which are agnostic to the underlying na-
51: if checkConnectivity(GI ) < ζ then ture of the population, mutation takes place by exchanging a
52: # find node to increase connectivity gene by a random gene thereby maintaining genetic diversity.
53: u ← f indCoordinator(GI ) Here we perform prediction of edges between existing team
54: if u 6= null then members and newly randomly selected team members. If a
55: addN ode(GI , u) threshold is surpassed, the randomly suggested team mem-
56: end if
bers is added to the team. This prediction approach ensures a
57: end if
58: # extract the node set NI ⊂ U from GI much higher likelihood that the new team member will work
59: NI ← getN odes(GI ) within the team more effectively. We propose random forests
60: return NI # node set of best team members for predicting edges between pairs of users. Random forests
are a simple yet effective and robust method for classifica-
3.2 Detailed Computational Steps tion problems. Prediction of edges between pairs of users is
performed by modelling features describing the relationship
In the following we detail the steps in the algorithm and ex- between two nodes u and v. At a high level, these features in-
plain additional functions that are invoked while executing clude common neighbours, the jaccard similarity index, joint
the formation algorithm. community interest, and joint package dependencies.
3.2.4 Fitness Function Algorithm 1 line 45). Later on ΦI is used to rank the pop-
ulation by fitness (see Algorithm 1 line 17) as well as when
The last important ingredient of our GA based approach is
calling f indBestIndividual (see Algorithm 1 line 48).
the design of a workable fitness function. A fitness function
Algorithm 2 details the computational steps of an individ-
is a type of objective function that is used to provide a sin-
ual I’s fitness as defined by Eq. 1. An approximation of a
gle number. The idea is to discard ‘bad’ team configurations
cost factor is provided by community popularity in terms of
(i.e., individuals) and to breed new ones from the good config-
number of neighbours in the social graph GS . Thus, according
urations. The search heuristic is terminated when either the
to this logic more popular users are also more expensive. A
overall fitness converges or the maximum number of iterations
perfect fitness score, given a set of skills S, would be 1. This
have been reached.
is however impossible to achieve because expertise is also in-
The fitness function computes the value ΦI as
fluenced by the user degree in GS . Thus, a suitable tradeoff
ΦI = w1 ∗ expertise + w2 ∗ cost + w3 ∗ distance (1) among these factors has to be found. The individual with the
highest fitness within the population is then selected and rec-
where w1 + w2 + w3 = 1. Each of the input factors ommended as the best team.
expertise, cost, distance needs to be scaled between 0 and
1. ΦI is computed when invoking the function evaluate (see 3.2.5 Coordinators
Algorithm 2 Fitness function. Based on the set of demanded skills and community structure,
it may not be possible to find teams with good connectiv-
1: input: skills S, individual I, ranking score mapping M
2: output: fitness value ΦI ∈ [0, 1] of individual I
ity among the team members. The last steps in Algorithm 1
3: metrics ← ∅ # metrics as basis for fitness would be to check the connectivity of I and to find a dedicated
4: score ← 0 # team expertise score node who is ideally connected to all nodes in I to mediate
5: popularity ← 0 # popularity - to approximate cost communication. All nodes in U matching this constraint are
6: for Skill s ∈ S do then ranked based on their averaged expertise given the skill
7: u ← I[s] # get member by skill set S. The discovered node acting is potential team coordina-
8: rs ← M [s][u] # expertise score by skill tor is added to the final team.
9: # perform feature scaling
max(M [s])−rs
10: rs0 ← 1 − max(M [s])−min(M [s])
4 Experimental Evaluation
11: score ← score + rs0
12: # get user degree in GS 4.1 R Ecosystem
13: ku ← degree(GS , u)
14: # perform feature scaling In this section we present our experiments. We focus on one
15: ku0 ← kmax −ku part of the R ecosystem called the Comprehensive R Archive
kmax
16: popularity ← popularity + ku0 Network (CRAN). Other R-based communities not considered
17: end for in this research are, for example, Bioconductor2 . We have im-
18: # add average team expertise score to metrics plemented a Web crawler to download3 and parse R software
19: addM etric(metrics, score/|S|) package meta information available as HTML pages. This in-
20: # add average popularity to metrics formation includes contributing authors and package depen-
21: # higher community popularity means higher cost dencies. In addition, CRAN provides so called CRAN task
22: addM etric(metrics, popularity/|S|) views, which are used in our analysis as skill or topic infor-
23: dist ← 0 # social distance mation. As an example, a given task view Bayesian Inference
24: Q ← queue(I)
would be a single skill.
25: while Q 6= ∅ do
26: u ← poll(Q) Figure 1 shows the package authorship graph GA .
27: for v ∈ Q do
28: # unweighted shortest path distance Wanhua Su
Alexandra Laflamme−Sanders
Greg Young
extRemes
Rick Katz
# d(u, v) computed in GS
lago
Huan Cheng
29:
distillery SpatialVx
smoothie
BASIX
in2extRemes
GeneFeST
Eric Gilleland
Bastian Pfeifer
Ulrich Wittelsbuerger
PopGenome
Bob Handsaker
A. G. Taylor
S. D. Cohen
C. A.J. Appelo
D. Gillespie R. Serban Liangying Zhang
Holly Janes S. R. Charlton
D. Shumaker
D. L. Parkhurst Tim Hoar
Tom Kraljevic phreeqc RadioSonde
Spencer Aiello
Amy Wang
Petr Maj
iid.test
Ariel Rao esd4all
h2o
Yanming Li replicationDemos
cyclones
Alexandra Imbert
duv = d(u, v)
Bin Nan clim.pact
MSGLasso anm
Nello Blaser
Rasmus E. Benestad
30:
Luisa Salazar Vizcaya
knnflex
Yan Liu Dennis D. Boos
Alejandro Cáceres Yubo Zou
Atina Dunlap Brooks
Yuzheng Zhang PrecipStat Dorit Hammerling
Yingwei Peng Rlab
Hexin Zhang qqplotter
Tia Lerud
MLPAstats
CNVassoc
Douglas Nychka Nathan Lenssen
Edsel A. Peña
LatticeKrig
CNVassocData
Doug Nychka gcmrec smcure
gvlma Emmanuel Sharef
Elizabeth H. Slate
Juan R. González
survrec Robert L. Strawderman
Stephan Sain splinesurv
Alan C. Hindmarsh Added Fortran David Ruppert
Stephen Sain
Chao Cai
ordBTL Piotr Romanski
David M. Spooner
John Paige
fields
Jiajia Zhang William Roca
Songfeng Wang
FSelector International Potato Center Rene Gomez
Giuseppe Casalicchio Marc Ghislain
exsic
mcpd
Barry Hurley Merideth Bonierbale
Jie Zhou
Vilma Hualla
Tobias Kuehn
quipu
Lars Kotthoff NPHMC
Talal Rahwan Pablo Carhuapoma
Erich Studerus
Reinhard Simon divagis
Leonard Judt Felipe de Mendiburu
TransModel Michael Braun Original Fortran Jorge Nunez
llama monographaR
Stef de Haan agricolae
Sangkyun Lee
H. Welsh Fraser Ian Lewis KappaV Thomas F. Coleman
Zachary Jones
Lluís Armengol
Fraser Lewis
if duv 6= null then
datacheck
sparseHessianFD
Lingbing Feng Xavier Solé
Marcelo Reginato Marta Pittavino SNPassoc
J. O’Neill Dan Dkin
abn
31:
Vincent Bonhomme Jorge J. More Elisabet Guinó
Sarah Ivorra
Cedric Gaucherel Evan Saitta
Jun Peng Burton S. Garbow
Sandrine Picq
Asher Wishkerman Jose Francisco Loff Fernando Colchero
Wenbin Lu Telmon
Hinrich W. H. GohlmannChiara Forcheh Linda R. Petzold Maren Rebke
Anyiawung
Ricardo Kriebel Ulrike von Luxburg
Willem Talloen Georgi NalbantovNorbert Kenneth Hillstrom
truncdist
Alain Berro loe
Neus Martinez tgcd
Momocs Hoksan Yip Alberto Murta
Yoshikazu Terada BaSTA
Lieven Clement SVMMaj
Reinhard Furrer Anu Sironen Jason Collison
Alexander Duerre
Geert Verbeke smds
beadarrayFilter hoardeR isopam
GeneticTools
ALKr simco
REPPlab Pushpike Thalikarathne
gaussquad
Patrick J. F. Groenen Enyelbert Muñoz Sebastian Schmidtlein
GENEAread sscor
Daniel Fischer
PamGeneMixed
orthopolynom
cluster.datasets
Hannes Feilhauer
aBioMarVsuit
liso Frederick Novomestky
LDRTools Eero Liski Owen Jones
Vincent T. van Hees Zhou Fang matrixcalc
goalprog
Carsten Oldenburg
Severine Sabia autopls Daniel Vogel
Ziv Shkedy
rportfolios
gMWT
GGIR Raquel Iniesta
Marion Deville
OjaNP Robert Maillardet
Emmanuel Rousseaux
BayHap Steven Carnie
Bradford B. Worrall Richard Brent
Stephen Ed Ionides John Burkardt
Michele M.
ABCoptim Sale R. Williams
PST Martin Kroll credule
Víctor Moreno
Jung−Ying Tzeng Carles Breto Olga Borovkova
OptGS
Jyrki Möttönen Rafael S. de Souza
Bertrand Le Nezet
Matthew J. Ferrari
spuRs RCEIM
spnet Jonathan Elliott
Hannu Oja Gilles Pujol
John Wambaugh moments
Steve Ellner
Dao Nguyen LOGIT
Sebastian Funk MuFiCokriging
Edward L. Ionides Simulation du Comportement
FastKM
Michael Lavine Bruce E. Kendall RandomFieldsUtils Alexandre Janon
Robert Pearce James Wason
googlePublicData
Sara Taskinen Bernardo Ramos Jimena Davis safi
CosmoPhotoz RsimMosaic
tsBSS MNM ICSNP Helen Wearing Sebastien Da Veiga Alberto Krone−Martins
httk Loic Le Gratiet Nisha Sipes
fICA Markus Matilainen
Dave Tyler
ICS David E. Tyler popEpi M. Rantanen Malte Jastrow Joseph M. Hilbe
Klaus NordhausenIngo Bulla Xavier Bay
BSSasymp Jari Miettinen Alexis Jinaphanh Jospeh Hilbe
JADE colcor COUNT
pomp
R. Woodrow Setzer SpatialNP Jaakko Nevalainen Omkar Muralidharan UPMASK
J. Pitkaniemi Thomas Muehlenstaedt
Jean−Francois Cardoso Jana Fruth
Rachel Marceau Olivier Jacquet Renext
Carles Martinez Breto Taieb Touati msme
Fang−Chi Hsu Daniel C. Reuman Seija Sirkia sensitivity
K. Seppa
Esa Ollila fanovaGraph equivalence Andre Moitinho
Stephen P. Ellner codadiags
Nicolas Durrande RenextGUI
mixfdr Joseph Guillaume financial
Brett T. McClintock
Lukasz Komsta
32: # perform feature scaling
Service d’Etudes
multimark Yves Deville
Bradley Efron GPlab
Acho Arnold Laurent Gilquin Energie Atomique S. Marmin Andrew Robinson moonsun
bit64 Werner Brannath DiceView kergp
mblm
AGSDest ref Paul Lemaitre
Clement Chevalier
DiceOptim quantchem
DiceKriging T. Wagner
Barry W. Brown rindex Olivier Roustant
Jérôme Guélat KrigInv
bit Vincent Moutoussamy Yann Richet David Ginsbourger dtt
James Lovato regtest Rick ReevesMarguerite A. Butler Nicolas Bousquet
Jens Oehlschlägel AHR GPareto
Jens Oehlschl Matthias Brueckner Victor Picheny
Cleve Moler
outliers
Clement Walter
Martin Schlather subplex
MarkedPointProcess ouch Gilles Defaux
Edward Sudicky
Aaron A. King Rene Therrien mistral
SoPhy Jessica Franco Mickael Binois
Delphine Dupuy DiceEval
Guanhua Chen Heike Schuhmacher
simex Rien van Genuchten C. Helbert
Wolfgang Lederer Bernd Huwe
fastM
DiceDesign
Heike Schumacher
kin.cohort
Andrew Sila Carl A. Mendoza
vectoptim Guillaume Damblin
Manuel Wiesenfarth Nilanjan Chatterjee
Thomas Terhoeven−Uselmans
soil.spec Thomas Terhoeven−Urselmans
SwissAir
Yusuke Sugomori Pius Korner−Nievergelt
Carl Morris Peter M. J. Herman
Ryan Curtin blmeco FAwR
RcppMLPACKRcppDL Joseph Kelly
AdaptFitOS ecolMod Bertrand Iooss Stefanie von Felten
Frederik De Laender
Rgbp
Niki Zumbrunnen
ToxLimKarel Van den Meersche
rootSolve BCE Bettina Almasi
Henrik Andersson
Theresa Albrecht ReacTran
Filip Meysman Rene Locher pvclass
OceanView limSolve femmeR
diagram Franscesca Mazzia
deTestSet Jeff Cash
ByeongChris−Carolin
Yeob Choi Schoen
StackOverflow
Karline Soetaert
Tobias Roth
diffEq
Thomas Unternährer LIM bvpSolve Dick van Oevelen James Orr
Yvonne Badke
Christian Reimer plot3Drgl Fraenzi Korner−Nievergelt
Malena Erbe Tatyana Krivobokova Andreas F. Hofmann Bernard Gentili
synbreed
Francesca Mazzia
AquaEnv
Lutz Duembgen
plot3D Heloise Lavigne
Peter VandeHaar seacarb
shape NetIndices Jean−Marie Epitalon
Pius Korner
Hans−Juergen Auinger
Julius Kipyegon Kones birdring
Larry Schaeffer
evd Filip J. R. Meysman Aurélien Proye
GSIF Barbara Hellriegel
Hannes Reuter AdaptFit Jim Orr
Alec G. Stephenson Chris Ferro carcass
Yunro Chung
Anastasia Ivanova Julius Kipkeygon Kones
Michael G. Hudgens
PlayerRatings Manuela Huso
Matthew A. Etterson
Brendan Malone James Rae
Jean−Pierre Gattuso
Olivier Lepais Bas Kempen Jeff Sonas
FedData Johanna G. Neslehova Rob Robinson
Murray Lark
MultiCNVDetect mev
Leo Belzile Oliver Behr
33: # lower values are better
Manuela M. P. Huso
Markus Baaske R. Kyle Bocinsky Dylan lcopula
Christian Genest Jeff D. Hamann Robert Brinkmann Dan Dalthorp pRSR
Gerard Heuvelink Ivo Niermann
Alexander J. McNeil
logcondens QRMlib Bhramar Mukherjee
Scott Ulman mFilter
SimRAD rconifers
R. Kyle Bocinsky
Doug Maguire
M. S. Islam
evir JointModeling Mehmet Balcilar
HDPenReg JointGLM
WaveCGH
Martin W. Ritchie
Joshua French nnc
Guenther Walther
nncRda pear Hyukjun Gweon
PhySim
Changjiang Xu
Ruo Xu Andrea Rau Dolph Schluter modehunt sltl
Lucia H. C. Anjos Luciano Seta bestglm
EvoRAG ascrda Mark M. Meerschaert
Dick Brus Stanislas Hubeaux Manuel Gentile
evdbayes reporttools
smerc
ExceedanceTools pequod deseasonalize
SurvRegCensCov Filippo Santambrogio
artfima
Quinn Payton
Ken Kleinman selectMeta FitAR
Kaspar Rufibach OrdFacReg Farzad Sabzikar
Hao Lin Kendall
Alberto Mirisola ebdbNet DeducerMMR A. I. McLeod
rtkpp smacpod Laura Hauser Ying Zhang FGN
Marc Weber rneos SpatialTools Dario La Guardia
DTK
OrdMonReg koRpus arfima Justin Q. Veenstra
systemfit mleur
ltsa
Zinovi Krougly
gogarch cents
Alexandre Brulet
rsatscan Hao Yu FitARMA censNID
Abdel El−Shaarawi
Pablo Garcia Rodriguez Fadoua Balabdaoui DiversitySampler rwm
Matthew K. Lau Earl Brown Nagham M. Mohammad
JohnsonDistribution hcc
Juan Carlos Ruiz Cuetos Serge Iovleff
Maria Eugenia Polo Garcia Rmpi
D. E. Hines dynia
cccp
enaR P. Sing
Lin Wang Joachim Dahl Vincent Kubicki Yun Shi
MixAll Leanna King
Qizhai Li Kathrin Weyermann pcenum
logcondiscr
VecStatGraphs3D Parmeet Bhatia William Wheeler Aude Longeville
Martin Andersen rtkore S. R. Borrett spcosa
FRAPO
Jinkun Xiao
Lieven Vandenberghe Benjamin Auder
Emmanuel Blondel D. J.J. Walvoort modelcf Yixin Chen
HTSDiff
cleangeo ARTP Rmixmod glmlep Florent Langrognet
Mathieu Ribatet VecStatGraphs2D Miguel Sousa Lobo
HTSCluster Kai Yu
D. J. Brus Henry Bart Jr
POT Stephen Boyd Jaap de Gruijter RobustEM
Angel Felicisimo
G. Govaert
Herve Lebret
rddtools Dennis Walvoort
Christophe Biernacki
J. J. de Gruijter LGEWIS Xin Dang
Lieven Vandenberge Remi Lebret
Aurora Cuartero blockcluster Aishat Aloba
RFA
BEQI2
J. C. Cuetos versions
P. G. Rodriguez GRaF Gilles Celeux Zihuai He
Javier F. Tabima M. E. Polo Willem van Loon LGRF
Marie−Laure Martin−Magniette
Cathy Maugis−Rabusseau
# max(GS ) is diameter of graph
Pallavi Singh
Hanna Jankowski
Grard Goavert rnn
Jonah C. Brooks
Seunggeun Lee
Parmeet Singh Bhatia
Gerard Goavert
34:
Geraldine Henningsen Nick Golding Bastiaan Quast
Stacy A. Krueger−Hadfield
CPHshape
Canhong Wen
Victor Kmmritz MetaSKAT
David B. Dahl decompr
rscala RmixmodCombi wiod gvc
poisson.glm.mix
learNN
jvmr
SelvarMix
Marie Davidian BayesComm Fei Wang
Tim David
Coelli J. Harris SKAT
diagonals Rihong Hui
Victor Kummritz Larisa Miropolsky
Anastasios A. Tsiatis James E. Hines
bamboo Vincent Brault
convexHaz
K. Ullas Karanth
Tobias Liboschik J.−P. Baudry
Min Zhang Wu
blender SPACECAP Jim Hines
Ivy Wang Michael WuMicheal
Michael E. Meredith
Panagiotis Papastamoulis
Hugh McCague Sumanta Mukherjee
Arjun M. Gopalaswamy
Yanfei Kang
Jon A. Wellner
TED cdcsis
robeth Devcharan Jathanna
Kendon Bell James D. Nichols
Andrew J. Royle
Mohammed Sedki
prism Danijel Belusic
rWBclimate label.switching N. Samba Kumar Xueqin Wang
Dipti Bharadwaj
Edmund Hart
RobustAFT
Mian Huang
EcoSimR Aaron Ellison
biorxivr capushe
VarSelLCM
Wenliang Pan
Nick Gotelli
robustloggamma lga Jean−Luc Muralti MvBinary
speff2trial Bertrand Michel
V. J. Victor Cathy Maugis
Matthieu Marbac
robustvarComp
Francisco Herrera Triguero
RSNNS Jean−Patrick Baudry
Jose Manuel Benitez Sylvain Arlot
Stefan Van Aelst frbs
Peter J. Green Alex Randriamiharisoa Francisco Herrera
Rmalschains Lala Septem Riza
José M. Benítez
Javier Otegui RoughSets
Xiaomin Lu Chris Cornelis Chaitanya Khadilkar
Karen Schettlinger Fields Development Team
Ella Roelant William H. Swanson
Dominik Slezak
Daniel Molina Paul H. Artes
simr Claudio Agostinelli
Sebastian Stawicki Mitchell W. Dul
Andrzej Janusz
Neil O’Leary
d0uv ← max(G S )−duv
LOGICOIL SCORER2 visualFields
Catriona MacLeod
wle Victor E. Malinovsky
circular
Derek N. Woolfson
gimme
Shiquan Wu Richard Russell
ImageMetrics Hallie Pike
CircStats localdepth
Thomas L. Vincent
Craig T. Armstrong
T. E. C. Common Mathematical
Library
bipartite
Marco Chiarandini Peter Molenaar
Michal Juraska
Ulric Lund David H. Foster
modelfree
Ivan Marin−Franch
Rouven Strauss
Mario Romanazzi Kathleen Gates Nimisha Chaturvedi
igraphtosonia
Aaron Clauset
35:
Stephanie Lane Kamila Zychaluk
Sean J. Westwood
Jochen Fruend
NetCluster Nils Bluethgen Peter B. Gilbert
Diego Vazquez
NetData
triads
Miguel Rodriguez−Girones Jelle Goeman
Dan McFarland
Luis Paquete penalized
Mariano Devoto Jose Iriondo Aldo Solari
snipEM Dario Basso MIIVsem
flip
Rosa Meijer
cherry Zachary Fisher
CALF
Alessio Farcomeni
Ken Bollen
robustHD Livio Finos Fredrik Nilsson Marco Rinaldo robumeta
someKfwer
Matteo Favero Elizabeth Tipton
perry Diana Perkins
seqDesign
sparseLTSEigen
cmprskContin
Mike Nowak egonet Clark Jeffries
Bernd Gruber
cvTools
ccaPP
max(GS )−1
F. Gioachin
Doug Grove
x12
A. Sciandra
David Simcha Yanqing Sun
x12GUI
PopGenReport
Angelika Meraner Ian W. McKeague
NEff
Aaron Adamack
growthrate
Klaus Henle
msgl Xuesong Yu Annegret Grimm
Russell Horton
Andrew Gorman
FREQ
dist ← dist + d0uv
Sara Lopez−Pintado
MarkKay Belardinelli
Wunsch Jordan Killpack Jaakko Jarvi
Thomas Schreiber
Garrett Miller
Matthew Hokanson Conrad Sanderson RcmdrPlugin.depthTools
Casey Kolderup
Holger Kantz Yoondong Lee Kristof Coussement Yuchao Jiang Simone Ecker
Gianluca Gazzola Armin Graber
depthTools Matthias Bogaert rrdflibs
Ryan Freebern Giulio Barcaroli Jeremy J. Shen
Koen W. De Bock
Dankyu Yoon
K. Adam White Aurora Torrente
arrayImpute Canopy
SamplingStrata MADAM
PPtree gCat
Mark Sample Martin Vincent GAMens Egon Willighagen seqCBS
Sung Gon Yi AggregateR
36:
Patrick Rodriguez snpXpert PPtreeViz Hyeong−Seok Lim Daniela Pagliuca
Michael Dewberry Yoojin Jung gSeg
dummy
AUC genalg
asVPC Jie Ding
Nathaniel Mitchell Andrey Volkov Eva Budinska Matthias Wieser
Taesung Park fit4NM
rrdf Hao Chen Mueller
Nancy R. Zhang Czech Republic Ralf Bundschuh
David E. Frankhouser
kernelFactory
Karl G. KuglerLaurin
Gyujeong Noh
nCal ORMDR Michel Ballings lift
arrayMissPattern Eun−Kyung Lee PKmodelFinder Jeroen D’haen PSCN Guido Marcucci
Dirk Van den Poel Michael G. Schimek
lsgl aCRM Steven Hoornaert
Haipeng Xing Yunting Sun
sglOptim
falcon TopKLists
QuACN
hybridEnsemble MethylCapSig
ProbeR Michiel Van Herwegen Helmholtz Centre Munich Pearlly Yan
TERAplusB rotationForest leapp Michael Schutte
interpretR
imputeMissings
Junghyun Namkung GrammR
Michiel Vanherwegen
Vendula Svendova
Dauwe Vercamer Shili Lin Javkhlan−Ochir Ganbat
BootCL Michael G. SchimekEva Budinska
Matthias Dehmer Deepak N. Ayyala
Laurin A. J. Mueller
MinSeok Kwon bayesQR Matthijs Meire
Krisztian Sebestyen
imputeMDR Li Lee esaBcv
Art B. Owen
Samsun Sung
Kun Chen Jingshu Wang
Youyi Fong Keming Yu
DIME
Heebal Kim
BOG
Taeyoung Hwang
Sunggon Yi Rahim Al−Hamzawi
dblcens Dustin Potter
Taeheon Lee cate
kyotil Dries F. Benoit optismixture
sptm Wonil Chung
krm Cenny Taslim
emplik
Jincheol Park
PermuteNGS ELYP Abbasali Khalili
chngpt prc
Hera Y. He
Mai Zhou Qingyuan Zhao
Bo−Young Lee
Aelasticnet
CVThresh Yifan Yang
Saheli Datta
seismic
emplik2
Hee−Seok Oh kmc
EMD
Rahim AlHamzawi rankreg
Donghoh Kim
William H. Barton
SpherWave
Murat Erdogdu
SynchWave
Jin Zhezhen
end if
Dongik Jang
Eugene Brevdo
37:
38: end for accrual
Matthew S. Mayo
Yu Jiang
Cen Wu
coreTDT
rvTDT
Andrew S. Allen
JAGUAR
Chaitanya R. Acharya
audit
Glen D. Meeden
Radu Lazar
polyapost
rcdd
Daniel J. Eck
ump
gdor
aster2
Charles J. Geyer
TSHRC
trust
pooh
Peihua Qiu
Martin Bezener
mnspc
Jun Sheng
sROC
NPsimex
fANCOVA
Xiao−Feng Wang
ADPclust
decon
Yifan Ethan Xu
gb
Bin Wang
bda
miRada
spt
distfree.cr
EBglmnet
Zhiqiu Hu
Anhui Huang
Zhiquan Wang
sparseSEM
PAS
Rongcai Yang
Xiaodong Cai
EBEN
Shizhong Xu
Qishan Wang
NAM
Tiago Pimenta
Alencar Xavier
Katy Rainey
William Muir
bWGR
William Beavis
SoyNAM
Brian Diers
James Specht
Jacob Coleman
acmeR Robert Wolpert
Zhi Ouyang
bark
Merlise Clyde tglm
BAS
Alan F. Karr
Lawrence H. Cox
Quanli Wang
EditImputeCont
Hang J. Kim
Jerome P. Reiter
NPBayesImpute
HyunJi Kim
discretization
Jingchen Hu
Marta Sestelo
Nora M. Villanueva
FWDselect
npregfast
seq2R
Javier Roca−Pardinas
Artur Araujo
genSurv
TPmsm
prop.comb.RR
p3state.msm
Maria Alvarez
Artur Agostinho Araújo
smoothHR
Luís Meira−Machado
survivalBIV
Ana Moreira
Olivier Broennimann
Valeria Di Cola
Frank Breiner
Daniel Scherrer
Christophe Randin
Olivier Broenniman
Ruben Garcia Mateo
Antoine Guisan
Anne Dubuis
Dorothea Pio
ecospat
Blaise Petitpierre
Wim Hordijk
Julien Pottier
MigClim
Nicolas Salamin
Loic Pellissier
Robin Engler
biomod2
Damien Georges
Wilfried Thuiller
Centre Tecnològic de Nutrició
Albert Navarro
Universitat Autònoma de
Barcelona
complex.surv.dat.sim
survsim
David Moriña
radir
Manuel Higueras
Pedro Puig
hermite
María Oliveira
NPCirc
Alberto Rodríguez−Casal
Rosa M. Crujeiras
Jacobo de Uña−
DTDA
Carla Moreira
Geneviève Lefebvre
Juli Atherton
BACprior
BCEE
Denis Talbot
Jacques Brisson
Nathalie Vandal
InferenceSMR
Thierry Duchesne
gte
Radu V. Craiu
TwoStepCLogit
Daniel Fortin
Mohammad Hossein Dehghan
Sophie Baillargeon
39: end while
Reka Howard
fuzzyRankTests Yingbo Li
mcmc alphahull
Michael Littman
Susana Faria TP.idm
aster
TEEReg Daniel Manrique−Vallier
potts Joyee Ghosh
nice Rcapture stratification
Sample.Size Komei Fukuda
bcpmeta caribou
rNMF Beatriz Pateiro−Lopez
Leif T. Johnson
Jacobo de Una−Alvarez
Vanesa Balboa−Barreiro
Wei Jiang
alphashape3d Helene Crepeau Louis−Paul Rivest Serge Couturier
grade ssd
sgof Thomas Lafarge
Yulei Wang
Jiayang Sun
hcp
Junheng Ma
nFCA Irene Castro Conde
Stephen J. Ganocy
curvetest
Guo−Qiang Zhang
Zhongfa Zhang
Justin J. Wilkins Martin J. Lercher KuangNan Xiong granova MOCCA Joan Guàrdia saws Giambattista Toller empiricalBayes Kohta Ishikawa Gerald Quon Keurcien Luu Joseph Dunlop
Ron Keizer Benjamin Braasch
Armin Biere Maribel Peró
rateratio.test Catalina V. Anghel Quaid D. Morris
Rajen Piernikarczyk bpcp
exact2x2
PSAgraphics Troy D. Hanson
Brian A. Danielak Johann M. Kraus exactci
Marc Andre Daxer RcmdrPlugin.steepness Zahra Montazeri Blum Michael
James E. Helmreich Fabio Zottele
Combine Katalin Csillery
perm HighProbability Michael G. B. Blum
xpose4 RcmdrPlugin.EACSPIR PairTrading pcadapt
sybil ClimClass Francis Nguyen InPosition
Abdelmoneim Desouki Robert M. Pruzek David Leiva ISOpureR
William E. J. Doane BoolNet Lemaire Louisiane abc.data
ssanv Shrinkage David R. Bickel
GiANT SCVA Han de Vries Michael P. Fay boussinesq Daniele Andreis
xpose4data MChtest LTR prettyGraphs DistatisR Jean−Michel Becu
Dao Zhou soilwater Statomica
40: GI ← extractSubGraph(GS , I)
xpose4classic Martin Hopfensitz RcmdrPlugin.SCDA Alaa Ali
granovaGG Antonio Solanas Mike Fay
clpAPI steepness Michael Blum abc Csillery Katalin
binseqtest RGENERATEPREC Marta Padilla FRBData Amit G. Deshwar
E. Niclas Jonsson Christoph Müssel geotopbricks
Syed Haider
Mats O. Karlsson Gabriel Gelius−Dietrich interval hbim Emanuele Eccel Shinichi Takayanagi Paul C. Boutros Derek Beaton
cplexAPI Hans A. Kestler Florian Schmid SLC asht PsiHat
Joakim Nyberg Corey M. Yanofsky Francois Olivier Herve Abdi
Andrew C. Hooker PSAboot
Isis Bulte Patrick Onghena Emanuele Cordano bios2mds
KsPlot Cherise R. Chin Fatt Marie Chabbert
Claus Jonathan Fritzemeier RMRAINGEN Kyle Leckett TInPosition
BiTrinA choplump Olivier François
ncappc nivm
RcmdrPlugin.SLC Jenn Kirk LFDR.MLE Daryl M. Waggott
sybilSBML Binarize Interpol.T Issei Kurahashi bedr
multilevelPSA makeR TunePareto NanoStringNorm apTreeshape Louisiane Lemaire
xpose4genericxpose4specific SIMMS TExPosition
PopED glpkAPI
Eric Stroemberg Jason Bryer Rumen Manolov RGENERATE RMAWGEN
Deya Alzoubi Tamara J. Blätte SCMA RFinanceYJ ExPositionMExPosition
SCRT Nicolas Bortolussi Julien Pele
EasyUpliftTree Nobuaki Oshiro
blockmatrix Zuojing Li Jenny Rieck
Ludwig Lausser
Louis Luangkesorn RSeed TriMatch Ye Yang Emilie Lalonde
timeline Stefan Mundus RClimMAWGEN
NADA
Zhenyu Yang Clement Fung Michal Grzadkowski
Sebastian Ueckert Eric Durand
Lopaka Lee glpk
Marco Foracchia rHadoopClient
sqlutils RSearchYJ
likert Yohei Sato
Annalisa Di Piazza
ChangeAnomalyDetection
RWebMA
Kimberly Speerschneider EasyHTMLReport
YjdnJlp
Terrence Brooks skewtools Marco Tamburini ic.infer Lehrkamp Matthias Margret C. FuchsManfred Fischer Peter J. Galante Mario Samigli Fentaw Abegaz James J. Yang Guido Raos Minjeong Jeon Roger Guimera Rlinkedin
Abdolreza Mohammadi
Maria Uriarte
FrF2.catlg128 Sophia Rabe−Hesketh
FrF2 Riccardo Inchingolo
Patricio S. La Rosa
F gure 1 Authorsh p graph (subset)
Christoph Schmidt Rachel K. Smedley Mariano Soley−Guardia Michael Piccirilli
Berkley Shands
Anne Buu
relaimpo BDgraph SparseTSCGM L. Keoki Williams ChargeTransport
Skye Buckner−Petty DiscreteInverseWeibull mcll Guilhem Doulcier
Javier E. Contreras−Reyes orcutt EILA rnetcarto
Matthias Lehrkamp Luminescence ENMeval Rpdb
bingat Ulrike Groemping Benjamin Lauderdale
StressStrength Rfacebook
Stefano Spada DoE.wrapper Robert P. Anderson
Javier Contreras−Reyes Lenth Russ Markus Fuchs Sebastian Kreutzer
HMPTrees Matteo Quartagno Jamie M. Kass DiscreteLaplace
HMP
Aleksandar Radosavljevic
Ernst C. Wit Julien Idé
DiscreteWeibull Daniel B. Stouffer Kenneth Benoit
jomo RcmdrPlugin.DoE Robert A. Boria Cari Kaufman
Robert Muscarella Jia Li Tina Zhuo log10
afmtools LICORS Fox John Michael Dietze Alessandro Barbiero glassomix quanteda
William D. Shannon George M. Goerg Christoph Burow spThin Matthew Hutchinson Pablo Barberá streamR
Elena Deych
Mirko Signorelli
DoE.base Anani Lotsi lpbrim
Matthew E. Aiello−Lammens ForImp
James Carpenter pnea
GenOrd
ForeCA Paul Nulty
Lars Skjaerven
41: # add average distance to metrics
Pier Alda Ferrari SunterSampling bio3d bigGP
copas Rollin Thomas
Georg M. Goerg EMMAgeo Timothee Poisot
RLumShiny MetaDE paco Fernando Cagua
Wilfredo Palma Jan Odvarko Kohei Watanabe
Benjamin Lipshitz instaR
Actigraphy Hongquan Xu Bruno Vilela Xingbin Wang Christina Ludwig
Boyko Amarov Giancarlo Manzi
LambertW GenForImp Barry Grant
Jia Wang metasens
Veronica Vinciotti betalink Tiago Dantas
Xin−Qiu Yao
Elisabeth Dietze
LSTS Christopher Paciorek
LSC Guido Schwarzer meta George C. Tseng tightClust Juan Antonio Balbuena
Ricardo Olea AnalytixWare digitize
Wing H. Wong aLFQ Jonne Guyt
Carlos Gonzalez enRich AAindex
letsR
Gerta Rücker Nadia Solaro DWreg
MetaQC
Subharup Guha
Yanchun Bao
glmmGS
Pilar Rubio MetaPCA George Rosenberger
netmeta Hannes Roest
Fabricio Villalobos
Don Kang
Louise Ryan Michele Morara
Ulrike Krahn Jochem König
Douglas Galagate mxkssd Jose Manuel Soria Irina Mahlstein Luitgard A. M. Veraart chillR fpc tawny Qiong Ding Zhouwen Liu C. Alex Buerkle lshorth Asun Lubiano Mateusz Bruno−Kaminski
pan FactMixtAnalysis
pcg Jacopo Ripoldi F. Din−Houn Lau futile.paradigm Konrad Talik
Helena Brunel fractalrock
Mark Liniger Piotr Kowenzowski
mix Jihong Huang
Angel Martinez−Perez Gunther Sawitzki
causaldrf futile.matrix
Bernhard Hausdorf Stanislaw Jastrzebski
Rebecca Hiller prabclus lambda.tools SAFD
nnlasso Bedia Jimenez systemicrisk chopthin
Jun Ma seqRFLP spaa Igor Sieradzki
B. N. Mandal ifa introgress
mkssd Eike Luedeling Sari Acra PhysicalActivity
quantileDA Christian Hennig Charles E. Matthews
solarius Cinzia Viroli nonlinearTseries
Joseph L. Schafer Alvaro A. Novo Stephane Heritier Alfonso Buil easyVerification Brian Lee Yung Rowe Maciej S. Buchowski gmum.r
norm lambda.r
survivalMPL Maciej Zgliczynski
Alexandre Perera Matteo De Felice futile.options D. Olivieri
lmm extlasso Christoph Spirig
Axel Gandy Wolfgang Trutschnig Marcin Data
Constantino A. Garcia
smoothmest tawny.types Jinlong Zhang J. Dustin Tracy L. Rodriguez−Linares
decisionSupport futile Zongshan Li dcv PhyActBedRest Leena Choi Michal Pletty
ibd Jonas Bhend trimcluster HK80 Zachariah Gompert
mixtNB Jonas Kuppler
Andreas Weigel Karol Jurek
Claudia Mignani futile.logger Wojciech Czarnecki
Dominique−Laurent Couturier Andrey Ziyatdinov spcadjust futile.any
Jan Terje Kvaloy
cat RHRV
dynRB Robert R. Junker
Zhiyi Xu ProfileLikelihood A. Mendez M. Lado
Fernando Tusell
simctest
Lutz Göhring Elisabetta Bonafede phylotools Kong Y. Chen bayespref
Fernando Tusell Original Baidya Nath Mandal afc
chemosensors Kristin Tolksdorf
Samir Kanaan−Izquierdo Manuela Schreyer
cpca
Ted Harding A. Otero X. Vila
Georg Hahn
Xiangcheng Mi
42: addM etric(metrics, dist/|edges(GI )|)
Zachary Marion Arne C. Bathke
Gaj Vidmar Patrick Rubin−Delanchy Nancai Pei
Alexandre Perera−Lluna Katharina Schueller rriskDistributions James A. Fordyce
Nickolay T. Trendafilov
Natalia Belgorodski hierDiversity
ordiBreadth
HRM
mppa Matthias Flor iteRates
Matthias Greiner Harrar W. Solomon
rriskBayes Premal Shah
Benjamin Fitzpatrick
Martin Happ
Nicholas A. Heard
Alexander Engelhardt
Yoshimasa Tsuruoka Peter Fader David Zahrieh Wei Shangguan Wenchao Yang Justine Shults mcGlobaloptim Ulrich Bauer Gabriel Bucur ror Marie−Eve Beauchamp Andrew Jackson Francesco Favero Seung−Mo Hong Simon L. Rinderknecht
Yao Zhang Wiktor Zelazny Vincent Rouvreau Andrea M. Marquard
Ann Lazar Roberto Molinari
Elea McDonnell Feit Richard Gelber Nic Jelinski Jan Reininghaus SungHwan Kim
ycinterextra smaa Ryan Patrick Kyle
Tommi Tervonen
qlspack
Arun Gopalakrishnan Bogdan Rosca Hongzhe Li Brittany T. Fasy siar
maxent Stephane Guerrier SIBER sequenza
gmwm panelaggregation kaps wSVM
stepp Thierry Moudiki WCE fitDRC
BTYD Daniel McCarthy William Barcella soiltexture Jose Lucas Safanelli TDA Dmitriy Morozov hitandrun
Timothy P. Jurka Tim Jurka Aron Charles Eklund Tejal Joshi
Marco Bonetti Jichun Xie
utility
Gert van Valkenhoef OutlierDM Nele Schuwirth
Bruce Hardie Chiara Gigliarano Survgini Budiman Minasny Clement Maria beeswarm Soo−Heang Eo
MCLIME ESGtoolkit James Sweeney Bchron
gemtc
Eric Schwartz Rainer Petzold Jisu Kim Andrew Parnell Aron C. Eklund
Victoria Wang capme scio Marie−Pierre Sylvestre Michal Abrahamowicz
Edward Wadsworth Julien Moeys Wassim Youssef ESG Fabrizio Lecci Matthias Bannert timeseriesdb HyungJun Cho
Chip Cole James Balamuta Bclim stoichcalc
sentiment Lukasz Dziurzynski Rodolfo Marcondes Silva Souza Weidong Liu Michael Kerber traj
Matthias Bannert’ Peter Reichert
Jean−Charles Croix Ulf−Dietrich Reips
squash ecoval
msos gemtc.jar
cIRT simmr
Frédéric Planchet Joel Kuiper Dan Vatnik OutlierDC Simone Langhans
T. Tony Cai PermAlgo
jetset
ELT dropR Thinh Doan CPGchron
John Marden
RAdwords rivernet
visualize Steve Culpepper personograph Lan Wang
clime Thad Edens
ecosim
Julien Tomas
Iain Marshall
Huixia Judy Wang
Todd MacKenzie
Qiyuan Li
Johannes Burkhardt
43: ΦI ← 0
RCriteo
Arnaud Estoup Xiangrui Meng Zhidong Tu Developer
Leo Breiman Developer wahc Yuriy Tyshetskiy Stephen Hogg edtdbg Yimin Wu Marcelo Araya−Salas evora isva Bin Yao Nicole Heussen IRTShiny Anja von Heydebreck bisoreg Virginie Guemas
Javier Garcia−Serrano
Rdsm
snort Atsushi Mizumoto
autoencoder RCryptsy
S. Nonyane Fabian Lienert
CTTShiny Isabel Andreu−Burillo
Mark Norrie vabayelMix S. McKay Curtis
Eugene Dubossarsky mlica2 PCDSpline GitHub
SAENET Norm Matloff warbleR Andrew E. Teschendorff CorrMixed William Kyle Hamilton oncomodel
mirf s2dverification
Paneldata freqparcoord Chun Pan
partools Yang Zhang Dieter Hilgers Wim Van der Elst
Yingkang Xie
Grace Smith Vidaurre MAVIS
Jing Qian Ludovic Auger
Alex Rumbaugh Hua Zhong Nicolau Manubens Gil
Andrea S. Foulkes Kathleen Coburn mcmcplots
mlica Surrogate
B. Aletta Lianming Wang Geert Molenberghs RStars
gensemble FunChisq
matpow GenCAT DART ICBayes Christiane Heiss Martin Ménégoz
Ariel Alonso Chloé Prodhomme
EffectTreat Ilya Goldin
Bo Cai
Burak Aydin
Evangelos Evangelou
ICsurv
Eric Reed
Peter Werner Joe Song Geert Molenbergs
Yan Jiao
Jack Norman Sara Nuez Xiaoyan Lin intcox survBayes
Christopher S. McMahan Vivekanda Roy geoBayes
Ckmeans.1d.dp
DARTData
Volkmar Henschel Ulrich Mansmann
Vivekananda Roy
Haizhou Wang
graphComp
Khadija El Amrani
FKF Miron Bartosz Kursa RcmdrPlugin.MA Dariya Malyarenko Vroni Retzer fgac J. Cortés ncdf4.helpers U. S. Geological Survey Northern mded Patrick Wheatley Emmanuel Müller
Stephan Günnemann
support.BWS
44: for m ∈ metrics do
RcmdrPlugin.MAd G. Gómez Kenneth B. Hoehn Thomas Scherer
Maureen Tracy Ulrich Drepper
Witold Remigiusz Rudnicki PCICt Prairie Wildlife Research Center perspectev
Philipp Erb WMBrukerParser David Bronaugh
Simon Otziger Witold R. Rudnicki dcens Pacific Climate Impacts Glen A. Sargeant Timo Thoms Timm Jansen
Thomas Seidl
Xinyu Tang David Lüthi ghyp Wolfgang Breymann simba Consortium Hideo Aizaki
RcmdrPlugin.MAc Veronica Andrea Gonzalez−Lopez
M. V. Moneta wild1 subspace
Boruta A. C. Del Re GDELTtools
rTOFsPRO
bwsurvival support.CEs
MAc O. Julià
DTR
compute.es V. Moneta John Beieler
schwartz97 William Cooke climdex.pcic zyp
C. Serrat Research Center Ira Assent
DCchoice Marwan Hassani
Gerald Jurasinski V. A. Gonzalez−Lopez
Maria Melguizo LIStest S. G. S. Northern Prairie Wildlife ccgarch Tomoaki Nakatani
MAd William T. Hoyt Karl Kuschner J. E. Garcia Stephen R. Haptonstahl Matthias Hansen
Miron B. Kursa
rWMBAT
rtape
Juri Hinz rFerns Arelia Werner
Anke Guenther dils
Kazuo Sato
flux Franziska Koebsch
Jesus Garcia FastImputation
triggr Qian Si
Ulrike Hagemann
Sascha Beetz mlCopulaSelection
Veronica Gonzalez−Lopez
Angel M. Garcia Fernando Silveira Marques plsRglm Andrew Azman AllPossibleSpellings Jialiang Li Max D. Price Christian Gerber Amelia Simo Jane Lawrence Sumner Cory Barr sigma2tools distributions Caleb King
Fernando Ferreira Johannes Ransijn
Juan Domingo
I. W. K. Health Center Frank Klawonn
BioStatR Biocomb
IniStatR Myriam Maumy−Bertrand LMERConvenienceFunctions
poibin
Oswaldo Santos EpiDynamics disp2D
Cristobal J. Carmona Frederic Bertrand Antoine Tremblay ADDT
Frank Pessler HUM Natalia Novoselova Guillermo Ayala GRANBase Fabio Frascati
SDR coarseDataTools zooaRch enviPat Anthropometry interactionTest
capm Nicolas Meyer Jesse Wolfhagen M. Victoria Ibanez
ReorderCluster switchrGist
Yili Hong
icaOcularCorrection ISA agreement
Nicholas G. Reich
Marcos Amaku Justin Lessler
plsRcox Junxi Wang
Federico M. Stefanini
vacem Erik Otarola−Castillo Dean Adams Bruno Mario Cesana
clusterPower isopat cstar
plsRbeta Jessica Metcalf LCFdata Yimeng Xie
Irene Epifanio Gabriel Becker staRt
Pedro Gonzalez Oswaldo Santos Baquero NeuroCognitive Imaging Lab Guillermo Vinue clusterSEs
Justin Esarey SearchTrees
Martin Loos
nontargetData SPREDA
Daniel Obeng Francesco Corona
geomorph
fastdigest
enviPick
Bob Jenkins switchr
ggsn Michael Collyer
nontarget heatmapFit
Emma Sherratt Zhibing Xu
ΦI ← ΦI + wm ∗ metric
Andrew Pierce
Jericho Du
45:
Doug Bronson Hugh J. Devlin Ryota Suzuki fbati Noah Silbert Celia Touraine Hiroto Udagawa Alexey G. Bukhovets Ron N. Buliung Anna Heath ChungHa Sung T. Lloyd Adam Vizina Philip Kokic
SangGi Hong Stanislav Horacek
geoscale Huidong Jin
Mark A. Bell
Randy Bui spTDyn
2 https://www.bioconductor.org
aspace Tao Ding BCEA
Junghoon Lee RIGHT Jae W. Lee strap Petr Maca bilan Ladislav Kasparek
RIFS
Voss Graeme T. Lloyd
PatternClass Andrea Berardi Claddis
bmeta
Gianluca Baio
Pavel V. Moskalev Tarmo K. Remmel JongHyun Bae Martin Hanel K. Shuvo Bakar
TaeJoon Song T. G. Masaryk Water Research
Mark A. Bell Graeme
SPSL
SECP BCEs0
hdeco
spTimer
K. S. Bakar
Sandor Kabos
Sujit K. Sahu
Oana Almasan Emilio Letón rsggm MEWMA Alfonso Iodice D’ Enza Cathryn M. Lewis
msgps Michele Scagliarini
Carmen Cadarso−Suárez
Davide Buttarazzi idm Graham H. M. Goddard
MPCI
Mihaela Hedesiu MCUSUM
Kei Hirose
RcmdrPlugin.EBM GsymPoint
RcmdrPlugin.ROC Edgar Santos−Fernandez REGENT
Graham Goddard
clustrd
Daniel−Corneliu Leucuta Angelos Markos
Mónica López−Ratón
Elisa M. Molanes−López
caGUI
fanc Johnson MSQC
Andrei Achimas
Michel Van de Velden Daniel J. M. Crouch
OptimalCutpoints
RcmdrPlugin.coin
Haruhisa Nagata
Michio Yamamoto
46: end for
Maria Xose Rodriguez−Alvarez LEAPFrOG
obliclus
Michael E. Weale
3 https://cran.r-project.org accessed on June 2016
Alfred Galichon
Rearrangement
Ivan Fernandez−Val
Victor Chernozhukov
P. H. D. Wurzweiler School
Wendy Zeitlin
Charles Auerbach
SSDforR
P. H. D. Wendy Zeitlin Schudrich
Social Work
Justin J. Hendrick
Wei Lu
Anna L. Tyler
cape
Greg W. Carter
Vivek M. Philip
Juan Zalapa
Walter Salazar
Brandon Schlautman
FragmanGiovanny
Luis Diaz−Garcia
Covarrubias−Pazaran
Meredith McDonald
Vladimir Jojic
Derek S. Lundberg
MInt
Jeffery L. Dangl
Surojit Biswas
Hsin−Ta Wu
Vivian Hsiao
Benjamin Raphael
cometExactTest
Max Leiserson
Fabio Vandin
Nick Mihailowski
Vignesh Prajapati
Kushan Shah
RGoogleAnalytics
Michael Pearmain
Nicolas Remy
Kerrie Mengersen
Udayanga Attanayake
sesem
Eric Lamb
Katherine Stewart
Steven Siciliano
Francial G. Libengu
C. Kokonendji
TRIANGG
Silvio S. Zocchi
TRIANG
Tristan Senga Kiess
Gang Wu Bernard Fichet Jean−Michel Poggi Johnny Villalobos kitagawa David Lorenz Ricardo de Matos Simoes
Karl Kornacker Jean Gaudart Oldemar Rodriguez
kelvin
bc3net
John Hogenesch Guillaume Barbet Jorge Arce Andrew J. Barbour
MetaCycle SPODT RSDA dataRetrieval
VSURF
Christine Tuleau−Malot Frank Emmert−Streib
Robin Genuer
Ron Anafi Roch Giorgi Roberto Zuniga Laura DeCicco
c3net
Michael Hughes Nathalie Graffeo Olger Calderon psd
rchallenge Robert Hirsch
David Myer Gokmen Altay
Adrien Todeschini
EGRET
dc3net
Robert L. Parker EGRETci
usdm imputation Antanas Marcelionis tsbugs CORE ParDNAcopy Ana Paula Assis Compind
nutshell fanplot Edgar Zanella Elisa Fusco
fastVAR
rts
Joseph Adler Babak Naimi Guy J. Abel Alex KrasnitzGuoli Sun Diogo Melo
Jeffrey Wong Francesco Vidoli
LargeRegression rAmCharts evolqg
Jeffery Petit
Benoit Thieurmel ssfa
visNetwork
semsfa
tsbridge
nutshell.audioscrobbler migest Guilherme Garcia
TimeProjection CNprep Giancarlo Ferrara
TBEST
Sebastien Vigroux
seasonal Johannes Hoehne
Peter Steiner Peter Nightingale
Christoph Sax
tempdisagg
seroincidence CAMAN
Daniel Lewandowski Maryna Verba
Peter Teunis Peter Schlattmann
D. Sculley Jeffrey Chrabaszcz bayesGDS
Michael Bockmayr
ionflows
RSofia gemmR Michael Braun
Ludwig Hothorn Fernando Cela Diaz Joe Tidwell
Michael King Michael Dougherty sparseMVN
EloChoice catmap M3
Kristen Foley
Jenise Swall
aqfig
Kristin K. Nicodemus
J. Kyle Roberts
Augustin Luna Kim Nimon
Rfun Ben Murrell NestedCohort gamboostMSM
Herve Cardot
onlinePCA
Hormuzd A. Katki
matie
Dan Murrell simMSM
CompareTests
Hugh Murrell David W. Edelstein
equate mvmeta cloudRmpi Delaporte SimCorMultRes
Ben Armstrong
Anthony Albano Antonio Gasparrini
Rwinsteps dlnm
Barnet Wagman
Pade
rreval
Xianyan Chen sspline Xiang−Yun Wu Xiang Zhan
NlcOptim ProNet KMDA
Xianhong Xie
Xiangrong Yin Xia−Yu Xia Debashis Ghosh
rgcvpack
Todd R. Jones Tobias Kley Tirthankar Dasgupta Tin−Yu Hui
Vanderlei Julio Debastiani Uwe Menzel TszKin Julian Chan Tong Zhang Tomas William Fitzgerald Thorsten Pohlert Thomas Lin Pedersen
Nathan Stephens Peter Nash Peter Hussey Sivasish Sindiri Pierre−Jerome Bergeron Mikhail Pogorelyy Nico Nagelkerke Hatto von Hatzfeld G. Brian Golding Perry D. Moerland ActiveQuant Gmb Joerg Helms M. Teresa Buglielli Elisabeth Waldmann Erik Andersen Erik R. Thomas Di Liu Lars Kai Hansen’s Paolo Tormene Janez Demsar Andrzej Bak John R. Stevens Stefan Birr Joseph J. Lee Imperial College London Robert Ackland Jacob Anhoej Jairo Cugliari J. Song Florian Meinfelder Pierre−Yves Boelle
SYNCSA EMT ezsim muscor ADM3 trend densityClust
Sven Ove Samuelsen Sven Knueppel Susie Jentoft Susan Holmes Sulc Zdenek Sudeep Srivastava Subhadeep Mukhopadhyay Stuart Baumann cthresh mlsjunkgen Stephen W. Erickson Tmisc Stephane Jankowski Stephan Gade Sten Ilmjarv gRcox sde Stefano Galelli Stefanie Kalus Stefan Zeugner bride Stefan Moeding Stack Overflow gcl Srivenkatesh Gandhi Sriharsha Veeramachaneni Soren Hojsgaard Sophie Arnaud−Haond Sonia Amodio Song Yang Sohrab Shah Snigdhansu Chatterjee Snaebjorn Palsson Simone Padoan Simon Mueller Simon Barthelme
AICcmodavg TideTables vegetarian multipleNCC HapEstXXR RcmdrPlugin.sampling distory nomclust GPseq LPTime schumaker SNPMClust MultNonParam RPPanalyzer demi reservoir RfmriVC BMS usl install.load DataLoader LinearizedSVR mhsmm RClone ConsRank YPmodel xseq WiSEBoot shapeR CompRandFld nfda imager
Stuart Barber Steve Myles Stephen Turner Steffen L. Lauritzen Stefano Maria Iacus Stefan Siegert Staal A. Vinterbo
Marc J. Mazerolle Moritz Mueller−Navarra Noah Charney Nathalie C. Stoer Klaus Rohde Johan Heldal John Chakerian Rezankova Hana Liang Chen Shinjini Nandi Margaryta Klymak Joshua Callaway John E. Kolassa Heiko Mannsperger Hendrik Luuk Sean Turner Ludwig Bothmann Martin Feldkircher Neil J. Gunther Irucka Embry Kreshnaa Raam S. Bethusamy Ken Williams Jared O’Connell Diane Bailleul Antonio D’Ambrosio Junlong Sun Jiarui Ding Megan Heyman Lisa Anne Libungan Moreno Bevilacqua Simon Mller Antoine Cecchi
waveband phonenumber qqman gRc opefimor SpecsVerification PrivateLR
Roberta Andrade de Araujo
Samuel G. Fadel Samuel Balmand Samantha Fernandez Salvatore Grimaldi Sairah Y. Malkin Sai Teja Ranuva zicounts S. Demeyer dendsort Ryan Sermas mchof Ryan C. Graebner mcparallelDo Rudradev Sengupta CatDyn Rong Pan Romain Frelat Roland Pfister Rohit Kumar Patra spanr Roeland Voorrips gstudio Lock5Data ConConPiWiFun Roberto Gatta Robert Stojnic Robert Jennrich Robert Ferstl Robert E. Fay Robert Castelo Robert Bruggner Robert A. Klopotek lhs QTLRel Risk Advisory Group Richard Zijdeman Richard Telford Richard J. Cook
Fagundes
pcaL1 GFMaps mp DESP TSTr LPM phytotools AnDE ATmet ChoiceModelR GeneticSubsetter IntegratedJM ALTopt ndvits schoRsch simest fitTetra spatialfil ISDA.R sequences GPArotation termstrc sae2 qp Rclusterpp regRSM its lar palaeoSig
S. M. Mwalili Ryo Sakai Ryan Grannell Russell S. Pierce Ruben H. Roa−Ureta Roger Marshall Rodney J. Dyer Robin Lock Robin Girard Rob Carnell Riyan Cheng
Ricardo Jorge de Almeida
Paul Brooks Kanika Arora Francisco M. Fatore Arnak Dalalyan Ricardo Merino Corrado Tallerini Greg M. Silsbe Nayyar Zaidi A. Allard John V. Colias Alfonso Cuesta−Marcos Nolen Joy Perualila Kangwon Seo Bruno Gerard Markus Janczyk Arun Kumar Kuchibhotla Gerrit Gort Nicola Dinapoli Laurent Gatto Coen Bernaards Josef Hayden Mamadou Diallo Alberto Roverato Michael Linderman Pawel Teisseyre Commerzbank Securities Josemiguel Lana−Berasain Mathias Trachsel
Queiroz Filho
baymvb gapmap needy listWithDefaults LifeHist srd popgraph Stat2Data VHDClassification triangle qtlmt
Rachael Maltiel R. Pelissier R. Doug Martin R. Barfield Przemyslaw Spurek Pingping Qu provenance Pieter Thijs Eendebak pnn Piergiorgio Palla Philip L. H. Yu devEMF Petri Toronen Peter X. K. Song normwhn.test PsumtSim Peter Macdonald bifactorial Peter Bogetoft Pengfei Li numOSL
Pedro−Pablo Garrido Abenza Pavel Michna CompetingRiskFrailty WMTregions Paul W. Eslinger Paul Fischer Paul Evangelista Paul Cool Paul Burton Patrick Kraft JGL Pascal Kerschke pch Pan Tong Pablo Tamayo Pablo Montero Manso Pablo Emilio Verde peakPick P. Wijekoon Ozlem Alpu
rafalib CpGassoc CEC greyzoneSurv oapackage RFmarkerDetector pmr mGSZ metafuse mixdist Benchmarking MixtureInf rplotengine RNetCDF SBRect pointdensityP DeducerSurvival ESPRESSO ArfimaMLM flacco PANDA RODM TSclust bamdit lrmest ltsbase
Pieter Vermeesch Pierre−Olivier Chasset Philip Johnson Peter Wickham Peter N. Steinmetz Peter Frommolt Peng Jun Pavel Khomski Pavel Bazovkin Patrick Danaher Paolo Frumento Paal O. Westermark
Michael I. Love K. Conneely Konrad Kamieniecki John Crowley P. T. Eendebak Giuliano Armano Paul H. Lee Pashupati Mishra Lu Tang Juan Du Lars Otto Jiahua Chen P. Pablo Garrido Milton Woods Heather Orr Astrid Hilbert Dave Beskow Matthew Ockendon Amadou Gaye Christopher Weber Jan Dagefoerde Hua Li Ari Mozes José Antonio Vilar Arnold Sykosch Ajith Dissanayake Betul Kan Kilinc
ArArRedux grnn adaptivetau normwn.test EffectsRelBaseline ic50 RadialPlotter TwoWaySurvival StochaTR SETPath flexPM HarmonicRegression
PurBayes Nial Friel svdvis Neil R. Clark wBoot Neha Rohatgi Ned Horning Nay John Nathan Medina−Rodriguez Natalie Koziol Natacha Nikolic Nadezda Frolova Murilo Coutinho Silva Mounir Aout Morten Moshagen Monique Graf Mingkun Chen fmsb LSMonteCarlo Mike Higgins Miguel Munoz Zuniga fsia Michel Wedel Michaela Dvorzak bcrm Michael Stewart Michael Sieger Michael Sandfort Michael Mahr Michael Gras Michael Grabchak Michael Eichenberger qlcMatrix Michael Bedward Melvin M. Varughese Maximilian Held Mauro Bernardi Matthias Wangler Matthias Kahm nadiv Matthew Salganik
extremogram ClustVarLV elec.strat GPC waved VisuClust antitrust Rchemcpp cuttlefish.model EntropyEstimation multiAssetOptions packcircles DiffusionRgqd qmethod MCS changeLOS grofit
Nicholas B. Larson Neo Christopher Chung Neil A. Weiss Minato Nakazawa Mikhail A. Beketov Michela Battauz Michael Sweeting Michael Cysouw Matthew Wolak
Ivor Cribben Evelyne Vigneau Hua Yang Jordan Ko Marc Raimondo Georg Ohmayer Charles Taragin Guenter Klambauer Jean−Paul Robin Lijuan Cao Carlo Rosa David Eppstein Etienne A. D. Pienaar Aiora Zabala Leopoldo Catania Jan Beyersmann Maik Kschischo
jackstraw wPerm pyramid HMMCont equateIRT pipe.design qlcVisualize ICC
Marieke van der Maria Mercedes
Mario Cortina−Borja Marie Paturel Maria L. Rizzo Maria Karlsson Maria Iannario Maria Amalia Jacome Margret−Ruth Oelker SWMPr Marco Smolla Marco Munda pawacc Marcin Studnicki Marcello Chiodi Manuela Azevedo Manuel Truppia cooptrees Majid Sarmad− paramlink Madeleine Thompson PKtools M. G. M. van Loon M. F. Kelly M. F. Carfora López−Moliner Joan Lyamine Hedjazi Luz Marina Rondon elec DMwR sglasso Lucia Tamburino Lucia Spangenberg Luca Weihs Luca Pozzi Luca Agnelli ReliabilityTheory Lothar Reichel Lorenzo Mercuri Lixi Yu
Maaten−Theunissen Gregorio−Dominguez
funreg phyloland CircNNTSR survPresmooth gvcm.cat ccChooser FLEDA tvm CircOutlier GaDiFPT quickpsy mQTL BayesGESM frt bcool TauStar ARAMIS FBN irlba MixedTS QualInt
Marcus W. Beck Marco Geraci Manuel Fontenla Magnus Dehli Vigeland M. Suzette Blanchard Luke Miratrix Luis Torgo Luigi Augugliaro Louis Aslett
John Dziak Louis Ranjard Juan Jose Fernandez−Duran Ignacio Lopez−de−Ullibarri Margret Oelker Konrad Debski Ernesto Jardim Juan Manuel Truppia Azade Ghazanfarihesari A. Buonocore Linares Daniel Jean−Baptiste Cazier Heleno Bolfarine Giangiacomo Bravo Hugo Naya Emin Martinian Antonietta Mira Adrian Andronache Jim Baglama Edit Rroji
NeuralNetTools lqmm optrees IBDsim TEQR textreg performanceEstimation dglars PhaseType
Konstantin Sering svs Kevin P. Barry Kevin Chang Kevin Arbuckle Kenneth Buker Ken Kelley titecrm MMST hglasso MOJOV Kaisa Välimäki Jungsik Noh Julian Wolfson Juan Luis García−Castaño Juan David Velasquez Josue Moises Polanco−Martinez Josselin Noirel fdrci interferenceCI
Jose A. Castellanos−Garzon Jorge Gonzalez Burgos Jonas Stein catspec RSelenium John Bunge John Braun Johannes Zimmermann Johan Segers Joel Tarning Jodavid Ferreira metansue Joan Maynou Jingqin Luo Jing Tao CoxPlus LearnEDA CIFsmry Jianan Tian Jesse Garrison Jens Keilwagen Jenny Häggström
Dominance infoDecompuTE MBESS Rhh W2CWM2C breakaway CHsharp BacArena spatialTailDep PharmPow waldwolf MEET DiagTest3Grp Depela MaXact audiolyzR PRROC
Koen Plevoets Ken Cheung Keith Halbert Kean Ming Tan Ke−Hao Wu Joshua Millstein Joseph Rigdon John Hendrickx John Harrison Joaquim Radua Jing Peng Jim Albert Jianing Li
Knut Krueger Katya Ruggiero Keke Lai Jussi Alho Josue M. Polanco−Martinez Amy Willis Douglas G. Woolford Eugen Bauer Anna Kiriliouk Frank Kloprogge Claudio Souza Erola Pairo Chengjie Xiong Andrew C. Chou Chenliang Xu Eric Stone Jan Grau
corregp dfcrm crackR sparseBC RYoudaoTranslate cit RI2by2 perturb rDVR occ PanelCount LearnBayes crskdiag
J. M. Calabrese Ioannis Ntzoufras Ioannis N. Athanasiadis Inmaculada Pérez−Bernabé Ian Donaldson Héctor Villalobos invGauss Hyun Jung Park Gordon J. Ross paleofire Giorgio Vacchiano Giorgio Arcara Gilda Garibotti German Aneiros Perez Gerhard Schöfl’ mgraph TFMPvalue Gary Anderson G. Roma AnthropMMD Frederic Commo Frank A. Schmid phylosignal FAmle Francis Roy−Desrosiers
edgar cpm Rothermel erpR nltm PLRModels reutils AMA HumMeth27QCReport nplr lossDev
Håkon K. Gjessing Global Paleofire Working Group George Owusu Ge Tan Frederic Santos Francois Keck Francois Aucoin
Bharat Patil G. N. U. Scientific Library Davide Ascoli Anna Petrova Alexander Tsodikov Ana Lopez Cheda Gerhard Schöfl Aneesh Raghunandan F. M. Mancuso Brian M. Bot Christopher W. Laws
Haplin GCD qrfactor TKF KappaGUI rleafmap FAdist
ringscale Constantinos Tsirogiannis Clive Loader Claudio A. Vasconcelos Claudia Martinez−Araneda Clare Buckeridge gooJSON Christophe Demattei pARccs nets Christian M. Hafner ddepn LogisticDx dae HighDimOut surveyeditor Chao Ye apt ChangJiang Xu
georob PhyloMeasures locfit iRegression dgmb PKNCA spatclus covLCA dslice GWsignif
Daniel Haase Christopher Steven Marcum Christiane Raemsch Christian T. Brownlees Christian Bender Chris Dardis Chris Brien Cheng Fan Char Leung Changyou Sun
Brody Sandel Catherine Loader A. Lima Neto Alba Martinez−Ruiz Bill Denney Christophe Dematte Aurelie Bertrand Bo Jiang
cccd anim.plots halp informR PARccs dynamo bootfs survMisc asremlPlus TSMining ComplexAnalysis erer
Xiuquan Wang Xiaoyao Yang Xiaomeng Ni Xiaolin Zhang Xiaohong Li Xiao Rong Xiao Lei Xiangwei Zhu Xavier Fernández Wouter D. Weeda Wouter BuytaertWolfgang Viechtbauer
Wolfgang PoessneckerWobbe P. Zijlstra Wim de Leeuw William Nicholson William Hughes William G. Jacoby
Willem Daniel Schutte Will Lowe Will Kurt Westa Domanova Wei E. Liang Wayne Oldford Wayne Jones Waqas Ahmed MalikWacek Kusnierczyk W. J. Braun Volkmar Liebscher Vladislav Navel Vitaly Efremov Vipavee Trivittayasil Vinh Nguyen Vincent Nijs Vincent CalcagnoVijay Krishnamurthy Victoria N. Nyaga
GeneF rSCA freestats SPMS RDataCanvas malaria.em deepnet RDota mixtox ggmcmc arf3DS4 topmodel metafor MRSP fwdmsa SigWinR DTMCPack MConjoint optiscale SOPIE events ScrabbleScore ksrlive hiPOD qqtest R2PPT vmv rbenchmark MPV gromovlab timetools stheoreme EEM inference radiant glmulti Covpath CopulaDTA
Steve Kalke Steve Juggins Steve Buyske Steve Bronder Stephen Oswald Stephen Meyers Stephen L. R. Ellison
Stephen Horng−Twu Lihn
Stephen A. Sefick Jr Stephanie Locke
Stephane Mikael Bottine Steinar Engen Steffen Moritz Steffen Kothe Steffen Greilich Stefano Meschiari Stefano Benati Stefanie Hieke Stefan Hengl Stefan Boehringer Stefan Behrendt
Stavroula A. Chrysanthopoulou
Statistics Norway Stanley E. Lazic Stan Yip Stamatis Kalogirou Sophie Ancelet Song Cai Sonar Inc Sofia Morfopoulou
Soeren Havelund Welling
Soeren Braehmer Slawomir Jarek Sina Rueeger Simulistics Ltd Simon Thornley Simon Spencer Simon Schwab Simon Guest Simon Bond Siew−Leng Teng Siakoulis Vasileios Shuichi Kawano Shuangcai Wang
cabootcrs clusterCons NHEMOtree pgnorm rioja mmlcr PANICr FlexParamCurve astrochron metRology LIHNPSD StreamMetabolism optiRum SPRT poilog imputeTS cmsaf libamtrack latex2exp qVarSel minPtest mota parallelize.dynamic lm.beta MILC RcmdrPlugin.EHESsampling desiR weatherr lctools Geneclust drmdel jSonarR metaMix forestFloor relen mvnormtest uniPlot Simile PubBias interventionalDBN emov OpenMPController mreg knorm acp spcr pbs
Roman Pahl Roman Jugai Roman Guchenko Romain Azais Roland Tanglao Rohit Arora Roger Woods Rodrigo Buhler Rodrigo Azuero Melo Rodrigo Aluizio Roberto ImpicciatoreRobert P. Bronaugh Robert Maier Robert Lowe Robert J. Gray Robert Hable Robert G. Garrett Robert Clements Rob Erhardt Rob Crouchley Rick Pechter Richie Cotton Richard Bourgon Richard Ambler Richard A. Bilonick Rich Nielsen Riadh ZaatourRenee Gonzalez Guzman
Reid F. Thompson Recai Yucel Raul Cruz−Cano Rasmus Bååth Raphael W. Majeed Raphael Mourad Randall Shane Ram Narasimhan Ralf Strobl Rahul Premraj Rahul Mehta Rahim Alhamzawi Rafael Dellen R. N. Edmondson R. Lafosse R. Hackathon R. H. BaayenR. C. S. Soil Survey Staff Qunhua Li Quinn N. Lathrop Qiao Kang Pros Naval Po−Hsien Huang Po Su Piter Bijma Piers Harding Pierre Lefeuvre Pierre Alquier Philippe Courcoux Philipp Limbourg Philipp Hermann Philipp Angerer Philip C. Schouten
Phil Novack−Gottshall Phil Joubert Petr Novak Petr Matousu Peter Xenopoulos Peter Lipman Peter Kampstra Peter Dutton Peter Carbonetto Peter Biber
GroupSeq anametrix rodd EstSimPDMP
ig.vancouver.2014.topcolour covmat icapca analyz gmapsdistance forams MAPLES phalen attfad rvmbinary permax imprProbEst rgr stppResid ABCExtremes sabreR MicroStrategyR learningr intervals trotter merror caseMatch hawkes NORRRM RadOnc mlmmm FRCC beepr violinmplot HiCfeat geoPlot weatherData km.ci mailR RDIDQ Brq PepPrep blocksdesign concor phylobase languageR sharpshootR idr cacIRT treeperm mopsocd lsl NORTARA SE.IGE RSAP BoSSA ISBF FreeSortR ipptoolboxFractalParameterEstimationnbconvertR Table1Heatmap ecospace SmithWilsonYieldCurve SeqGrapheR sievetest Sabermetrics CGene beanplot EurosarcBayes varbvs cycloids
Oliver Stirrup Oleh Komashko Oleg Yegorov Nolan A. Wages Noemie Robil Noemi Andor Noboru Nomura Noah Silverman Niraj Poudyal Nils Arrigo Nikolaos Giallousis Nikola Kasprikova
Nicole Mee−Hyaang Jinn
Nicolas Robette Nicolas E. Campione Nicolas Baskiotis Nick Kennedy Nick Guy Nick Bond Nicholas Vinen Neville Jackson Nelson Ray Neil Diamond Neal Richardson
Nathaniel Malachi HallinanNathan WelchNathan S. Watson−Haigh
Nathan G. Swenson Nathan Esau Nathan Carroll Natalya Pya Nadine SchoeneN. Benjamin Erichson Myles English Murzintcev Nikita Mun−Gwan Hong Muhammad Kashif Moritz BergerMonica Fernanda Monica
Rinaldi Calderon−Santiago
Mohsen Ahmadi Mohit Dayal Mohamed SoueidattMizanur Khondoker Miroslav Morhac Miriam Marusiakova Mini Huang Ming Wang Milan Hiersche Mikko Rönkkö Miki Horiguchi Mike Wurm Mike W.−L. Cheung Mike Malecki Mihai Tivadar Miguel A. R. Manese Michele Grassi Michele De Meo Michel Semenou Michel Prombo Michel Meulders Michel Crucifix Michel Berkelaar Michal Burda Michal Bojanowski Michael Silva Michael Scholz
Michael Salter−Townshend
Michael S. Pratte Michael Rustler Michael North Michael Netzer Michael Matschiner Michael LundholmMichael LaimighoferMichael J. Grayling Michael Gutkin Michael G. CampanaMichael E. Schaffer Michael Cyosuw Michael Chajewski Michael Buecker Michael Anderson Micha Sammeth Meryam Krit Menachem Fromer Melanie Wilson Melanie Quintana Meik Michalke Mehrad Mahmoudian Maxime Wack Maxim Yurchuk Max Sommerfeld Max Hughes Max Conway Mauro Sereno Maurits Kaptein
covBM nlWaldTest rlm pocrm KANT expands mvprpb makeProject StVAR R2G2 Scale LTPDvar severity GDAtools MASSTIMATE TreeRank forestmodel radar hydrostats uncompress dmm filterviewR VizCompX crunch HyPhy brewdata PCIT lefse m4fe oglmx scam NPMPM rsvd hydrogeo ldatuning MDimNormn COMBIA DIFtree PROTOLIDAR MScombine SimuChemPC cepp Funclustering optBiomarker Peaks forensic MSIseq geesmv postgwas matrixpls SSRMST ordinalNet metaSEM Stack OasisR SQLiteDF CircE bethel CompR OutrankingTools plfm palinsol lpSolve lfl intergraph blsAPI clickstream VBLPCM hbmem kwb.hantush SchemaOnRead BiomarkeR ageprior sifds SurvRank phaseR SlimPLS corrsieve rtf qlcData rela lcda bdoc ops EWGoF xhmmScripts MISA BVS XiMpLe varhandle cosmosR CfEstimateQuantiles cope adaptsmoFMRI gsheet lira RStorm
Marius Wirths Mario Quaranta Mario Pineda−Krch Marina Evangelou Marek Spruzina Marek Molas Marcus Scherl Marcus Rowcliffe Marcus G. Daniels Marco Torchiano Marco Lugo Marco Giordan Marco D. Visser Marco AndrelloMarcelo Goulart Correia
Marcelo C. PereiraMarcelo Brutti Righi Marcello D’Orazio
Marc VandemeulebroeckeMarc Johannes Marc J. Lajeunesse Marc Fortier Marc Fasel Marc Delord Marc Adams Maogui Hu Manel Salamero Mandy Vogel Majid Einian
Mahmoud Mosalman Yazdi
Mahmoud Ghandi Magdalena Chrapek Maciek Sykulski Maarten Kruijver M. Shahidul Islam M. Sawada M. Kondrin M. D. Vreeburg Lutz Prechelt Lutz P. Breitling Luke Jostins Lukasz Nieweglowski Lukas Burk Luis Carvalho Lucas Venezian PovoaLucas Leemann Luca Sartore Luc Wouters Luana Cecilia Meireles Loïc Schwaller Lora Murphy Lisa McFerrin Lionel Guy Ling−Yun Wu Lindsay V. Clark Lindsay P. Scheef Libo Sun Liang Liu Liang Jing Lianbo Yu Liam D. Bailey LiLin−Yin Li Qinglong Li Hua Yue Lev Kuznetsov Letaw Alathea Leonardo Di Donato
Leonardo Castelo Branco Leo Guelman Leeyoung Park Lee Kelvin Leah R. Jager Lauri Mehtatalo Laura Chihara Larsen Kim Lars Gidskehaug Lam Opal HuangLaercio Junio da SilvaL. W. Pembleton L. Andries Kyle Bittinger Kyle A. Caudle Kurt Wollenberg Kuangnan Xiong Krzysztof Ciupke Kristina Veljkovic Kristian E. Markon Kobi Perl Kiwamu Ishikura Kitty Lo Kiran Garimella Kimberly F. McManus Kevin Toohey Kevin R. Sanft Kevin Kokot Kevin Keenan Kevin Dunn
OneArmPhaseTwoStudy SetMethods GillespieSSA PAGWAS b6e6rl HGLMMM validator activity hdf5 effsize CANSIM2R ber aprof ConnMatToolsRcmdrPlugin.RMTCJags LSDinterface riskR StatMatch adaptTest pathClass metagear mfblock STI MIICD lm.br ibeemd TestScorer childsds BCDating topsis gkmSVM Records rpca DNAprofiles WaveCD MATTOOLS RGrace SocrataR agsemisc dagR Mangrove clv tRakt kolmim precintcon StratSel spMC mpm biasbetareg saturnin likelihood HDMD genoPlotR CRF polysat MAR1 ORCI phybase geoCount fmt climwin CMplot StatMethRank branchLars injectoR captioner hrr phenability uplift IFP astro phitest lmfor CarletonStats Information beadarrayMSV TRD laercio StAMPP mokken qiimer flowfield aaMI roughrf mpcv MetFns infutil mHG Rjpstatdb RAPIDR gsalib popRange SimilarityMeasures StochKit2R LRcontrast diveRsity pid
Josemir Neves Josef Brejcha Jose M. Pavia Jose Jimenez Jose E. Lozano Alonso Jose D. Loera Jos Feys Jort de Vreeze Jorge N. Tendeiro Jordie Croteau Jordan Mackie Jonas Mueller Jonas Klasen Jonas Haslbeck Jon Lefcheck John Whalen John Waller John Shea John R. Dixon John Fieberg John Ferguson John Curtin John Boik Johannes Radinger Johannes Hüsing Johann Laurent
Johann Gagnon−BartschJoerg Schaber Joel Gombin Joel Carlson Joe Sexton Jochen Knaus Joao Pinelo Silva Joachim Zuckarelli Jo Hardin Jiri Kadlec Jinyoung Yang Jingfan Sun Jimmy Oh Jim Yi Jiehuan Sun Jiayi Liu Jiat Chow Tan Jiang Li Ji−Ping Wang
Jesus Maria Rodriguez Rodriguez
Jesse Krijthe Jerome Collet Jeremy Thoms Hetzel Jeremy Stanley Jeremy Oakley Jens J. Rogmann Jennifer Kirk Jenine K. Harris Jelena Kovacic Jeffrey M. Dick Jeffrey H. Gove Jeffrey Dunn Jeff Binder Jedrzej S. Bojanowski Jed Long Jean−Pierre Rossi
Jean−Francois CoeurjollyJean V. Adams Jean Sanderson Jaynal Abedin Jayanth Varma Javier Hidalgo CarrioJavier G. Corripio
Javier Fernandez−Macho
Javier Celigueta Muoz Jason Wilson Jason W. Morgan Jason D. Nielsen Jared Murray Jan Wolfertz Jan Ulbricht Jan Saputra Mueller Jan Michael Yap Jan Klaschka Jan Gorecki Jan Gertheiss Jan Dul Jan C. Thiele Jamie Sergeant James Ridgway James P. HowardJames Holland Jones James Hiebert James Grange James Eustace Jalpa Joshi Dave Jakub Stoklosa Jairo A. Fuquene Jaime Egido Jae H. Kim Jack G. Gambino
spi BivarP GoFKernel abodOutlier mem Sofi npIntFactRep apaStyle PerFit fat2Lpoly cycleRtools plmDE hit mgm piecewiseSEM walkscoreAPI EasyMARK knitLatex gbm2sas SightabilityModel averisk lmSupport mixlow fishmove resper dashboard ruv pheno qualvar radiomics gbev snowfall OptiQuantR quantification biwt WaterML atmcmc powerr TableToLongForm SUE SQDA CRAC probemod DOSim SPECIES tablaxlsx Rtsne subrank trapezoid tidyjson SHELF orddom rmac ergmharris iRepro CHNOSZ sampSurf compoisson bursts sirad wildlifeDI rich dvfBm LW1949 adwave edeR jrvFinance allanvar insol wavemulcor PlotPrjNetworks PCS boolean3 crimCV bfa AlgebraicHaploPackage lqa rdetools WCQ BlakerCI Rbitcoin ordPens NCA RNetLogo RII EPGLM waterfall demogR udunits2 trimr MSVAR RGoogleAnalyticsPremium PL.popN ClinicalRobustPriors dynBiplotGUI vrtest pps
10000
10000
500
500
1000
1000
Num Clusters
Num Users
Num Users
Num Users
50
100
100
50
5 10
10
10
5
1
1
1
1
5 10 50 500 5000 1 2 5 10 20 50 100 1 2 5 10 20 1 2 5 10 20 50
Num Nodes Degree Num Views Num Packages
Figure 2. Clusters. Figure 3. Degree GS . Figure 4. Views. Figure 5. Packages.
In Fig. 1, only a subset is shown due to space limits. There
5000
are many more small clusters as those at the bottom of the
figure. The community has one large cluster, the largest con-
500
Num Packages
nected component (LCC) of the graph, containing 8985 nodes
50
which are either users or packages. There are many smaller
clusters with few users contributing to packages and also
5
many users that contribute only to one single package.
Figure 2 shows the number of clusters versus the number
1
1 2 5 10 20
of nodes in it (log-log scale). The LCC is depicted by the dot
Num Dependencies
at the very bottom right corner of the figure. The majority of
clusters has only few or just a single user package tuple. This Figure 6. Dependencies.
also means that only users within the single largest connected
component will be relevant for our analysis. Since we heavily
rely on user degree in the social graph GS , many users in the
small clusters will have very low importance.
Figure 3 shows the degree distribution of the user graph GS .
Low degree of many users is explained by the large number
of small clusters, which are mainly individual contributors
of single packages. The graph GS consists of 11189 users. A
fraction of 14% has a degree of 0 and 60% of users have a
degree smaller or equal 3. The median4 degree is 85. A fraction
of 0.7% of users (74 users) have a degree larger than 85. Such
degree distributions are typical in online communities.
The next Fig. 4 shows the relationship between CRAN task Figure 7. User dependencies.
views and users. These views are interpreted as skills and ex-
pertise ranking is performed within the context of individual
task views. In total, 33 views exist. 7316 users are not associ-
4.2 Experiments Setup
ated with any view (because their packages are not listed in
any view). Thus, those users will not be considered in the for- In our experiments, we sampled a random set of CRAN task
mation algorithm. 3873 users are associated with one or more views representing the demanded team skills. The crossover
views. The median value for the number of views within this probability is set to 0.7 and the mutation probability to
user segment is 11. This provides already a good diversity in 0.05. The metric weights for fitness calculation are set to
terms of users having different skills. wscore = wcost = wdistance = 31 . In each experiment run,
The next step is to analyse the relationship between users a population of 200 individuals plus 5 individuals for elitism
and software packages. Figure 5 shows the number of users has been created. We evaluate the quality of the expertise
over packages. The median value for the number of software mining approach by checking key metrics such as degree of a
packages is 20. The dependencies of packages are visualized user, number of packages in a given view (where the user is
by Fig. 6. 4550 packages have exactly one dependency (the R top-ranked), and number of all packages.
environment). The median value is 10.
Finally, Fig. 7 shows the average number of dependencies
by the number of users. The median value for the average 4.3 Qualitative Evidence
number of dependencies is clearly 2. A team with the highest fitness for 5 skills is depicted by Fig. 8
and detailed in Table 1. The team has an average expertise
score of 0.8, a cost of 0.6, and distance 1.0. These are excellent
values. A perfect fitness of 1.0 is not attainable because there
4 The median is used to separate the higher half of the user popu- is a tradeoff between expertise and cost. Indeed, the maximum
lation from the lower half. achievable fitness depends on the selected skills.
Table 1. Example of team recommendation for 5 skills.
Skill User Rank Score Score (GS ) Ch. Deg. (GS ) Pkg All Pkg Deg. (T)
Num. Math. A. Gebhardt 9 0.07681 0.00027 201 36 2 11 4
Bayesian M. Mächler 1 0.87039 0.00150 0 276 1 57 4
Meta Analysis T. Lumley 2 0.36787 0.00060 21 86 4 34 4
Social Sciences W. N. Venables 8 0.19568 0.00021 509 50 5 10 4
Psychometrics K. Hornik 1 0.85807 0.00113 4 318 3 71 4
100 110 120 130 140 150
0.85
0.80
Thomas Lumley (MetaAnalysis)
Best Individual Fitness
Overall Fitness
0.75
0.70
Brian D. Ripley () Martin Mächler (Bayesian)
0.65
90
0.60
80
2 4 6 8 10 2 4 6 8 10
Iteration Iteration
Figure 9. P -fitness. Figure 10. I-fitness.
Albrecht Gebhardt William N. Venables
(NumericalMathematics) (SocialSciences)
In the following we answer the question whether an in-
creasing number of skills results in more disconnected teams.
Kurt Hornik (Psychometrics) Fig. 11 compares different populations with an increasing
number of skills (from 5 to 9 skills depicted by S5 to S9).
Figure 8. Example team spanning 5 skills. S5
40 80
0
Table 1 shows further team member details. Rank is the 2 4 6 8 10
user rank within the specific task view (community rank for S6
Index
the given skill). Score is the numeric ranking score based on
30
our advanced expertise mining model and Score (GS ) is the
0
ranking score computed in GS using a standard PageRank. 2 4 6 8 10
This comparison essentially demonstrates the impact of our S7
Index
advanced context-based ranking model (see [27, 28]). Ch. is
30
the ranking change (context-based vs. standard PageRank).
0
One can see that context has a high impact in terms of ranking
2 4 6 8 10
position. We positively validated these results by checking the
S8
Index
online profiles of the top-ranked users because most users have
30
public Web sites. Deg. (GS ) depicts the degree (number of
co-authors) in GS . High degree typically means high commu-
0
2 4 6 8 10
nity standing. Pkg depicts the number of packages in a given
S9
Index
view and All Pkg all packages of the given user. Deg. (T)
is the team degree. Here we see a perfectly connected team
30
where each member is connected to all other members. The
0
user ‘Brian D. Ripley’ is selected as the coordinator. 2 4 6 8 10
Figure 11. Skills vs. number of disconnected team.
4.4 Performance Evaluation
We show the convergence of fitness values for both entire pop-
ulations (depicted as P -fitness in Fig. 9) and for the best in- The x-axis shows the number of disconnected team mem-
dividuals in each population (depicted as I-fitness in Fig. 10). bers and the y-axis the number of teams within the population
Convergence means that no major changes between one iter- (total number of populations is 205). The figures show that
ation to the next iteration are observed. 5 skills have been by increasing the number of skills the number of teams where
selected randomly. Different color codes depict 5 runs of GA only few members are disconnected decreases. More skills to
formation algorithm. The best teams were identified after 7 be satisfied by individual users increases the chance that team
iterations. Population fitness started to settle at 10. members are disconnected from the rest of the team.
5 Conclusions Digital Ecosystems Technologies (DEST), 2012 6th IEEE In-
ternational Conference on, pp. 1–6, (June 2012).
This work introduced team formation mechanisms for soft- [12] Jungpil Hahn, Jae Y. Moon, and Chen Zhang, ‘Emergence of
ware ecosystems. We apply a genetic algorithm including a new project teams from open source software developer net-
works: Impact of prior collaboration ties.’, Information Sys-
novel extension called relationship-driven mutation. In devel- tems Research, 19(3), 369–391, (2008).
opment teams, performance and quality are affected by the [13] Geir K. Hanssen, ‘A longitudinal case study of an emerging
programming skills and domain experiences of the project’s software ecosystem: Implications for practice and theory’, J.
team members. We apply an advanced expertise mining ap- Syst. Softw., 85(7), 1455–1466, (July 2012).
[14] Qiaona Hong, Sunghun Kim, S. C. Cheung, and Christian
proach to address this problem. Empirical results confirm the Bird, ‘Understanding a developer social network and its evo-
applicability of our presented methods. lution’, in Proceedings of the 2011 27th IEEE International
Future work includes the following aspects. Estimating col- Conference on Software Maintenance, ICSM ’11, pp. 323–
laboration cost is not trivial and may depend on many fac- 332, Washington, DC, USA, (2011). IEEE Computer Society.
tors, such as physical co-location, geographical distribution [15] S. Jansen, A. Finkelstein, and S. Brinkkemper, ‘A sense of
community: A research agenda for software ecosystems’, in
(including issues related to time difference), or cultural fac- Software Engineering - Companion Volume, 2009. ICSE-
tors. We will analyse and model cost of collaboration. We Companion 2009. 31st International Conference on, pp. 187–
will perform further evaluations of the approach in two di- 190, (May 2009).
rections. First, we want to validate results by engaging com- [16] Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Eliz-
abeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease,
munity members of the R ecosystem to get direct feedback and John Horton, ‘The future of crowd work’, in Proceed-
on formation and expertise ranking results. Second, we will ings of the 2013 Conference on Computer Supported Cooper-
broaden team formation experiments for other types of soft- ative Work, CSCW ’13, pp. 1301–1318, New York, NY, USA,
ware ecosystems including industrial and company internal (2013). ACM.
software ecosystems. An additional area of investigation is [17] Theodoros Lappas, Kun Liu, and Evimaria Terzi, ‘Finding
a team of experts in social networks’, in Proceedings of the
the consideration of formal skill frameworks such as SFIA5 . 15th ACM SIGKDD international conference on Knowledge
discovery and data mining, KDD ’09, pp. 467–476, New York,
NY, USA, (2009). ACM.
REFERENCES [18] Anirban Majumder, Samik Datta, and K.V.M. Naidu, ‘Ca-
pacitated team formation problem on social networks’, in Pro-
[1] Newcastle University, Roulette wheel selection. ceedings of the 18th ACM SIGKDD international conference
http://goo.gl/5CGi8t, Jan. 2007. on Knowledge discovery and data mining, KDD ’12, pp. 1005–
[2] Aris Anagnostopoulos, Luca Becchetti, Carlos Castillo, Aris- 1013, New York, NY, USA, (2012). ACM.
tides Gionis, and Stefano Leonardi, ‘Online team formation in [19] Konstantinos Manikas and Klaus Marius Hansen, ‘Software
social networks’, in Proceedings of the 21st international con- ecosystems - a systematic literature review’, J. Syst. Softw.,
ference on World Wide Web, WWW ’12, pp. 839–848, New 86(5), 1294–1306, (May 2013).
York, NY, USA, (2012). ACM. [20] Tom Mens, Malick Claes, Philippe Grosjean, and Alexander
[3] John Anvik and Gail C. Murphy, ‘Reducing the effort of bug Serebrenik, ‘Studying evolving software ecosystems based on
report triage: Recommenders for development-oriented deci- ecological models’, in Evolving Software Systems, eds., Tom
sions’, ACM Trans. Softw. Eng. Methodol., 20(3), 10:1–10:35, Mens, Alexander Serebrenik, and Anthony Cleve, 297–326,
(August 2011). Springer Berlin Heidelberg, (2014).
[4] Jakob Axelsson and Mats Skoglund, ‘Quality assurance in [21] Tom Mens and Philippe Grosjean, ‘The ecology of software
software ecosystems: A systematic literature mapping and re- ecosystems’, IEEE Computer, 48(10), 85–87, (2015).
search agenda’, Journal of Systems and Software, 114, 69 – [22] David G. Messerschmitt and Clemens Szyperski, Software
81, (2016). Ecosystem: Understanding an Indispensable Technology and
[5] Justin M. Beaver and Guy A. Schiavone, ‘The effects of de- Industry, MIT Press, Cambridge, MA, USA, 2003.
velopment team skill on software product quality’, SIGSOFT [23] Melanie Mitchell, An Introduction to Genetic Algorithms,
Softw. Eng. Notes, 31(3), 1–5, (May 2006). MIT Press, Cambridge, MA, USA, 1998.
[6] Christian Bird, David Pattison, Raissa D’Souza, Vladimir [24] Daniel Schall, ‘Expertise ranking using activity and contex-
Filkov, and Premkumar Devanbu, ‘Latent social structure in tual link measures’, Data Knowl. Eng., 71(1), 92–113, (2012).
open source projects’, in Proceedings of the 16th ACM SIG- [25] Daniel Schall, Service Oriented Crowdsourcing: Architecture,
SOFT International Symposium on Foundations of Software Protocols and Algorithms, Springer Briefs in Computer Sci-
Engineering, SIGSOFT ’08/FSE-16, pp. 24–35, New York, ence, Springer New York, New York, NY, USA, 2012.
NY, USA, (2008). ACM. [26] Daniel Schall, ‘Formation and interaction patterns in social
[7] Barry W. Boehm, ‘Improving software productivity’, Com- crowdsourcing environments’, Int. J. Commun. Netw. Dis-
puter, 43–47, (1987). trib. Syst., 11(1), 42–58, (June 2013).
[8] Jan Bosch, ‘From software product lines to software ecosys- [27] Daniel Schall, ‘Measuring contextual partner importance in
tems’, in Proceedings of the 13th International Software Prod- scientific collaboration networks’, Journal of Informetrics,
uct Line Conference, SPLC ’09, pp. 111–119, Pittsburgh, PA, 7(3), 730 – 736, (2013).
USA, (2009). Carnegie Mellon University. [28] Daniel Schall, Social Network-Based Recommender Systems,
[9] M. Claes, T. Mens, and P. Grosjean, ‘On the maintainability Springer International Publishing, 2015.
of cran packages’, in Software Maintenance, Reengineering [29] M. Srinivas and L.M. Patnaik, ‘Genetic algorithms: a survey’,
and Reverse Engineering, 2014 Software Evolution Week, pp. Computer, 27(6), 17–26, (June 1994).
308–312, (Feb 2014). [30] Cédric Teyton, Marc Palyart, Jean-Rémy Falleri, Floréal
[10] Massimiliano Di Penta, Mark Harman, and Giuliano An- Morandat, and Xavier Blanc, ‘Automatic extraction of de-
toniol, ‘The use of search-based optimization techniques to veloper expertise’, in 18th International Conference on Eval-
schedule and staff software projects: An approach and an em- uation and Assessment in Software Engineering, EASE ’14,
pirical study’, Softw. Pract. Exper., 41(5), 495–519, (April pp. 8:1–8:10, New York, NY, USA, (2014). ACM.
2011). [31] Hyeongon Wi, Seungjin Oh, Jungtae Mun, and Mooyoung
[11] R.P. dos Santos and C.M.L. Werner, ‘Treating social dimen- Jung, ‘A team formation model based on knowledge and col-
sion in software ecosystems through reuseecos approach’, in laboration’, Expert Systems with Applications, 36(5), 9121 –
5 https://www.sfia-online.org
9134, (2009).