Skill-based Team Formation in Software Ecosystems Daniel Schall 1 Abstract. This paper introduces novel techniques for the A key challenge in software ecosystems is to manage quality discovery and formation of teams in software ecosystems. For- of software [8, 9, 15] and addressing nonfunctional require- mation techniques have a wide range of applications including ments (NFRs) in general [4]. Software vendors may require the assembly of expert teams in open development ecosys- their plug-in developers to maintain certain quality levels to tems, finding optimal teams for ad-hoc tasks in large enter- deserve an approved status. As shown in earlier research, indi- prises, or working on complex tasks in crowdsourcing environ- vidual as well as development team skill has a significant effect ments. Software development performance and software qual- on the quality of a software product [5, 7, 12]. The approach in ity are affected by the skills and application domain experi- this work takes a socio-technical view on software ecosystems ences that the team members bring to the project. Team for- wherein ecosystems are understood as the interplay between mation in software ecosystems poses new challenges because the social system and the technical system [11, 13]. Software development activities are no longer coordinated by a sin- development teams composed of members with prior joint gle organization but rather evolve much more flexibly within project experience may be more effective in coordinating pro- communities. A suitable approach for finding optimal teams grammers’ distributed expertise because they have developed must consider expertise, user load, social distance and collab- knowledge of who knows what [12]. This paper addresses the oration cost of team members. We have designed this model problem of team formation in open, dynamic software ecosys- specifically for the analysis of large-scale software ecosystems tems. In open source development teams are more often than wherein users perform development activities. We have stud- not formed spontaneously (i.e., they ‘emerge’) based on peo- ied our approach by analysing the R ecosystem and find that ple’s availability, willingness to collaborate and to contribute our approach is well suited for the team discovery in software to a certain task. ecosystems. Potential tasks for expert teams in software ecosystems and open source development include: 1 Introduction • Come up with a software design and/or implementation of a complex component or subsystem. Establishing a software ecosystem becomes increasingly im- • Perform software architecture review of an existing imple- portant for a companies’ collaboration strategy with other mentation or provide expert opinion about an emerging companies, open source developers and end users. The idea technology. behind software ecosystems differs from traditional outsourc- To give a concrete example of a potential high-level task, a ing techniques [19, 22]. The initiating actor does not necessar- complex design or implementation may involve the analysis of ily own the software produced by the contributing actors nor timeseries data including data extraction from a source sys- are contributing actors hired by the initiator (e.g., a firm). tem, transformation of the data, storage, processing, and visu- All actors as well as software artefacts, however, coexist in an alization. Clearly, this task typically requires multiple people interdependent way. For example, actors jointly develop ap- with distinct skills such as data modelling, statistical knowl- plications and thus there is a relationship among the actors. edge (uni-/multivariate data analysis), data persistence man- Software components may depend on each and thus there is a agement, and data visualization using various technologies relationship among the components. This is a parallel to nat- and toolkits. Indeed, the high-level task needs to be further ural ecosystems where the different members of the ecosys- decomposed into smaller task. The goals of this work is to tems (e.g., the plants, animals, or insects) are part of a food find a team of experts given a set of high level skills. Once network where the existence of one species depends on the the team has been discovered, detailed task decomposition rest. In contrast to natural ecosystems, some software ecosys- and work planning can be performed, which is however not in tems may be mainly top-down controlled, with most changes the focus of this work. driven by change requests and bug reports coming from other We provide the following key contributions: actors [20]. Other software ecosystems may be controlled in a bottom-up manner, primarily driven by input from its core • A novel approach supporting team formation in software developers [21]. ecosystems based on user expertise, load, social distance among team members, and collaboration cost. 1 Siemens Corporate Technology, Siemensstrasse 90, 1211 Vienna, • Support the discovery of potential mediators if social con- Austria, email: daniel.schall@siemens.com nectedness in teams is low. • Analysis of user expertise to recommend the most suitable 3 Formation in Ecosystems team members. • Evaluation of the concepts using data collected from the Here we present the overall team formation algorithm. We Comprehensive R Archive Network (CRAN) - the largest employ a genetic algorithm that attempts to find the best public repository of R packages. team. Genetic algorithms (GAs) mimic Darwinian forces of natural selection to find optimal values of some function [23]. For each team a single number denoting the team’s fitness The remainder of this paper is organized as follows. In Sec- is calculated (where larger values are better). tion 2 we overview related work and concepts. In Section 3 we introduce our team formation approach. Experiments are detailed in Section 4. The paper is concluded in Section 5. 3.1 Genetic Algorithm Outline There are multiple objectives that need to be optimized. The objectives are to: 2 Related Work • maximize the average expertise score for given skills • minimize the average cost • minimize the average distance The success of a project depends not only on the expertise The team with the best trade-off among these objectives of the people who are involved, but also on how effectively shall obtain the highest fitness and will be recommended as they collaborate, communicate and work together as a team the best fitting team. The required team skills are stated by [17, 26]. On the one hand, a team must contain the right set customers who wish to assign a specific complex task to a of expertise, but on the other hand one should determine a team of experts — be it within a corporation or outsourcing staffing level that, while comprising all the needed expertise, a specific task to the crowd. Our assumption is that such com- minimizes the cost and contributes to meeting the project plex tasks demand for the expertise of multiple team mem- deadline [10]. The most critical resource for knowledge teams bers. Within our formation approach, additional constraints is expertise and specialized skills in using and handling tools. can be considered such as one person shall only cover one skill But the mere presence of expertise in a team is insufficient and not multiple ones. Indeed, in practice people are familiar to produce high-quality work. A team must collaborate in an with multiple topics thereby covering multiple skills. Factors effective manner. It has been found that prior collaborative such as matching user load with complexity, effort, and dead- ties have a profound effect on developers’ project joining de- line of a task are not in focus of this work. The reader may cisions [12]. refer to [25] for further information on these topics. In software engineering, team formation is often needed to The main computational steps of our formation approach perform a development or maintenance activity. A general are introduced and elaborated in detail in Algorithm 1. The trend is the growing number of large scale software projects, relevant lines in Algorithm 1 are specified in parenthesis. software development and maintenance activities demanding for the participation of larger groups [6, 14]. In [3], the authors 1. Based on the set S = {s1 , s2 , . . . , sn } of demanded skills, proposed assignment of experts to handle bug reports based prepare a mapping structure that holds skills, users U , and on previous activity of the expert. Social collaboration on expertise ranking scores (lines 6-9). In this step, current GitHub including team formation has been addressed in [18]. user load is evaluated (based on previously assigned tasks) The authors [2] study the problem of online team formation. and users with high load are filtered out. In their work, a setting in which people possess different skills 2. Initialize a population of individuals. An individual is a and compatibility among potential team members is modelled team consisting of n team members where n can be config- by a social network. It has to be noted that team formation in ured. The parameter n is given by the size of the skill set social networks is an NP -hard problem. Thus, optimization S if each team member has to provision exactly one skill algorithms such as genetic algorithms [29] should be consid- (line 12). ered in solving the team composition problem [31]. 3. Loop until max iterations have been reached and compute Expertise identification is one of the key challanges and suc- the main portion of the genetic algorithm based search cess factors for team work and collaboration [16]. The discov- heuristic. Finally, after this loop select the team with the ery of experts is becoming critical to ease the communication highest fitness (lines 14-48). between developers in case of global software development or 4. Depending on the ecosystems community structure and the to better know members of large software communities [30]. demanded set of skills, teams may have good or poor con- Network analysis techniques offer a rich set of theories and nectivity in terms of social links among team members. tools to analyse the topological features and human behaviour Therefore, construct a subgraph of the social collaboration in online communities. We have extensively studied the au- graph GS containing only the nodes from the best team tomatic extraction of expertise profiles in our prior research and their edges between each other. Analyse the connec- (see [24, 25, 27]) and build upon our social network based tivity of this subgraph by computing the average number expertise mining framework. of neighbours (lines 50-51). With regards to team formation in open source commu- 5. If connectivity is low, try to find a dedicated coordinator nities as well as software ecosystems, there is still a gap in who is ideally connected to all team members through so- related work and to our best knowledge there is no existing cial links. The role of the coordinator is to mediate com- approach that supports formation based on mined expertise munication among members and strengthen team cohesion profiles. (line 53-56). Algorithm 1 Formation algorithm. 3.2.1 Rank Expertise by Skills 1: input: skills S, population size p size, Expertise profiles are not created in a predefined, static man- 2: elitism elitism k, max iterations max iter ner. Expertise is calculated based on actual community contri- 3: output: best individual - team with the highest fitness butions. However, we do not attempt to analyse detailed user 4: # init mappings of skills, users, scores contributions in terms of software versioning control systems 5: M ← ∅ 6: for Skill s ∈ S do but rather focus on ‘high-level’ package metadata thereby fol- 7: m ← getRankingScoreM apping(s) lowing a less privacy intrusive approach. The method used for 8: M [s] ← m # add mapping determining the expertise scores is not within the focus of this 9: end for work due to space limits. The interested reader may refer to 10: # initialize population of [27, 28] for information on the basic approach. 11: # randomly composed individuals 12: P ← createP opulation(S, M, p size) 13: iter ← 0 # iteration counter 3.2.2 Basic Selection Strategies 14: while iter++ < max iter do An initial set of candidate solutions are created and their cor- 15: # new population responding fitness values are calculated. This set of solutions 16: P 0 ← createEmptyP opulation(p size) 17: # obtain a ranked list is referred to as a population and each solution as an indi- 18: R ← rankP opulationByF itness(P) vidual (i.e., the team composed of team members). The indi- 19: # elitism - copy small part of the fittest viduals with the best fitness values are selected and combined 20: for i = 0 . . . elitism k − 1 do randomly to produce offsprings, which make up the next pop- 21: # add to population ulation. The approach of selecting a subset of individuals with 22: P 0 [i] ← getIndividualByRankIndex(R, i) the best fitness values is called elitism. Elitism is realized by 23: end for copying the fittest individuals to the next population. 24: # build new population To maintain a demanded population of individuals, individ- 25: i ← elitism k uals are selected and undergo crossover (mimicking genetic 26: while i < p size do 27: # stochastically select from P reproduction). For the team formation approach this means 28: I[] ← rouletteW heelSelection(P) that team members are swapped between two teams. The ba- 29: # crossover sic part of the selection process is to stochastically select from 30: if randomN umber < crossover rate then one generation to create the basis of the next generation (see 31: # assign new offspring rouletteW heelSelection in line 28). The requirement is that 32: I[0] ← crossover(I[0], I[1]) the fittest individuals have a greater chance of survival than 33: end if weaker ones. This replicates nature in that fitter individuals 34: # mutation will tend to have a better probability of survival and will go 35: if randomN umber < mutation rate then forward. Weaker individuals are not without a chance. In na- 36: # assign mutated individual 37: I[0] ← mutate(I[0]) ture such individuals may have genetic coding that may prove 38: end if useful to future generations [1]. 39: # add new offspring Individuals are also subject to random mutations. However, 40: P 0 [i++] ← I[0] the probability of mutation is low because otherwise the ge- 41: end while netic algorithm would be just a random search procedure. In 42: # assign the new population our work we apply a ‘smart’ approach to mutation and do 43: P ← P0 not select a replacement team member at random. Rather, a 44: # evaluate population - total fitness new team member is predicted and voting is performed by 45: evaluate(P) the existing team members. 46: end while 47: # this is the best team 48: I ← f indBestIndividual(P) 3.2.3 Relationship-driven Mutation 49: # construct a graph GI based on I 50: GI ← extractSubGraph(GS , I) In traditional GAs, which are agnostic to the underlying na- 51: if checkConnectivity(GI ) < ζ then ture of the population, mutation takes place by exchanging a 52: # find node to increase connectivity gene by a random gene thereby maintaining genetic diversity. 53: u ← f indCoordinator(GI ) Here we perform prediction of edges between existing team 54: if u 6= null then members and newly randomly selected team members. If a 55: addN ode(GI , u) threshold is surpassed, the randomly suggested team mem- 56: end if bers is added to the team. This prediction approach ensures a 57: end if 58: # extract the node set NI ⊂ U from GI much higher likelihood that the new team member will work 59: NI ← getN odes(GI ) within the team more effectively. We propose random forests 60: return NI # node set of best team members for predicting edges between pairs of users. Random forests are a simple yet effective and robust method for classifica- 3.2 Detailed Computational Steps tion problems. Prediction of edges between pairs of users is performed by modelling features describing the relationship In the following we detail the steps in the algorithm and ex- between two nodes u and v. At a high level, these features in- plain additional functions that are invoked while executing clude common neighbours, the jaccard similarity index, joint the formation algorithm. community interest, and joint package dependencies. 3.2.4 Fitness Function Algorithm 1 line 45). Later on ΦI is used to rank the pop- ulation by fitness (see Algorithm 1 line 17) as well as when The last important ingredient of our GA based approach is calling f indBestIndividual (see Algorithm 1 line 48). the design of a workable fitness function. A fitness function Algorithm 2 details the computational steps of an individ- is a type of objective function that is used to provide a sin- ual I’s fitness as defined by Eq. 1. An approximation of a gle number. The idea is to discard ‘bad’ team configurations cost factor is provided by community popularity in terms of (i.e., individuals) and to breed new ones from the good config- number of neighbours in the social graph GS . Thus, according urations. The search heuristic is terminated when either the to this logic more popular users are also more expensive. A overall fitness converges or the maximum number of iterations perfect fitness score, given a set of skills S, would be 1. This have been reached. is however impossible to achieve because expertise is also in- The fitness function computes the value ΦI as fluenced by the user degree in GS . Thus, a suitable tradeoff ΦI = w1 ∗ expertise + w2 ∗ cost + w3 ∗ distance (1) among these factors has to be found. The individual with the highest fitness within the population is then selected and rec- where w1 + w2 + w3 = 1. Each of the input factors ommended as the best team. expertise, cost, distance needs to be scaled between 0 and 1. ΦI is computed when invoking the function evaluate (see 3.2.5 Coordinators Algorithm 2 Fitness function. Based on the set of demanded skills and community structure, it may not be possible to find teams with good connectiv- 1: input: skills S, individual I, ranking score mapping M 2: output: fitness value ΦI ∈ [0, 1] of individual I ity among the team members. The last steps in Algorithm 1 3: metrics ← ∅ # metrics as basis for fitness would be to check the connectivity of I and to find a dedicated 4: score ← 0 # team expertise score node who is ideally connected to all nodes in I to mediate 5: popularity ← 0 # popularity - to approximate cost communication. All nodes in U matching this constraint are 6: for Skill s ∈ S do then ranked based on their averaged expertise given the skill 7: u ← I[s] # get member by skill set S. The discovered node acting is potential team coordina- 8: rs ← M [s][u] # expertise score by skill tor is added to the final team. 9: # perform feature scaling max(M [s])−rs 10: rs0 ← 1 − max(M [s])−min(M [s]) 4 Experimental Evaluation 11: score ← score + rs0 12: # get user degree in GS 4.1 R Ecosystem 13: ku ← degree(GS , u) 14: # perform feature scaling In this section we present our experiments. We focus on one 15: ku0 ← kmax −ku part of the R ecosystem called the Comprehensive R Archive kmax 16: popularity ← popularity + ku0 Network (CRAN). Other R-based communities not considered 17: end for in this research are, for example, Bioconductor2 . We have im- 18: # add average team expertise score to metrics plemented a Web crawler to download3 and parse R software 19: addM etric(metrics, score/|S|) package meta information available as HTML pages. This in- 20: # add average popularity to metrics formation includes contributing authors and package depen- 21: # higher community popularity means higher cost dencies. In addition, CRAN provides so called CRAN task 22: addM etric(metrics, popularity/|S|) views, which are used in our analysis as skill or topic infor- 23: dist ← 0 # social distance mation. As an example, a given task view Bayesian Inference 24: Q ← queue(I) would be a single skill. 25: while Q 6= ∅ do 26: u ← poll(Q) Figure 1 shows the package authorship graph GA . 27: for v ∈ Q do 28: # unweighted shortest path distance Wanhua Su Alexandra Laflamme−Sanders Greg Young extRemes Rick Katz # d(u, v) computed in GS lago Huan Cheng 29: distillery SpatialVx smoothie BASIX in2extRemes GeneFeST Eric Gilleland Bastian Pfeifer Ulrich Wittelsbuerger PopGenome Bob Handsaker A. G. Taylor S. D. Cohen C. A.J. Appelo D. Gillespie R. Serban Liangying Zhang Holly Janes S. R. Charlton D. Shumaker D. L. Parkhurst Tim Hoar Tom Kraljevic phreeqc RadioSonde Spencer Aiello Amy Wang Petr Maj iid.test Ariel Rao esd4all h2o Yanming Li replicationDemos cyclones Alexandra Imbert duv = d(u, v) Bin Nan clim.pact MSGLasso anm Nello Blaser Rasmus E. Benestad 30: Luisa Salazar Vizcaya knnflex Yan Liu Dennis D. Boos Alejandro Cáceres Yubo Zou Atina Dunlap Brooks Yuzheng Zhang PrecipStat Dorit Hammerling Yingwei Peng Rlab Hexin Zhang qqplotter Tia Lerud MLPAstats CNVassoc Douglas Nychka Nathan Lenssen Edsel A. Peña LatticeKrig CNVassocData Doug Nychka gcmrec smcure gvlma Emmanuel Sharef Elizabeth H. Slate Juan R. González survrec Robert L. Strawderman Stephan Sain splinesurv Alan C. Hindmarsh Added Fortran David Ruppert Stephen Sain Chao Cai ordBTL Piotr Romanski David M. Spooner John Paige fields Jiajia Zhang William Roca Songfeng Wang FSelector International Potato Center Rene Gomez Giuseppe Casalicchio Marc Ghislain exsic mcpd Barry Hurley Merideth Bonierbale Jie Zhou Vilma Hualla Tobias Kuehn quipu Lars Kotthoff NPHMC Talal Rahwan Pablo Carhuapoma Erich Studerus Reinhard Simon divagis Leonard Judt Felipe de Mendiburu TransModel Michael Braun Original Fortran Jorge Nunez llama monographaR Stef de Haan agricolae Sangkyun Lee H. Welsh Fraser Ian Lewis KappaV Thomas F. Coleman Zachary Jones Lluís Armengol Fraser Lewis if duv 6= null then datacheck sparseHessianFD Lingbing Feng Xavier Solé Marcelo Reginato Marta Pittavino SNPassoc J. O’Neill Dan Dkin abn 31: Vincent Bonhomme Jorge J. More Elisabet Guinó Sarah Ivorra Cedric Gaucherel Evan Saitta Jun Peng Burton S. Garbow Sandrine Picq Asher Wishkerman Jose Francisco Loff Fernando Colchero Wenbin Lu Telmon Hinrich W. H. GohlmannChiara Forcheh Linda R. Petzold Maren Rebke Anyiawung Ricardo Kriebel Ulrike von Luxburg Willem Talloen Georgi NalbantovNorbert Kenneth Hillstrom truncdist Alain Berro loe Neus Martinez tgcd Momocs Hoksan Yip Alberto Murta Yoshikazu Terada BaSTA Lieven Clement SVMMaj Reinhard Furrer Anu Sironen Jason Collison Alexander Duerre Geert Verbeke smds beadarrayFilter hoardeR isopam GeneticTools ALKr simco REPPlab Pushpike Thalikarathne gaussquad Patrick J. F. Groenen Enyelbert Muñoz Sebastian Schmidtlein GENEAread sscor Daniel Fischer PamGeneMixed orthopolynom cluster.datasets Hannes Feilhauer aBioMarVsuit liso Frederick Novomestky LDRTools Eero Liski Owen Jones Vincent T. van Hees Zhou Fang matrixcalc goalprog Carsten Oldenburg Severine Sabia autopls Daniel Vogel Ziv Shkedy rportfolios gMWT GGIR Raquel Iniesta Marion Deville OjaNP Robert Maillardet Emmanuel Rousseaux BayHap Steven Carnie Bradford B. Worrall Richard Brent Stephen Ed Ionides John Burkardt Michele M. ABCoptim Sale R. Williams PST Martin Kroll credule Víctor Moreno Jung−Ying Tzeng Carles Breto Olga Borovkova OptGS Jyrki Möttönen Rafael S. de Souza Bertrand Le Nezet Matthew J. Ferrari spuRs RCEIM spnet Jonathan Elliott Hannu Oja Gilles Pujol John Wambaugh moments Steve Ellner Dao Nguyen LOGIT Sebastian Funk MuFiCokriging Edward L. Ionides Simulation du Comportement FastKM Michael Lavine Bruce E. Kendall RandomFieldsUtils Alexandre Janon Robert Pearce James Wason googlePublicData Sara Taskinen Bernardo Ramos Jimena Davis safi CosmoPhotoz RsimMosaic tsBSS MNM ICSNP Helen Wearing Sebastien Da Veiga Alberto Krone−Martins httk Loic Le Gratiet Nisha Sipes fICA Markus Matilainen Dave Tyler ICS David E. Tyler popEpi M. Rantanen Malte Jastrow Joseph M. Hilbe Klaus NordhausenIngo Bulla Xavier Bay BSSasymp Jari Miettinen Alexis Jinaphanh Jospeh Hilbe JADE colcor COUNT pomp R. Woodrow Setzer SpatialNP Jaakko Nevalainen Omkar Muralidharan UPMASK J. Pitkaniemi Thomas Muehlenstaedt Jean−Francois Cardoso Jana Fruth Rachel Marceau Olivier Jacquet Renext Carles Martinez Breto Taieb Touati msme Fang−Chi Hsu Daniel C. Reuman Seija Sirkia sensitivity K. Seppa Esa Ollila fanovaGraph equivalence Andre Moitinho Stephen P. Ellner codadiags Nicolas Durrande RenextGUI mixfdr Joseph Guillaume financial Brett T. McClintock Lukasz Komsta 32: # perform feature scaling Service d’Etudes multimark Yves Deville Bradley Efron GPlab Acho Arnold Laurent Gilquin Energie Atomique S. Marmin Andrew Robinson moonsun bit64 Werner Brannath DiceView kergp mblm AGSDest ref Paul Lemaitre Clement Chevalier DiceOptim quantchem DiceKriging T. Wagner Barry W. Brown rindex Olivier Roustant Jérôme Guélat KrigInv bit Vincent Moutoussamy Yann Richet David Ginsbourger dtt James Lovato regtest Rick ReevesMarguerite A. Butler Nicolas Bousquet Jens Oehlschlägel AHR GPareto Jens Oehlschl Matthias Brueckner Victor Picheny Cleve Moler outliers Clement Walter Martin Schlather subplex MarkedPointProcess ouch Gilles Defaux Edward Sudicky Aaron A. King Rene Therrien mistral SoPhy Jessica Franco Mickael Binois Delphine Dupuy DiceEval Guanhua Chen Heike Schuhmacher simex Rien van Genuchten C. Helbert Wolfgang Lederer Bernd Huwe fastM DiceDesign Heike Schumacher kin.cohort Andrew Sila Carl A. Mendoza vectoptim Guillaume Damblin Manuel Wiesenfarth Nilanjan Chatterjee Thomas Terhoeven−Uselmans soil.spec Thomas Terhoeven−Urselmans SwissAir Yusuke Sugomori Pius Korner−Nievergelt Carl Morris Peter M. J. Herman Ryan Curtin blmeco FAwR RcppMLPACKRcppDL Joseph Kelly AdaptFitOS ecolMod Bertrand Iooss Stefanie von Felten Frederik De Laender Rgbp Niki Zumbrunnen ToxLimKarel Van den Meersche rootSolve BCE Bettina Almasi Henrik Andersson Theresa Albrecht ReacTran Filip Meysman Rene Locher pvclass OceanView limSolve femmeR diagram Franscesca Mazzia deTestSet Jeff Cash ByeongChris−Carolin Yeob Choi Schoen StackOverflow Karline Soetaert Tobias Roth diffEq Thomas Unternährer LIM bvpSolve Dick van Oevelen James Orr Yvonne Badke Christian Reimer plot3Drgl Fraenzi Korner−Nievergelt Malena Erbe Tatyana Krivobokova Andreas F. Hofmann Bernard Gentili synbreed Francesca Mazzia AquaEnv Lutz Duembgen plot3D Heloise Lavigne Peter VandeHaar seacarb shape NetIndices Jean−Marie Epitalon Pius Korner Hans−Juergen Auinger Julius Kipyegon Kones birdring Larry Schaeffer evd Filip J. R. Meysman Aurélien Proye GSIF Barbara Hellriegel Hannes Reuter AdaptFit Jim Orr Alec G. Stephenson Chris Ferro carcass Yunro Chung Anastasia Ivanova Julius Kipkeygon Kones Michael G. Hudgens PlayerRatings Manuela Huso Matthew A. Etterson Brendan Malone James Rae Jean−Pierre Gattuso Olivier Lepais Bas Kempen Jeff Sonas FedData Johanna G. Neslehova Rob Robinson Murray Lark MultiCNVDetect mev Leo Belzile Oliver Behr 33: # lower values are better Manuela M. P. Huso Markus Baaske R. Kyle Bocinsky Dylan lcopula Christian Genest Jeff D. Hamann Robert Brinkmann Dan Dalthorp pRSR Gerard Heuvelink Ivo Niermann Alexander J. McNeil logcondens QRMlib Bhramar Mukherjee Scott Ulman mFilter SimRAD rconifers R. Kyle Bocinsky Doug Maguire M. S. Islam evir JointModeling Mehmet Balcilar HDPenReg JointGLM WaveCGH Martin W. Ritchie Joshua French nnc Guenther Walther nncRda pear Hyukjun Gweon PhySim Changjiang Xu Ruo Xu Andrea Rau Dolph Schluter modehunt sltl Lucia H. C. Anjos Luciano Seta bestglm EvoRAG ascrda Mark M. Meerschaert Dick Brus Stanislas Hubeaux Manuel Gentile evdbayes reporttools smerc ExceedanceTools pequod deseasonalize SurvRegCensCov Filippo Santambrogio artfima Quinn Payton Ken Kleinman selectMeta FitAR Kaspar Rufibach OrdFacReg Farzad Sabzikar Hao Lin Kendall Alberto Mirisola ebdbNet DeducerMMR A. I. McLeod rtkpp smacpod Laura Hauser Ying Zhang FGN Marc Weber rneos SpatialTools Dario La Guardia DTK OrdMonReg koRpus arfima Justin Q. Veenstra systemfit mleur ltsa Zinovi Krougly gogarch cents Alexandre Brulet rsatscan Hao Yu FitARMA censNID Abdel El−Shaarawi Pablo Garcia Rodriguez Fadoua Balabdaoui DiversitySampler rwm Matthew K. Lau Earl Brown Nagham M. Mohammad JohnsonDistribution hcc Juan Carlos Ruiz Cuetos Serge Iovleff Maria Eugenia Polo Garcia Rmpi D. E. Hines dynia cccp enaR P. Sing Lin Wang Joachim Dahl Vincent Kubicki Yun Shi MixAll Leanna King Qizhai Li Kathrin Weyermann pcenum logcondiscr VecStatGraphs3D Parmeet Bhatia William Wheeler Aude Longeville Martin Andersen rtkore S. R. Borrett spcosa FRAPO Jinkun Xiao Lieven Vandenberghe Benjamin Auder Emmanuel Blondel D. J.J. Walvoort modelcf Yixin Chen HTSDiff cleangeo ARTP Rmixmod glmlep Florent Langrognet Mathieu Ribatet VecStatGraphs2D Miguel Sousa Lobo HTSCluster Kai Yu D. J. Brus Henry Bart Jr POT Stephen Boyd Jaap de Gruijter RobustEM Angel Felicisimo G. Govaert Herve Lebret rddtools Dennis Walvoort Christophe Biernacki J. J. de Gruijter LGEWIS Xin Dang Lieven Vandenberge Remi Lebret Aurora Cuartero blockcluster Aishat Aloba RFA BEQI2 J. C. Cuetos versions P. G. Rodriguez GRaF Gilles Celeux Zihuai He Javier F. Tabima M. E. Polo Willem van Loon LGRF Marie−Laure Martin−Magniette Cathy Maugis−Rabusseau # max(GS ) is diameter of graph Pallavi Singh Hanna Jankowski Grard Goavert rnn Jonah C. Brooks Seunggeun Lee Parmeet Singh Bhatia Gerard Goavert 34: Geraldine Henningsen Nick Golding Bastiaan Quast Stacy A. Krueger−Hadfield CPHshape Canhong Wen Victor Kmmritz MetaSKAT David B. Dahl decompr rscala RmixmodCombi wiod gvc poisson.glm.mix learNN jvmr SelvarMix Marie Davidian BayesComm Fei Wang Tim David Coelli J. Harris SKAT diagonals Rihong Hui Victor Kummritz Larisa Miropolsky Anastasios A. Tsiatis James E. Hines bamboo Vincent Brault convexHaz K. Ullas Karanth Tobias Liboschik J.−P. Baudry Min Zhang Wu blender SPACECAP Jim Hines Ivy Wang Michael WuMicheal Michael E. Meredith Panagiotis Papastamoulis Hugh McCague Sumanta Mukherjee Arjun M. Gopalaswamy Yanfei Kang Jon A. Wellner TED cdcsis robeth Devcharan Jathanna Kendon Bell James D. Nichols Andrew J. Royle Mohammed Sedki prism Danijel Belusic rWBclimate label.switching N. Samba Kumar Xueqin Wang Dipti Bharadwaj Edmund Hart RobustAFT Mian Huang EcoSimR Aaron Ellison biorxivr capushe VarSelLCM Wenliang Pan Nick Gotelli robustloggamma lga Jean−Luc Muralti MvBinary speff2trial Bertrand Michel V. J. Victor Cathy Maugis Matthieu Marbac robustvarComp Francisco Herrera Triguero RSNNS Jean−Patrick Baudry Jose Manuel Benitez Sylvain Arlot Stefan Van Aelst frbs Peter J. Green Alex Randriamiharisoa Francisco Herrera Rmalschains Lala Septem Riza José M. Benítez Javier Otegui RoughSets Xiaomin Lu Chris Cornelis Chaitanya Khadilkar Karen Schettlinger Fields Development Team Ella Roelant William H. Swanson Dominik Slezak Daniel Molina Paul H. Artes simr Claudio Agostinelli Sebastian Stawicki Mitchell W. Dul Andrzej Janusz Neil O’Leary d0uv ← max(G S )−duv LOGICOIL SCORER2 visualFields Catriona MacLeod wle Victor E. Malinovsky circular Derek N. Woolfson gimme Shiquan Wu Richard Russell ImageMetrics Hallie Pike CircStats localdepth Thomas L. Vincent Craig T. Armstrong T. E. C. Common Mathematical Library bipartite Marco Chiarandini Peter Molenaar Michal Juraska Ulric Lund David H. Foster modelfree Ivan Marin−Franch Rouven Strauss Mario Romanazzi Kathleen Gates Nimisha Chaturvedi igraphtosonia Aaron Clauset 35: Stephanie Lane Kamila Zychaluk Sean J. Westwood Jochen Fruend NetCluster Nils Bluethgen Peter B. Gilbert Diego Vazquez NetData triads Miguel Rodriguez−Girones Jelle Goeman Dan McFarland Luis Paquete penalized Mariano Devoto Jose Iriondo Aldo Solari snipEM Dario Basso MIIVsem flip Rosa Meijer cherry Zachary Fisher CALF Alessio Farcomeni Ken Bollen robustHD Livio Finos Fredrik Nilsson Marco Rinaldo robumeta someKfwer Matteo Favero Elizabeth Tipton perry Diana Perkins seqDesign sparseLTSEigen cmprskContin Mike Nowak egonet Clark Jeffries Bernd Gruber cvTools ccaPP max(GS )−1 F. Gioachin Doug Grove x12 A. Sciandra David Simcha Yanqing Sun x12GUI PopGenReport Angelika Meraner Ian W. McKeague NEff Aaron Adamack growthrate Klaus Henle msgl Xuesong Yu Annegret Grimm Russell Horton Andrew Gorman FREQ dist ← dist + d0uv Sara Lopez−Pintado MarkKay Belardinelli Wunsch Jordan Killpack Jaakko Jarvi Thomas Schreiber Garrett Miller Matthew Hokanson Conrad Sanderson RcmdrPlugin.depthTools Casey Kolderup Holger Kantz Yoondong Lee Kristof Coussement Yuchao Jiang Simone Ecker Gianluca Gazzola Armin Graber depthTools Matthias Bogaert rrdflibs Ryan Freebern Giulio Barcaroli Jeremy J. Shen Koen W. De Bock Dankyu Yoon K. Adam White Aurora Torrente arrayImpute Canopy SamplingStrata MADAM PPtree gCat Mark Sample Martin Vincent GAMens Egon Willighagen seqCBS Sung Gon Yi AggregateR 36: Patrick Rodriguez snpXpert PPtreeViz Hyeong−Seok Lim Daniela Pagliuca Michael Dewberry Yoojin Jung gSeg dummy AUC genalg asVPC Jie Ding Nathaniel Mitchell Andrey Volkov Eva Budinska Matthias Wieser Taesung Park fit4NM rrdf Hao Chen Mueller Nancy R. Zhang Czech Republic Ralf Bundschuh David E. Frankhouser kernelFactory Karl G. KuglerLaurin Gyujeong Noh nCal ORMDR Michel Ballings lift arrayMissPattern Eun−Kyung Lee PKmodelFinder Jeroen D’haen PSCN Guido Marcucci Dirk Van den Poel Michael G. Schimek lsgl aCRM Steven Hoornaert Haipeng Xing Yunting Sun sglOptim falcon TopKLists QuACN hybridEnsemble MethylCapSig ProbeR Michiel Van Herwegen Helmholtz Centre Munich Pearlly Yan TERAplusB rotationForest leapp Michael Schutte interpretR imputeMissings Junghyun Namkung GrammR Michiel Vanherwegen Vendula Svendova Dauwe Vercamer Shili Lin Javkhlan−Ochir Ganbat BootCL Michael G. SchimekEva Budinska Matthias Dehmer Deepak N. Ayyala Laurin A. J. Mueller MinSeok Kwon bayesQR Matthijs Meire Krisztian Sebestyen imputeMDR Li Lee esaBcv Art B. Owen Samsun Sung Kun Chen Jingshu Wang Youyi Fong Keming Yu DIME Heebal Kim BOG Taeyoung Hwang Sunggon Yi Rahim Al−Hamzawi dblcens Dustin Potter Taeheon Lee cate kyotil Dries F. Benoit optismixture sptm Wonil Chung krm Cenny Taslim emplik Jincheol Park PermuteNGS ELYP Abbasali Khalili chngpt prc Hera Y. He Mai Zhou Qingyuan Zhao Bo−Young Lee Aelasticnet CVThresh Yifan Yang Saheli Datta seismic emplik2 Hee−Seok Oh kmc EMD Rahim AlHamzawi rankreg Donghoh Kim William H. Barton SpherWave Murat Erdogdu SynchWave Jin Zhezhen end if Dongik Jang Eugene Brevdo 37: 38: end for accrual Matthew S. Mayo Yu Jiang Cen Wu coreTDT rvTDT Andrew S. Allen JAGUAR Chaitanya R. Acharya audit Glen D. Meeden Radu Lazar polyapost rcdd Daniel J. Eck ump gdor aster2 Charles J. Geyer TSHRC trust pooh Peihua Qiu Martin Bezener mnspc Jun Sheng sROC NPsimex fANCOVA Xiao−Feng Wang ADPclust decon Yifan Ethan Xu gb Bin Wang bda miRada spt distfree.cr EBglmnet Zhiqiu Hu Anhui Huang Zhiquan Wang sparseSEM PAS Rongcai Yang Xiaodong Cai EBEN Shizhong Xu Qishan Wang NAM Tiago Pimenta Alencar Xavier Katy Rainey William Muir bWGR William Beavis SoyNAM Brian Diers James Specht Jacob Coleman acmeR Robert Wolpert Zhi Ouyang bark Merlise Clyde tglm BAS Alan F. Karr Lawrence H. Cox Quanli Wang EditImputeCont Hang J. Kim Jerome P. Reiter NPBayesImpute HyunJi Kim discretization Jingchen Hu Marta Sestelo Nora M. Villanueva FWDselect npregfast seq2R Javier Roca−Pardinas Artur Araujo genSurv TPmsm prop.comb.RR p3state.msm Maria Alvarez Artur Agostinho Araújo smoothHR Luís Meira−Machado survivalBIV Ana Moreira Olivier Broennimann Valeria Di Cola Frank Breiner Daniel Scherrer Christophe Randin Olivier Broenniman Ruben Garcia Mateo Antoine Guisan Anne Dubuis Dorothea Pio ecospat Blaise Petitpierre Wim Hordijk Julien Pottier MigClim Nicolas Salamin Loic Pellissier Robin Engler biomod2 Damien Georges Wilfried Thuiller Centre Tecnològic de Nutrició Albert Navarro Universitat Autònoma de Barcelona complex.surv.dat.sim survsim David Moriña radir Manuel Higueras Pedro Puig hermite María Oliveira NPCirc Alberto Rodríguez−Casal Rosa M. Crujeiras Jacobo de Uña− DTDA Carla Moreira Geneviève Lefebvre Juli Atherton BACprior BCEE Denis Talbot Jacques Brisson Nathalie Vandal InferenceSMR Thierry Duchesne gte Radu V. Craiu TwoStepCLogit Daniel Fortin Mohammad Hossein Dehghan Sophie Baillargeon 39: end while Reka Howard fuzzyRankTests Yingbo Li mcmc alphahull Michael Littman Susana Faria TP.idm aster TEEReg Daniel Manrique−Vallier potts Joyee Ghosh nice Rcapture stratification Sample.Size Komei Fukuda bcpmeta caribou rNMF Beatriz Pateiro−Lopez Leif T. Johnson Jacobo de Una−Alvarez Vanesa Balboa−Barreiro Wei Jiang alphashape3d Helene Crepeau Louis−Paul Rivest Serge Couturier grade ssd sgof Thomas Lafarge Yulei Wang Jiayang Sun hcp Junheng Ma nFCA Irene Castro Conde Stephen J. Ganocy curvetest Guo−Qiang Zhang Zhongfa Zhang Justin J. Wilkins Martin J. Lercher KuangNan Xiong granova MOCCA Joan Guàrdia saws Giambattista Toller empiricalBayes Kohta Ishikawa Gerald Quon Keurcien Luu Joseph Dunlop Ron Keizer Benjamin Braasch Armin Biere Maribel Peró rateratio.test Catalina V. Anghel Quaid D. Morris Rajen Piernikarczyk bpcp exact2x2 PSAgraphics Troy D. Hanson Brian A. Danielak Johann M. Kraus exactci Marc Andre Daxer RcmdrPlugin.steepness Zahra Montazeri Blum Michael James E. Helmreich Fabio Zottele Combine Katalin Csillery perm HighProbability Michael G. B. Blum xpose4 RcmdrPlugin.EACSPIR PairTrading pcadapt sybil ClimClass Francis Nguyen InPosition Abdelmoneim Desouki Robert M. Pruzek David Leiva ISOpureR William E. J. Doane BoolNet Lemaire Louisiane abc.data ssanv Shrinkage David R. Bickel GiANT SCVA Han de Vries Michael P. Fay boussinesq Daniele Andreis xpose4data MChtest LTR prettyGraphs DistatisR Jean−Michel Becu Dao Zhou soilwater Statomica 40: GI ← extractSubGraph(GS , I) xpose4classic Martin Hopfensitz RcmdrPlugin.SCDA Alaa Ali granovaGG Antonio Solanas Mike Fay clpAPI steepness Michael Blum abc Csillery Katalin binseqtest RGENERATEPREC Marta Padilla FRBData Amit G. Deshwar E. Niclas Jonsson Christoph Müssel geotopbricks Syed Haider Mats O. Karlsson Gabriel Gelius−Dietrich interval hbim Emanuele Eccel Shinichi Takayanagi Paul C. Boutros Derek Beaton cplexAPI Hans A. Kestler Florian Schmid SLC asht PsiHat Joakim Nyberg Corey M. Yanofsky Francois Olivier Herve Abdi Andrew C. Hooker PSAboot Isis Bulte Patrick Onghena Emanuele Cordano bios2mds KsPlot Cherise R. Chin Fatt Marie Chabbert Claus Jonathan Fritzemeier RMRAINGEN Kyle Leckett TInPosition BiTrinA choplump Olivier François ncappc nivm RcmdrPlugin.SLC Jenn Kirk LFDR.MLE Daryl M. Waggott sybilSBML Binarize Interpol.T Issei Kurahashi bedr multilevelPSA makeR TunePareto NanoStringNorm apTreeshape Louisiane Lemaire xpose4genericxpose4specific SIMMS TExPosition PopED glpkAPI Eric Stroemberg Jason Bryer Rumen Manolov RGENERATE RMAWGEN Deya Alzoubi Tamara J. Blätte SCMA RFinanceYJ ExPositionMExPosition SCRT Nicolas Bortolussi Julien Pele EasyUpliftTree Nobuaki Oshiro blockmatrix Zuojing Li Jenny Rieck Ludwig Lausser Louis Luangkesorn RSeed TriMatch Ye Yang Emilie Lalonde timeline Stefan Mundus RClimMAWGEN NADA Zhenyu Yang Clement Fung Michal Grzadkowski Sebastian Ueckert Eric Durand Lopaka Lee glpk Marco Foracchia rHadoopClient sqlutils RSearchYJ likert Yohei Sato Annalisa Di Piazza ChangeAnomalyDetection RWebMA Kimberly Speerschneider EasyHTMLReport YjdnJlp Terrence Brooks skewtools Marco Tamburini ic.infer Lehrkamp Matthias Margret C. FuchsManfred Fischer Peter J. Galante Mario Samigli Fentaw Abegaz James J. Yang Guido Raos Minjeong Jeon Roger Guimera Rlinkedin Abdolreza Mohammadi Maria Uriarte FrF2.catlg128 Sophia Rabe−Hesketh FrF2 Riccardo Inchingolo Patricio S. La Rosa F gure 1 Authorsh p graph (subset) Christoph Schmidt Rachel K. Smedley Mariano Soley−Guardia Michael Piccirilli Berkley Shands Anne Buu relaimpo BDgraph SparseTSCGM L. Keoki Williams ChargeTransport Skye Buckner−Petty DiscreteInverseWeibull mcll Guilhem Doulcier Javier E. Contreras−Reyes orcutt EILA rnetcarto Matthias Lehrkamp Luminescence ENMeval Rpdb bingat Ulrike Groemping Benjamin Lauderdale StressStrength Rfacebook Stefano Spada DoE.wrapper Robert P. Anderson Javier Contreras−Reyes Lenth Russ Markus Fuchs Sebastian Kreutzer HMPTrees Matteo Quartagno Jamie M. Kass DiscreteLaplace HMP Aleksandar Radosavljevic Ernst C. Wit Julien Idé DiscreteWeibull Daniel B. Stouffer Kenneth Benoit jomo RcmdrPlugin.DoE Robert A. Boria Cari Kaufman Robert Muscarella Jia Li Tina Zhuo log10 afmtools LICORS Fox John Michael Dietze Alessandro Barbiero glassomix quanteda William D. Shannon George M. Goerg Christoph Burow spThin Matthew Hutchinson Pablo Barberá streamR Elena Deych Mirko Signorelli DoE.base Anani Lotsi lpbrim Matthew E. Aiello−Lammens ForImp James Carpenter pnea GenOrd ForeCA Paul Nulty Lars Skjaerven 41: # add average distance to metrics Pier Alda Ferrari SunterSampling bio3d bigGP copas Rollin Thomas Georg M. Goerg EMMAgeo Timothee Poisot RLumShiny MetaDE paco Fernando Cagua Wilfredo Palma Jan Odvarko Kohei Watanabe Benjamin Lipshitz instaR Actigraphy Hongquan Xu Bruno Vilela Xingbin Wang Christina Ludwig Boyko Amarov Giancarlo Manzi LambertW GenForImp Barry Grant Jia Wang metasens Veronica Vinciotti betalink Tiago Dantas Xin−Qiu Yao Elisabeth Dietze LSTS Christopher Paciorek LSC Guido Schwarzer meta George C. Tseng tightClust Juan Antonio Balbuena Ricardo Olea AnalytixWare digitize Wing H. Wong aLFQ Jonne Guyt Carlos Gonzalez enRich AAindex letsR Gerta Rücker Nadia Solaro DWreg MetaQC Subharup Guha Yanchun Bao glmmGS Pilar Rubio MetaPCA George Rosenberger netmeta Hannes Roest Fabricio Villalobos Don Kang Louise Ryan Michele Morara Ulrike Krahn Jochem König Douglas Galagate mxkssd Jose Manuel Soria Irina Mahlstein Luitgard A. M. Veraart chillR fpc tawny Qiong Ding Zhouwen Liu C. Alex Buerkle lshorth Asun Lubiano Mateusz Bruno−Kaminski pan FactMixtAnalysis pcg Jacopo Ripoldi F. Din−Houn Lau futile.paradigm Konrad Talik Helena Brunel fractalrock Mark Liniger Piotr Kowenzowski mix Jihong Huang Angel Martinez−Perez Gunther Sawitzki causaldrf futile.matrix Bernhard Hausdorf Stanislaw Jastrzebski Rebecca Hiller prabclus lambda.tools SAFD nnlasso Bedia Jimenez systemicrisk chopthin Jun Ma seqRFLP spaa Igor Sieradzki B. N. Mandal ifa introgress mkssd Eike Luedeling Sari Acra PhysicalActivity quantileDA Christian Hennig Charles E. Matthews solarius Cinzia Viroli nonlinearTseries Joseph L. Schafer Alvaro A. Novo Stephane Heritier Alfonso Buil easyVerification Brian Lee Yung Rowe Maciej S. Buchowski gmum.r norm lambda.r survivalMPL Maciej Zgliczynski Alexandre Perera Matteo De Felice futile.options D. Olivieri lmm extlasso Christoph Spirig Axel Gandy Wolfgang Trutschnig Marcin Data Constantino A. Garcia smoothmest tawny.types Jinlong Zhang J. Dustin Tracy L. Rodriguez−Linares decisionSupport futile Zongshan Li dcv PhyActBedRest Leena Choi Michal Pletty ibd Jonas Bhend trimcluster HK80 Zachariah Gompert mixtNB Jonas Kuppler Andreas Weigel Karol Jurek Claudia Mignani futile.logger Wojciech Czarnecki Dominique−Laurent Couturier Andrey Ziyatdinov spcadjust futile.any Jan Terje Kvaloy cat RHRV dynRB Robert R. Junker Zhiyi Xu ProfileLikelihood A. Mendez M. Lado Fernando Tusell simctest Lutz Göhring Elisabetta Bonafede phylotools Kong Y. Chen bayespref Fernando Tusell Original Baidya Nath Mandal afc chemosensors Kristin Tolksdorf Samir Kanaan−Izquierdo Manuela Schreyer cpca Ted Harding A. Otero X. Vila Georg Hahn Xiangcheng Mi 42: addM etric(metrics, dist/|edges(GI )|) Zachary Marion Arne C. Bathke Gaj Vidmar Patrick Rubin−Delanchy Nancai Pei Alexandre Perera−Lluna Katharina Schueller rriskDistributions James A. Fordyce Nickolay T. Trendafilov Natalia Belgorodski hierDiversity ordiBreadth HRM mppa Matthias Flor iteRates Matthias Greiner Harrar W. Solomon rriskBayes Premal Shah Benjamin Fitzpatrick Martin Happ Nicholas A. Heard Alexander Engelhardt Yoshimasa Tsuruoka Peter Fader David Zahrieh Wei Shangguan Wenchao Yang Justine Shults mcGlobaloptim Ulrich Bauer Gabriel Bucur ror Marie−Eve Beauchamp Andrew Jackson Francesco Favero Seung−Mo Hong Simon L. Rinderknecht Yao Zhang Wiktor Zelazny Vincent Rouvreau Andrea M. Marquard Ann Lazar Roberto Molinari Elea McDonnell Feit Richard Gelber Nic Jelinski Jan Reininghaus SungHwan Kim ycinterextra smaa Ryan Patrick Kyle Tommi Tervonen qlspack Arun Gopalakrishnan Bogdan Rosca Hongzhe Li Brittany T. Fasy siar maxent Stephane Guerrier SIBER sequenza gmwm panelaggregation kaps wSVM stepp Thierry Moudiki WCE fitDRC BTYD Daniel McCarthy William Barcella soiltexture Jose Lucas Safanelli TDA Dmitriy Morozov hitandrun Timothy P. Jurka Tim Jurka Aron Charles Eklund Tejal Joshi Marco Bonetti Jichun Xie utility Gert van Valkenhoef OutlierDM Nele Schuwirth Bruce Hardie Chiara Gigliarano Survgini Budiman Minasny Clement Maria beeswarm Soo−Heang Eo MCLIME ESGtoolkit James Sweeney Bchron gemtc Eric Schwartz Rainer Petzold Jisu Kim Andrew Parnell Aron C. Eklund Victoria Wang capme scio Marie−Pierre Sylvestre Michal Abrahamowicz Edward Wadsworth Julien Moeys Wassim Youssef ESG Fabrizio Lecci Matthias Bannert timeseriesdb HyungJun Cho Chip Cole James Balamuta Bclim stoichcalc sentiment Lukasz Dziurzynski Rodolfo Marcondes Silva Souza Weidong Liu Michael Kerber traj Matthias Bannert’ Peter Reichert Jean−Charles Croix Ulf−Dietrich Reips squash ecoval msos gemtc.jar cIRT simmr Frédéric Planchet Joel Kuiper Dan Vatnik OutlierDC Simone Langhans T. Tony Cai PermAlgo jetset ELT dropR Thinh Doan CPGchron John Marden RAdwords rivernet visualize Steve Culpepper personograph Lan Wang clime Thad Edens ecosim Julien Tomas Iain Marshall Huixia Judy Wang Todd MacKenzie Qiyuan Li Johannes Burkhardt 43: ΦI ← 0 RCriteo Arnaud Estoup Xiangrui Meng Zhidong Tu Developer Leo Breiman Developer wahc Yuriy Tyshetskiy Stephen Hogg edtdbg Yimin Wu Marcelo Araya−Salas evora isva Bin Yao Nicole Heussen IRTShiny Anja von Heydebreck bisoreg Virginie Guemas Javier Garcia−Serrano Rdsm snort Atsushi Mizumoto autoencoder RCryptsy S. Nonyane Fabian Lienert CTTShiny Isabel Andreu−Burillo Mark Norrie vabayelMix S. McKay Curtis Eugene Dubossarsky mlica2 PCDSpline GitHub SAENET Norm Matloff warbleR Andrew E. Teschendorff CorrMixed William Kyle Hamilton oncomodel mirf s2dverification Paneldata freqparcoord Chun Pan partools Yang Zhang Dieter Hilgers Wim Van der Elst Yingkang Xie Grace Smith Vidaurre MAVIS Jing Qian Ludovic Auger Alex Rumbaugh Hua Zhong Nicolau Manubens Gil Andrea S. Foulkes Kathleen Coburn mcmcplots mlica Surrogate B. Aletta Lianming Wang Geert Molenberghs RStars gensemble FunChisq matpow GenCAT DART ICBayes Christiane Heiss Martin Ménégoz Ariel Alonso Chloé Prodhomme EffectTreat Ilya Goldin Bo Cai Burak Aydin Evangelos Evangelou ICsurv Eric Reed Peter Werner Joe Song Geert Molenbergs Yan Jiao Jack Norman Sara Nuez Xiaoyan Lin intcox survBayes Christopher S. McMahan Vivekanda Roy geoBayes Ckmeans.1d.dp DARTData Volkmar Henschel Ulrich Mansmann Vivekananda Roy Haizhou Wang graphComp Khadija El Amrani FKF Miron Bartosz Kursa RcmdrPlugin.MA Dariya Malyarenko Vroni Retzer fgac J. Cortés ncdf4.helpers U. S. Geological Survey Northern mded Patrick Wheatley Emmanuel Müller Stephan Günnemann support.BWS 44: for m ∈ metrics do RcmdrPlugin.MAd G. Gómez Kenneth B. Hoehn Thomas Scherer Maureen Tracy Ulrich Drepper Witold Remigiusz Rudnicki PCICt Prairie Wildlife Research Center perspectev Philipp Erb WMBrukerParser David Bronaugh Simon Otziger Witold R. Rudnicki dcens Pacific Climate Impacts Glen A. Sargeant Timo Thoms Timm Jansen Thomas Seidl Xinyu Tang David Lüthi ghyp Wolfgang Breymann simba Consortium Hideo Aizaki RcmdrPlugin.MAc Veronica Andrea Gonzalez−Lopez M. V. Moneta wild1 subspace Boruta A. C. Del Re GDELTtools rTOFsPRO bwsurvival support.CEs MAc O. Julià DTR compute.es V. Moneta John Beieler schwartz97 William Cooke climdex.pcic zyp C. Serrat Research Center Ira Assent DCchoice Marwan Hassani Gerald Jurasinski V. A. Gonzalez−Lopez Maria Melguizo LIStest S. G. S. Northern Prairie Wildlife ccgarch Tomoaki Nakatani MAd William T. Hoyt Karl Kuschner J. E. Garcia Stephen R. Haptonstahl Matthias Hansen Miron B. Kursa rWMBAT rtape Juri Hinz rFerns Arelia Werner Anke Guenther dils Kazuo Sato flux Franziska Koebsch Jesus Garcia FastImputation triggr Qian Si Ulrike Hagemann Sascha Beetz mlCopulaSelection Veronica Gonzalez−Lopez Angel M. Garcia Fernando Silveira Marques plsRglm Andrew Azman AllPossibleSpellings Jialiang Li Max D. Price Christian Gerber Amelia Simo Jane Lawrence Sumner Cory Barr sigma2tools distributions Caleb King Fernando Ferreira Johannes Ransijn Juan Domingo I. W. K. Health Center Frank Klawonn BioStatR Biocomb IniStatR Myriam Maumy−Bertrand LMERConvenienceFunctions poibin Oswaldo Santos EpiDynamics disp2D Cristobal J. Carmona Frederic Bertrand Antoine Tremblay ADDT Frank Pessler HUM Natalia Novoselova Guillermo Ayala GRANBase Fabio Frascati SDR coarseDataTools zooaRch enviPat Anthropometry interactionTest capm Nicolas Meyer Jesse Wolfhagen M. Victoria Ibanez ReorderCluster switchrGist Yili Hong icaOcularCorrection ISA agreement Nicholas G. Reich Marcos Amaku Justin Lessler plsRcox Junxi Wang Federico M. Stefanini vacem Erik Otarola−Castillo Dean Adams Bruno Mario Cesana clusterPower isopat cstar plsRbeta Jessica Metcalf LCFdata Yimeng Xie Irene Epifanio Gabriel Becker staRt Pedro Gonzalez Oswaldo Santos Baquero NeuroCognitive Imaging Lab Guillermo Vinue clusterSEs Justin Esarey SearchTrees Martin Loos nontargetData SPREDA Daniel Obeng Francesco Corona geomorph fastdigest enviPick Bob Jenkins switchr ggsn Michael Collyer nontarget heatmapFit Emma Sherratt Zhibing Xu ΦI ← ΦI + wm ∗ metric Andrew Pierce Jericho Du 45: Doug Bronson Hugh J. Devlin Ryota Suzuki fbati Noah Silbert Celia Touraine Hiroto Udagawa Alexey G. Bukhovets Ron N. Buliung Anna Heath ChungHa Sung T. Lloyd Adam Vizina Philip Kokic SangGi Hong Stanislav Horacek geoscale Huidong Jin Mark A. Bell Randy Bui spTDyn 2 https://www.bioconductor.org aspace Tao Ding BCEA Junghoon Lee RIGHT Jae W. Lee strap Petr Maca bilan Ladislav Kasparek RIFS Voss Graeme T. Lloyd PatternClass Andrea Berardi Claddis bmeta Gianluca Baio Pavel V. Moskalev Tarmo K. Remmel JongHyun Bae Martin Hanel K. Shuvo Bakar TaeJoon Song T. G. Masaryk Water Research Mark A. Bell Graeme SPSL SECP BCEs0 hdeco spTimer K. S. Bakar Sandor Kabos Sujit K. Sahu Oana Almasan Emilio Letón rsggm MEWMA Alfonso Iodice D’ Enza Cathryn M. Lewis msgps Michele Scagliarini Carmen Cadarso−Suárez Davide Buttarazzi idm Graham H. M. Goddard MPCI Mihaela Hedesiu MCUSUM Kei Hirose RcmdrPlugin.EBM GsymPoint RcmdrPlugin.ROC Edgar Santos−Fernandez REGENT Graham Goddard clustrd Daniel−Corneliu Leucuta Angelos Markos Mónica López−Ratón Elisa M. Molanes−López caGUI fanc Johnson MSQC Andrei Achimas Michel Van de Velden Daniel J. M. Crouch OptimalCutpoints RcmdrPlugin.coin Haruhisa Nagata Michio Yamamoto 46: end for Maria Xose Rodriguez−Alvarez LEAPFrOG obliclus Michael E. Weale 3 https://cran.r-project.org accessed on June 2016 Alfred Galichon Rearrangement Ivan Fernandez−Val Victor Chernozhukov P. H. D. Wurzweiler School Wendy Zeitlin Charles Auerbach SSDforR P. H. D. Wendy Zeitlin Schudrich Social Work Justin J. Hendrick Wei Lu Anna L. Tyler cape Greg W. Carter Vivek M. Philip Juan Zalapa Walter Salazar Brandon Schlautman FragmanGiovanny Luis Diaz−Garcia Covarrubias−Pazaran Meredith McDonald Vladimir Jojic Derek S. Lundberg MInt Jeffery L. Dangl Surojit Biswas Hsin−Ta Wu Vivian Hsiao Benjamin Raphael cometExactTest Max Leiserson Fabio Vandin Nick Mihailowski Vignesh Prajapati Kushan Shah RGoogleAnalytics Michael Pearmain Nicolas Remy Kerrie Mengersen Udayanga Attanayake sesem Eric Lamb Katherine Stewart Steven Siciliano Francial G. Libengu C. Kokonendji TRIANGG Silvio S. Zocchi TRIANG Tristan Senga Kiess Gang Wu Bernard Fichet Jean−Michel Poggi Johnny Villalobos kitagawa David Lorenz Ricardo de Matos Simoes Karl Kornacker Jean Gaudart Oldemar Rodriguez kelvin bc3net John Hogenesch Guillaume Barbet Jorge Arce Andrew J. Barbour MetaCycle SPODT RSDA dataRetrieval VSURF Christine Tuleau−Malot Frank Emmert−Streib Robin Genuer Ron Anafi Roch Giorgi Roberto Zuniga Laura DeCicco c3net Michael Hughes Nathalie Graffeo Olger Calderon psd rchallenge Robert Hirsch David Myer Gokmen Altay Adrien Todeschini EGRET dc3net Robert L. Parker EGRETci usdm imputation Antanas Marcelionis tsbugs CORE ParDNAcopy Ana Paula Assis Compind nutshell fanplot Edgar Zanella Elisa Fusco fastVAR rts Joseph Adler Babak Naimi Guy J. Abel Alex KrasnitzGuoli Sun Diogo Melo Jeffrey Wong Francesco Vidoli LargeRegression rAmCharts evolqg Jeffery Petit Benoit Thieurmel ssfa visNetwork semsfa tsbridge nutshell.audioscrobbler migest Guilherme Garcia TimeProjection CNprep Giancarlo Ferrara TBEST Sebastien Vigroux seasonal Johannes Hoehne Peter Steiner Peter Nightingale Christoph Sax tempdisagg seroincidence CAMAN Daniel Lewandowski Maryna Verba Peter Teunis Peter Schlattmann D. Sculley Jeffrey Chrabaszcz bayesGDS Michael Bockmayr ionflows RSofia gemmR Michael Braun Ludwig Hothorn Fernando Cela Diaz Joe Tidwell Michael King Michael Dougherty sparseMVN EloChoice catmap M3 Kristen Foley Jenise Swall aqfig Kristin K. Nicodemus J. Kyle Roberts Augustin Luna Kim Nimon Rfun Ben Murrell NestedCohort gamboostMSM Herve Cardot onlinePCA Hormuzd A. Katki matie Dan Murrell simMSM CompareTests Hugh Murrell David W. Edelstein equate mvmeta cloudRmpi Delaporte SimCorMultRes Ben Armstrong Anthony Albano Antonio Gasparrini Rwinsteps dlnm Barnet Wagman Pade rreval Xianyan Chen sspline Xiang−Yun Wu Xiang Zhan NlcOptim ProNet KMDA Xianhong Xie Xiangrong Yin Xia−Yu Xia Debashis Ghosh rgcvpack Todd R. Jones Tobias Kley Tirthankar Dasgupta Tin−Yu Hui Vanderlei Julio Debastiani Uwe Menzel TszKin Julian Chan Tong Zhang Tomas William Fitzgerald Thorsten Pohlert Thomas Lin Pedersen Nathan Stephens Peter Nash Peter Hussey Sivasish Sindiri Pierre−Jerome Bergeron Mikhail Pogorelyy Nico Nagelkerke Hatto von Hatzfeld G. Brian Golding Perry D. Moerland ActiveQuant Gmb Joerg Helms M. Teresa Buglielli Elisabeth Waldmann Erik Andersen Erik R. Thomas Di Liu Lars Kai Hansen’s Paolo Tormene Janez Demsar Andrzej Bak John R. Stevens Stefan Birr Joseph J. Lee Imperial College London Robert Ackland Jacob Anhoej Jairo Cugliari J. Song Florian Meinfelder Pierre−Yves Boelle SYNCSA EMT ezsim muscor ADM3 trend densityClust Sven Ove Samuelsen Sven Knueppel Susie Jentoft Susan Holmes Sulc Zdenek Sudeep Srivastava Subhadeep Mukhopadhyay Stuart Baumann cthresh mlsjunkgen Stephen W. Erickson Tmisc Stephane Jankowski Stephan Gade Sten Ilmjarv gRcox sde Stefano Galelli Stefanie Kalus Stefan Zeugner bride Stefan Moeding Stack Overflow gcl Srivenkatesh Gandhi Sriharsha Veeramachaneni Soren Hojsgaard Sophie Arnaud−Haond Sonia Amodio Song Yang Sohrab Shah Snigdhansu Chatterjee Snaebjorn Palsson Simone Padoan Simon Mueller Simon Barthelme AICcmodavg TideTables vegetarian multipleNCC HapEstXXR RcmdrPlugin.sampling distory nomclust GPseq LPTime schumaker SNPMClust MultNonParam RPPanalyzer demi reservoir RfmriVC BMS usl install.load DataLoader LinearizedSVR mhsmm RClone ConsRank YPmodel xseq WiSEBoot shapeR CompRandFld nfda imager Stuart Barber Steve Myles Stephen Turner Steffen L. Lauritzen Stefano Maria Iacus Stefan Siegert Staal A. Vinterbo Marc J. Mazerolle Moritz Mueller−Navarra Noah Charney Nathalie C. Stoer Klaus Rohde Johan Heldal John Chakerian Rezankova Hana Liang Chen Shinjini Nandi Margaryta Klymak Joshua Callaway John E. Kolassa Heiko Mannsperger Hendrik Luuk Sean Turner Ludwig Bothmann Martin Feldkircher Neil J. Gunther Irucka Embry Kreshnaa Raam S. Bethusamy Ken Williams Jared O’Connell Diane Bailleul Antonio D’Ambrosio Junlong Sun Jiarui Ding Megan Heyman Lisa Anne Libungan Moreno Bevilacqua Simon Mller Antoine Cecchi waveband phonenumber qqman gRc opefimor SpecsVerification PrivateLR Roberta Andrade de Araujo Samuel G. Fadel Samuel Balmand Samantha Fernandez Salvatore Grimaldi Sairah Y. Malkin Sai Teja Ranuva zicounts S. Demeyer dendsort Ryan Sermas mchof Ryan C. Graebner mcparallelDo Rudradev Sengupta CatDyn Rong Pan Romain Frelat Roland Pfister Rohit Kumar Patra spanr Roeland Voorrips gstudio Lock5Data ConConPiWiFun Roberto Gatta Robert Stojnic Robert Jennrich Robert Ferstl Robert E. Fay Robert Castelo Robert Bruggner Robert A. Klopotek lhs QTLRel Risk Advisory Group Richard Zijdeman Richard Telford Richard J. Cook Fagundes pcaL1 GFMaps mp DESP TSTr LPM phytotools AnDE ATmet ChoiceModelR GeneticSubsetter IntegratedJM ALTopt ndvits schoRsch simest fitTetra spatialfil ISDA.R sequences GPArotation termstrc sae2 qp Rclusterpp regRSM its lar palaeoSig S. M. Mwalili Ryo Sakai Ryan Grannell Russell S. Pierce Ruben H. Roa−Ureta Roger Marshall Rodney J. Dyer Robin Lock Robin Girard Rob Carnell Riyan Cheng Ricardo Jorge de Almeida Paul Brooks Kanika Arora Francisco M. Fatore Arnak Dalalyan Ricardo Merino Corrado Tallerini Greg M. Silsbe Nayyar Zaidi A. Allard John V. Colias Alfonso Cuesta−Marcos Nolen Joy Perualila Kangwon Seo Bruno Gerard Markus Janczyk Arun Kumar Kuchibhotla Gerrit Gort Nicola Dinapoli Laurent Gatto Coen Bernaards Josef Hayden Mamadou Diallo Alberto Roverato Michael Linderman Pawel Teisseyre Commerzbank Securities Josemiguel Lana−Berasain Mathias Trachsel Queiroz Filho baymvb gapmap needy listWithDefaults LifeHist srd popgraph Stat2Data VHDClassification triangle qtlmt Rachael Maltiel R. Pelissier R. Doug Martin R. Barfield Przemyslaw Spurek Pingping Qu provenance Pieter Thijs Eendebak pnn Piergiorgio Palla Philip L. H. Yu devEMF Petri Toronen Peter X. K. Song normwhn.test PsumtSim Peter Macdonald bifactorial Peter Bogetoft Pengfei Li numOSL Pedro−Pablo Garrido Abenza Pavel Michna CompetingRiskFrailty WMTregions Paul W. Eslinger Paul Fischer Paul Evangelista Paul Cool Paul Burton Patrick Kraft JGL Pascal Kerschke pch Pan Tong Pablo Tamayo Pablo Montero Manso Pablo Emilio Verde peakPick P. Wijekoon Ozlem Alpu rafalib CpGassoc CEC greyzoneSurv oapackage RFmarkerDetector pmr mGSZ metafuse mixdist Benchmarking MixtureInf rplotengine RNetCDF SBRect pointdensityP DeducerSurvival ESPRESSO ArfimaMLM flacco PANDA RODM TSclust bamdit lrmest ltsbase Pieter Vermeesch Pierre−Olivier Chasset Philip Johnson Peter Wickham Peter N. Steinmetz Peter Frommolt Peng Jun Pavel Khomski Pavel Bazovkin Patrick Danaher Paolo Frumento Paal O. Westermark Michael I. Love K. Conneely Konrad Kamieniecki John Crowley P. T. Eendebak Giuliano Armano Paul H. Lee Pashupati Mishra Lu Tang Juan Du Lars Otto Jiahua Chen P. Pablo Garrido Milton Woods Heather Orr Astrid Hilbert Dave Beskow Matthew Ockendon Amadou Gaye Christopher Weber Jan Dagefoerde Hua Li Ari Mozes José Antonio Vilar Arnold Sykosch Ajith Dissanayake Betul Kan Kilinc ArArRedux grnn adaptivetau normwn.test EffectsRelBaseline ic50 RadialPlotter TwoWaySurvival StochaTR SETPath flexPM HarmonicRegression PurBayes Nial Friel svdvis Neil R. Clark wBoot Neha Rohatgi Ned Horning Nay John Nathan Medina−Rodriguez Natalie Koziol Natacha Nikolic Nadezda Frolova Murilo Coutinho Silva Mounir Aout Morten Moshagen Monique Graf Mingkun Chen fmsb LSMonteCarlo Mike Higgins Miguel Munoz Zuniga fsia Michel Wedel Michaela Dvorzak bcrm Michael Stewart Michael Sieger Michael Sandfort Michael Mahr Michael Gras Michael Grabchak Michael Eichenberger qlcMatrix Michael Bedward Melvin M. Varughese Maximilian Held Mauro Bernardi Matthias Wangler Matthias Kahm nadiv Matthew Salganik extremogram ClustVarLV elec.strat GPC waved VisuClust antitrust Rchemcpp cuttlefish.model EntropyEstimation multiAssetOptions packcircles DiffusionRgqd qmethod MCS changeLOS grofit Nicholas B. Larson Neo Christopher Chung Neil A. Weiss Minato Nakazawa Mikhail A. Beketov Michela Battauz Michael Sweeting Michael Cysouw Matthew Wolak Ivor Cribben Evelyne Vigneau Hua Yang Jordan Ko Marc Raimondo Georg Ohmayer Charles Taragin Guenter Klambauer Jean−Paul Robin Lijuan Cao Carlo Rosa David Eppstein Etienne A. D. Pienaar Aiora Zabala Leopoldo Catania Jan Beyersmann Maik Kschischo jackstraw wPerm pyramid HMMCont equateIRT pipe.design qlcVisualize ICC Marieke van der Maria Mercedes Mario Cortina−Borja Marie Paturel Maria L. Rizzo Maria Karlsson Maria Iannario Maria Amalia Jacome Margret−Ruth Oelker SWMPr Marco Smolla Marco Munda pawacc Marcin Studnicki Marcello Chiodi Manuela Azevedo Manuel Truppia cooptrees Majid Sarmad− paramlink Madeleine Thompson PKtools M. G. M. van Loon M. F. Kelly M. F. Carfora López−Moliner Joan Lyamine Hedjazi Luz Marina Rondon elec DMwR sglasso Lucia Tamburino Lucia Spangenberg Luca Weihs Luca Pozzi Luca Agnelli ReliabilityTheory Lothar Reichel Lorenzo Mercuri Lixi Yu Maaten−Theunissen Gregorio−Dominguez funreg phyloland CircNNTSR survPresmooth gvcm.cat ccChooser FLEDA tvm CircOutlier GaDiFPT quickpsy mQTL BayesGESM frt bcool TauStar ARAMIS FBN irlba MixedTS QualInt Marcus W. Beck Marco Geraci Manuel Fontenla Magnus Dehli Vigeland M. Suzette Blanchard Luke Miratrix Luis Torgo Luigi Augugliaro Louis Aslett John Dziak Louis Ranjard Juan Jose Fernandez−Duran Ignacio Lopez−de−Ullibarri Margret Oelker Konrad Debski Ernesto Jardim Juan Manuel Truppia Azade Ghazanfarihesari A. Buonocore Linares Daniel Jean−Baptiste Cazier Heleno Bolfarine Giangiacomo Bravo Hugo Naya Emin Martinian Antonietta Mira Adrian Andronache Jim Baglama Edit Rroji NeuralNetTools lqmm optrees IBDsim TEQR textreg performanceEstimation dglars PhaseType Konstantin Sering svs Kevin P. Barry Kevin Chang Kevin Arbuckle Kenneth Buker Ken Kelley titecrm MMST hglasso MOJOV Kaisa Välimäki Jungsik Noh Julian Wolfson Juan Luis García−Castaño Juan David Velasquez Josue Moises Polanco−Martinez Josselin Noirel fdrci interferenceCI Jose A. Castellanos−Garzon Jorge Gonzalez Burgos Jonas Stein catspec RSelenium John Bunge John Braun Johannes Zimmermann Johan Segers Joel Tarning Jodavid Ferreira metansue Joan Maynou Jingqin Luo Jing Tao CoxPlus LearnEDA CIFsmry Jianan Tian Jesse Garrison Jens Keilwagen Jenny Häggström Dominance infoDecompuTE MBESS Rhh W2CWM2C breakaway CHsharp BacArena spatialTailDep PharmPow waldwolf MEET DiagTest3Grp Depela MaXact audiolyzR PRROC Koen Plevoets Ken Cheung Keith Halbert Kean Ming Tan Ke−Hao Wu Joshua Millstein Joseph Rigdon John Hendrickx John Harrison Joaquim Radua Jing Peng Jim Albert Jianing Li Knut Krueger Katya Ruggiero Keke Lai Jussi Alho Josue M. Polanco−Martinez Amy Willis Douglas G. Woolford Eugen Bauer Anna Kiriliouk Frank Kloprogge Claudio Souza Erola Pairo Chengjie Xiong Andrew C. Chou Chenliang Xu Eric Stone Jan Grau corregp dfcrm crackR sparseBC RYoudaoTranslate cit RI2by2 perturb rDVR occ PanelCount LearnBayes crskdiag J. M. Calabrese Ioannis Ntzoufras Ioannis N. Athanasiadis Inmaculada Pérez−Bernabé Ian Donaldson Héctor Villalobos invGauss Hyun Jung Park Gordon J. Ross paleofire Giorgio Vacchiano Giorgio Arcara Gilda Garibotti German Aneiros Perez Gerhard Schöfl’ mgraph TFMPvalue Gary Anderson G. Roma AnthropMMD Frederic Commo Frank A. Schmid phylosignal FAmle Francis Roy−Desrosiers edgar cpm Rothermel erpR nltm PLRModels reutils AMA HumMeth27QCReport nplr lossDev Håkon K. Gjessing Global Paleofire Working Group George Owusu Ge Tan Frederic Santos Francois Keck Francois Aucoin Bharat Patil G. N. U. Scientific Library Davide Ascoli Anna Petrova Alexander Tsodikov Ana Lopez Cheda Gerhard Schöfl Aneesh Raghunandan F. M. Mancuso Brian M. Bot Christopher W. Laws Haplin GCD qrfactor TKF KappaGUI rleafmap FAdist ringscale Constantinos Tsirogiannis Clive Loader Claudio A. Vasconcelos Claudia Martinez−Araneda Clare Buckeridge gooJSON Christophe Demattei pARccs nets Christian M. Hafner ddepn LogisticDx dae HighDimOut surveyeditor Chao Ye apt ChangJiang Xu georob PhyloMeasures locfit iRegression dgmb PKNCA spatclus covLCA dslice GWsignif Daniel Haase Christopher Steven Marcum Christiane Raemsch Christian T. Brownlees Christian Bender Chris Dardis Chris Brien Cheng Fan Char Leung Changyou Sun Brody Sandel Catherine Loader A. Lima Neto Alba Martinez−Ruiz Bill Denney Christophe Dematte Aurelie Bertrand Bo Jiang cccd anim.plots halp informR PARccs dynamo bootfs survMisc asremlPlus TSMining ComplexAnalysis erer Xiuquan Wang Xiaoyao Yang Xiaomeng Ni Xiaolin Zhang Xiaohong Li Xiao Rong Xiao Lei Xiangwei Zhu Xavier Fernández Wouter D. Weeda Wouter BuytaertWolfgang Viechtbauer Wolfgang PoessneckerWobbe P. Zijlstra Wim de Leeuw William Nicholson William Hughes William G. Jacoby Willem Daniel Schutte Will Lowe Will Kurt Westa Domanova Wei E. Liang Wayne Oldford Wayne Jones Waqas Ahmed MalikWacek Kusnierczyk W. J. Braun Volkmar Liebscher Vladislav Navel Vitaly Efremov Vipavee Trivittayasil Vinh Nguyen Vincent Nijs Vincent CalcagnoVijay Krishnamurthy Victoria N. Nyaga GeneF rSCA freestats SPMS RDataCanvas malaria.em deepnet RDota mixtox ggmcmc arf3DS4 topmodel metafor MRSP fwdmsa SigWinR DTMCPack MConjoint optiscale SOPIE events ScrabbleScore ksrlive hiPOD qqtest R2PPT vmv rbenchmark MPV gromovlab timetools stheoreme EEM inference radiant glmulti Covpath CopulaDTA Steve Kalke Steve Juggins Steve Buyske Steve Bronder Stephen Oswald Stephen Meyers Stephen L. R. Ellison Stephen Horng−Twu Lihn Stephen A. Sefick Jr Stephanie Locke Stephane Mikael Bottine Steinar Engen Steffen Moritz Steffen Kothe Steffen Greilich Stefano Meschiari Stefano Benati Stefanie Hieke Stefan Hengl Stefan Boehringer Stefan Behrendt Stavroula A. Chrysanthopoulou Statistics Norway Stanley E. Lazic Stan Yip Stamatis Kalogirou Sophie Ancelet Song Cai Sonar Inc Sofia Morfopoulou Soeren Havelund Welling Soeren Braehmer Slawomir Jarek Sina Rueeger Simulistics Ltd Simon Thornley Simon Spencer Simon Schwab Simon Guest Simon Bond Siew−Leng Teng Siakoulis Vasileios Shuichi Kawano Shuangcai Wang cabootcrs clusterCons NHEMOtree pgnorm rioja mmlcr PANICr FlexParamCurve astrochron metRology LIHNPSD StreamMetabolism optiRum SPRT poilog imputeTS cmsaf libamtrack latex2exp qVarSel minPtest mota parallelize.dynamic lm.beta MILC RcmdrPlugin.EHESsampling desiR weatherr lctools Geneclust drmdel jSonarR metaMix forestFloor relen mvnormtest uniPlot Simile PubBias interventionalDBN emov OpenMPController mreg knorm acp spcr pbs Roman Pahl Roman Jugai Roman Guchenko Romain Azais Roland Tanglao Rohit Arora Roger Woods Rodrigo Buhler Rodrigo Azuero Melo Rodrigo Aluizio Roberto ImpicciatoreRobert P. Bronaugh Robert Maier Robert Lowe Robert J. Gray Robert Hable Robert G. Garrett Robert Clements Rob Erhardt Rob Crouchley Rick Pechter Richie Cotton Richard Bourgon Richard Ambler Richard A. Bilonick Rich Nielsen Riadh ZaatourRenee Gonzalez Guzman Reid F. Thompson Recai Yucel Raul Cruz−Cano Rasmus Bååth Raphael W. Majeed Raphael Mourad Randall Shane Ram Narasimhan Ralf Strobl Rahul Premraj Rahul Mehta Rahim Alhamzawi Rafael Dellen R. N. Edmondson R. Lafosse R. Hackathon R. H. BaayenR. C. S. Soil Survey Staff Qunhua Li Quinn N. Lathrop Qiao Kang Pros Naval Po−Hsien Huang Po Su Piter Bijma Piers Harding Pierre Lefeuvre Pierre Alquier Philippe Courcoux Philipp Limbourg Philipp Hermann Philipp Angerer Philip C. Schouten Phil Novack−Gottshall Phil Joubert Petr Novak Petr Matousu Peter Xenopoulos Peter Lipman Peter Kampstra Peter Dutton Peter Carbonetto Peter Biber GroupSeq anametrix rodd EstSimPDMP ig.vancouver.2014.topcolour covmat icapca analyz gmapsdistance forams MAPLES phalen attfad rvmbinary permax imprProbEst rgr stppResid ABCExtremes sabreR MicroStrategyR learningr intervals trotter merror caseMatch hawkes NORRRM RadOnc mlmmm FRCC beepr violinmplot HiCfeat geoPlot weatherData km.ci mailR RDIDQ Brq PepPrep blocksdesign concor phylobase languageR sharpshootR idr cacIRT treeperm mopsocd lsl NORTARA SE.IGE RSAP BoSSA ISBF FreeSortR ipptoolboxFractalParameterEstimationnbconvertR Table1Heatmap ecospace SmithWilsonYieldCurve SeqGrapheR sievetest Sabermetrics CGene beanplot EurosarcBayes varbvs cycloids Oliver Stirrup Oleh Komashko Oleg Yegorov Nolan A. Wages Noemie Robil Noemi Andor Noboru Nomura Noah Silverman Niraj Poudyal Nils Arrigo Nikolaos Giallousis Nikola Kasprikova Nicole Mee−Hyaang Jinn Nicolas Robette Nicolas E. Campione Nicolas Baskiotis Nick Kennedy Nick Guy Nick Bond Nicholas Vinen Neville Jackson Nelson Ray Neil Diamond Neal Richardson Nathaniel Malachi HallinanNathan WelchNathan S. Watson−Haigh Nathan G. Swenson Nathan Esau Nathan Carroll Natalya Pya Nadine SchoeneN. Benjamin Erichson Myles English Murzintcev Nikita Mun−Gwan Hong Muhammad Kashif Moritz BergerMonica Fernanda Monica Rinaldi Calderon−Santiago Mohsen Ahmadi Mohit Dayal Mohamed SoueidattMizanur Khondoker Miroslav Morhac Miriam Marusiakova Mini Huang Ming Wang Milan Hiersche Mikko Rönkkö Miki Horiguchi Mike Wurm Mike W.−L. Cheung Mike Malecki Mihai Tivadar Miguel A. R. Manese Michele Grassi Michele De Meo Michel Semenou Michel Prombo Michel Meulders Michel Crucifix Michel Berkelaar Michal Burda Michal Bojanowski Michael Silva Michael Scholz Michael Salter−Townshend Michael S. Pratte Michael Rustler Michael North Michael Netzer Michael Matschiner Michael LundholmMichael LaimighoferMichael J. Grayling Michael Gutkin Michael G. CampanaMichael E. Schaffer Michael Cyosuw Michael Chajewski Michael Buecker Michael Anderson Micha Sammeth Meryam Krit Menachem Fromer Melanie Wilson Melanie Quintana Meik Michalke Mehrad Mahmoudian Maxime Wack Maxim Yurchuk Max Sommerfeld Max Hughes Max Conway Mauro Sereno Maurits Kaptein covBM nlWaldTest rlm pocrm KANT expands mvprpb makeProject StVAR R2G2 Scale LTPDvar severity GDAtools MASSTIMATE TreeRank forestmodel radar hydrostats uncompress dmm filterviewR VizCompX crunch HyPhy brewdata PCIT lefse m4fe oglmx scam NPMPM rsvd hydrogeo ldatuning MDimNormn COMBIA DIFtree PROTOLIDAR MScombine SimuChemPC cepp Funclustering optBiomarker Peaks forensic MSIseq geesmv postgwas matrixpls SSRMST ordinalNet metaSEM Stack OasisR SQLiteDF CircE bethel CompR OutrankingTools plfm palinsol lpSolve lfl intergraph blsAPI clickstream VBLPCM hbmem kwb.hantush SchemaOnRead BiomarkeR ageprior sifds SurvRank phaseR SlimPLS corrsieve rtf qlcData rela lcda bdoc ops EWGoF xhmmScripts MISA BVS XiMpLe varhandle cosmosR CfEstimateQuantiles cope adaptsmoFMRI gsheet lira RStorm Marius Wirths Mario Quaranta Mario Pineda−Krch Marina Evangelou Marek Spruzina Marek Molas Marcus Scherl Marcus Rowcliffe Marcus G. Daniels Marco Torchiano Marco Lugo Marco Giordan Marco D. Visser Marco AndrelloMarcelo Goulart Correia Marcelo C. PereiraMarcelo Brutti Righi Marcello D’Orazio Marc VandemeulebroeckeMarc Johannes Marc J. Lajeunesse Marc Fortier Marc Fasel Marc Delord Marc Adams Maogui Hu Manel Salamero Mandy Vogel Majid Einian Mahmoud Mosalman Yazdi Mahmoud Ghandi Magdalena Chrapek Maciek Sykulski Maarten Kruijver M. Shahidul Islam M. Sawada M. Kondrin M. D. Vreeburg Lutz Prechelt Lutz P. Breitling Luke Jostins Lukasz Nieweglowski Lukas Burk Luis Carvalho Lucas Venezian PovoaLucas Leemann Luca Sartore Luc Wouters Luana Cecilia Meireles Loïc Schwaller Lora Murphy Lisa McFerrin Lionel Guy Ling−Yun Wu Lindsay V. Clark Lindsay P. Scheef Libo Sun Liang Liu Liang Jing Lianbo Yu Liam D. Bailey LiLin−Yin Li Qinglong Li Hua Yue Lev Kuznetsov Letaw Alathea Leonardo Di Donato Leonardo Castelo Branco Leo Guelman Leeyoung Park Lee Kelvin Leah R. Jager Lauri Mehtatalo Laura Chihara Larsen Kim Lars Gidskehaug Lam Opal HuangLaercio Junio da SilvaL. W. Pembleton L. Andries Kyle Bittinger Kyle A. Caudle Kurt Wollenberg Kuangnan Xiong Krzysztof Ciupke Kristina Veljkovic Kristian E. Markon Kobi Perl Kiwamu Ishikura Kitty Lo Kiran Garimella Kimberly F. McManus Kevin Toohey Kevin R. Sanft Kevin Kokot Kevin Keenan Kevin Dunn OneArmPhaseTwoStudy SetMethods GillespieSSA PAGWAS b6e6rl HGLMMM validator activity hdf5 effsize CANSIM2R ber aprof ConnMatToolsRcmdrPlugin.RMTCJags LSDinterface riskR StatMatch adaptTest pathClass metagear mfblock STI MIICD lm.br ibeemd TestScorer childsds BCDating topsis gkmSVM Records rpca DNAprofiles WaveCD MATTOOLS RGrace SocrataR agsemisc dagR Mangrove clv tRakt kolmim precintcon StratSel spMC mpm biasbetareg saturnin likelihood HDMD genoPlotR CRF polysat MAR1 ORCI phybase geoCount fmt climwin CMplot StatMethRank branchLars injectoR captioner hrr phenability uplift IFP astro phitest lmfor CarletonStats Information beadarrayMSV TRD laercio StAMPP mokken qiimer flowfield aaMI roughrf mpcv MetFns infutil mHG Rjpstatdb RAPIDR gsalib popRange SimilarityMeasures StochKit2R LRcontrast diveRsity pid Josemir Neves Josef Brejcha Jose M. Pavia Jose Jimenez Jose E. Lozano Alonso Jose D. Loera Jos Feys Jort de Vreeze Jorge N. Tendeiro Jordie Croteau Jordan Mackie Jonas Mueller Jonas Klasen Jonas Haslbeck Jon Lefcheck John Whalen John Waller John Shea John R. Dixon John Fieberg John Ferguson John Curtin John Boik Johannes Radinger Johannes Hüsing Johann Laurent Johann Gagnon−BartschJoerg Schaber Joel Gombin Joel Carlson Joe Sexton Jochen Knaus Joao Pinelo Silva Joachim Zuckarelli Jo Hardin Jiri Kadlec Jinyoung Yang Jingfan Sun Jimmy Oh Jim Yi Jiehuan Sun Jiayi Liu Jiat Chow Tan Jiang Li Ji−Ping Wang Jesus Maria Rodriguez Rodriguez Jesse Krijthe Jerome Collet Jeremy Thoms Hetzel Jeremy Stanley Jeremy Oakley Jens J. Rogmann Jennifer Kirk Jenine K. Harris Jelena Kovacic Jeffrey M. Dick Jeffrey H. Gove Jeffrey Dunn Jeff Binder Jedrzej S. Bojanowski Jed Long Jean−Pierre Rossi Jean−Francois CoeurjollyJean V. Adams Jean Sanderson Jaynal Abedin Jayanth Varma Javier Hidalgo CarrioJavier G. Corripio Javier Fernandez−Macho Javier Celigueta Muoz Jason Wilson Jason W. Morgan Jason D. Nielsen Jared Murray Jan Wolfertz Jan Ulbricht Jan Saputra Mueller Jan Michael Yap Jan Klaschka Jan Gorecki Jan Gertheiss Jan Dul Jan C. Thiele Jamie Sergeant James Ridgway James P. HowardJames Holland Jones James Hiebert James Grange James Eustace Jalpa Joshi Dave Jakub Stoklosa Jairo A. Fuquene Jaime Egido Jae H. Kim Jack G. Gambino spi BivarP GoFKernel abodOutlier mem Sofi npIntFactRep apaStyle PerFit fat2Lpoly cycleRtools plmDE hit mgm piecewiseSEM walkscoreAPI EasyMARK knitLatex gbm2sas SightabilityModel averisk lmSupport mixlow fishmove resper dashboard ruv pheno qualvar radiomics gbev snowfall OptiQuantR quantification biwt WaterML atmcmc powerr TableToLongForm SUE SQDA CRAC probemod DOSim SPECIES tablaxlsx Rtsne subrank trapezoid tidyjson SHELF orddom rmac ergmharris iRepro CHNOSZ sampSurf compoisson bursts sirad wildlifeDI rich dvfBm LW1949 adwave edeR jrvFinance allanvar insol wavemulcor PlotPrjNetworks PCS boolean3 crimCV bfa AlgebraicHaploPackage lqa rdetools WCQ BlakerCI Rbitcoin ordPens NCA RNetLogo RII EPGLM waterfall demogR udunits2 trimr MSVAR RGoogleAnalyticsPremium PL.popN ClinicalRobustPriors dynBiplotGUI vrtest pps 10000 10000 500 500 1000 1000 Num Clusters Num Users Num Users Num Users 50 100 100 50 5 10 10 10 5 1 1 1 1 5 10 50 500 5000 1 2 5 10 20 50 100 1 2 5 10 20 1 2 5 10 20 50 Num Nodes Degree Num Views Num Packages Figure 2. Clusters. Figure 3. Degree GS . Figure 4. Views. Figure 5. Packages. In Fig. 1, only a subset is shown due to space limits. There 5000 are many more small clusters as those at the bottom of the figure. The community has one large cluster, the largest con- 500 Num Packages nected component (LCC) of the graph, containing 8985 nodes 50 which are either users or packages. There are many smaller clusters with few users contributing to packages and also 5 many users that contribute only to one single package. Figure 2 shows the number of clusters versus the number 1 1 2 5 10 20 of nodes in it (log-log scale). The LCC is depicted by the dot Num Dependencies at the very bottom right corner of the figure. The majority of clusters has only few or just a single user package tuple. This Figure 6. Dependencies. also means that only users within the single largest connected component will be relevant for our analysis. Since we heavily rely on user degree in the social graph GS , many users in the small clusters will have very low importance. Figure 3 shows the degree distribution of the user graph GS . Low degree of many users is explained by the large number of small clusters, which are mainly individual contributors of single packages. The graph GS consists of 11189 users. A fraction of 14% has a degree of 0 and 60% of users have a degree smaller or equal 3. The median4 degree is 85. A fraction of 0.7% of users (74 users) have a degree larger than 85. Such degree distributions are typical in online communities. The next Fig. 4 shows the relationship between CRAN task Figure 7. User dependencies. views and users. These views are interpreted as skills and ex- pertise ranking is performed within the context of individual task views. In total, 33 views exist. 7316 users are not associ- 4.2 Experiments Setup ated with any view (because their packages are not listed in any view). Thus, those users will not be considered in the for- In our experiments, we sampled a random set of CRAN task mation algorithm. 3873 users are associated with one or more views representing the demanded team skills. The crossover views. The median value for the number of views within this probability is set to 0.7 and the mutation probability to user segment is 11. This provides already a good diversity in 0.05. The metric weights for fitness calculation are set to terms of users having different skills. wscore = wcost = wdistance = 31 . In each experiment run, The next step is to analyse the relationship between users a population of 200 individuals plus 5 individuals for elitism and software packages. Figure 5 shows the number of users has been created. We evaluate the quality of the expertise over packages. The median value for the number of software mining approach by checking key metrics such as degree of a packages is 20. The dependencies of packages are visualized user, number of packages in a given view (where the user is by Fig. 6. 4550 packages have exactly one dependency (the R top-ranked), and number of all packages. environment). The median value is 10. Finally, Fig. 7 shows the average number of dependencies by the number of users. The median value for the average 4.3 Qualitative Evidence number of dependencies is clearly 2. A team with the highest fitness for 5 skills is depicted by Fig. 8 and detailed in Table 1. The team has an average expertise score of 0.8, a cost of 0.6, and distance 1.0. These are excellent values. A perfect fitness of 1.0 is not attainable because there 4 The median is used to separate the higher half of the user popu- is a tradeoff between expertise and cost. Indeed, the maximum lation from the lower half. achievable fitness depends on the selected skills. Table 1. Example of team recommendation for 5 skills. Skill User Rank Score Score (GS ) Ch. Deg. (GS ) Pkg All Pkg Deg. (T) Num. Math. A. Gebhardt 9 0.07681 0.00027 201 36 2 11 4 Bayesian M. Mächler 1 0.87039 0.00150 0 276 1 57 4 Meta Analysis T. Lumley 2 0.36787 0.00060 21 86 4 34 4 Social Sciences W. N. Venables 8 0.19568 0.00021 509 50 5 10 4 Psychometrics K. Hornik 1 0.85807 0.00113 4 318 3 71 4 100 110 120 130 140 150 0.85 0.80 Thomas Lumley (MetaAnalysis) Best Individual Fitness Overall Fitness 0.75 0.70 Brian D. Ripley () Martin Mächler (Bayesian) 0.65 90 0.60 80 2 4 6 8 10 2 4 6 8 10 Iteration Iteration Figure 9. P -fitness. Figure 10. I-fitness. Albrecht Gebhardt William N. Venables (NumericalMathematics) (SocialSciences) In the following we answer the question whether an in- creasing number of skills results in more disconnected teams. Kurt Hornik (Psychometrics) Fig. 11 compares different populations with an increasing number of skills (from 5 to 9 skills depicted by S5 to S9). Figure 8. Example team spanning 5 skills. S5 40 80 0 Table 1 shows further team member details. Rank is the 2 4 6 8 10 user rank within the specific task view (community rank for S6 Index the given skill). Score is the numeric ranking score based on 30 our advanced expertise mining model and Score (GS ) is the 0 ranking score computed in GS using a standard PageRank. 2 4 6 8 10 This comparison essentially demonstrates the impact of our S7 Index advanced context-based ranking model (see [27, 28]). Ch. is 30 the ranking change (context-based vs. standard PageRank). 0 One can see that context has a high impact in terms of ranking 2 4 6 8 10 position. We positively validated these results by checking the S8 Index online profiles of the top-ranked users because most users have 30 public Web sites. Deg. (GS ) depicts the degree (number of co-authors) in GS . High degree typically means high commu- 0 2 4 6 8 10 nity standing. Pkg depicts the number of packages in a given S9 Index view and All Pkg all packages of the given user. Deg. (T) is the team degree. Here we see a perfectly connected team 30 where each member is connected to all other members. The 0 user ‘Brian D. Ripley’ is selected as the coordinator. 2 4 6 8 10 Figure 11. Skills vs. number of disconnected team. 4.4 Performance Evaluation We show the convergence of fitness values for both entire pop- ulations (depicted as P -fitness in Fig. 9) and for the best in- The x-axis shows the number of disconnected team mem- dividuals in each population (depicted as I-fitness in Fig. 10). bers and the y-axis the number of teams within the population Convergence means that no major changes between one iter- (total number of populations is 205). The figures show that ation to the next iteration are observed. 5 skills have been by increasing the number of skills the number of teams where selected randomly. Different color codes depict 5 runs of GA only few members are disconnected decreases. More skills to formation algorithm. The best teams were identified after 7 be satisfied by individual users increases the chance that team iterations. Population fitness started to settle at 10. members are disconnected from the rest of the team. 5 Conclusions Digital Ecosystems Technologies (DEST), 2012 6th IEEE In- ternational Conference on, pp. 1–6, (June 2012). This work introduced team formation mechanisms for soft- [12] Jungpil Hahn, Jae Y. Moon, and Chen Zhang, ‘Emergence of ware ecosystems. We apply a genetic algorithm including a new project teams from open source software developer net- works: Impact of prior collaboration ties.’, Information Sys- novel extension called relationship-driven mutation. In devel- tems Research, 19(3), 369–391, (2008). opment teams, performance and quality are affected by the [13] Geir K. Hanssen, ‘A longitudinal case study of an emerging programming skills and domain experiences of the project’s software ecosystem: Implications for practice and theory’, J. team members. We apply an advanced expertise mining ap- Syst. Softw., 85(7), 1455–1466, (July 2012). [14] Qiaona Hong, Sunghun Kim, S. C. Cheung, and Christian proach to address this problem. Empirical results confirm the Bird, ‘Understanding a developer social network and its evo- applicability of our presented methods. lution’, in Proceedings of the 2011 27th IEEE International Future work includes the following aspects. Estimating col- Conference on Software Maintenance, ICSM ’11, pp. 323– laboration cost is not trivial and may depend on many fac- 332, Washington, DC, USA, (2011). IEEE Computer Society. tors, such as physical co-location, geographical distribution [15] S. Jansen, A. Finkelstein, and S. Brinkkemper, ‘A sense of community: A research agenda for software ecosystems’, in (including issues related to time difference), or cultural fac- Software Engineering - Companion Volume, 2009. ICSE- tors. We will analyse and model cost of collaboration. We Companion 2009. 31st International Conference on, pp. 187– will perform further evaluations of the approach in two di- 190, (May 2009). rections. First, we want to validate results by engaging com- [16] Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Eliz- abeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, munity members of the R ecosystem to get direct feedback and John Horton, ‘The future of crowd work’, in Proceed- on formation and expertise ranking results. Second, we will ings of the 2013 Conference on Computer Supported Cooper- broaden team formation experiments for other types of soft- ative Work, CSCW ’13, pp. 1301–1318, New York, NY, USA, ware ecosystems including industrial and company internal (2013). ACM. software ecosystems. An additional area of investigation is [17] Theodoros Lappas, Kun Liu, and Evimaria Terzi, ‘Finding a team of experts in social networks’, in Proceedings of the the consideration of formal skill frameworks such as SFIA5 . 15th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’09, pp. 467–476, New York, NY, USA, (2009). ACM. REFERENCES [18] Anirban Majumder, Samik Datta, and K.V.M. Naidu, ‘Ca- pacitated team formation problem on social networks’, in Pro- [1] Newcastle University, Roulette wheel selection. ceedings of the 18th ACM SIGKDD international conference http://goo.gl/5CGi8t, Jan. 2007. on Knowledge discovery and data mining, KDD ’12, pp. 1005– [2] Aris Anagnostopoulos, Luca Becchetti, Carlos Castillo, Aris- 1013, New York, NY, USA, (2012). ACM. tides Gionis, and Stefano Leonardi, ‘Online team formation in [19] Konstantinos Manikas and Klaus Marius Hansen, ‘Software social networks’, in Proceedings of the 21st international con- ecosystems - a systematic literature review’, J. Syst. Softw., ference on World Wide Web, WWW ’12, pp. 839–848, New 86(5), 1294–1306, (May 2013). York, NY, USA, (2012). ACM. [20] Tom Mens, Malick Claes, Philippe Grosjean, and Alexander [3] John Anvik and Gail C. Murphy, ‘Reducing the effort of bug Serebrenik, ‘Studying evolving software ecosystems based on report triage: Recommenders for development-oriented deci- ecological models’, in Evolving Software Systems, eds., Tom sions’, ACM Trans. Softw. Eng. Methodol., 20(3), 10:1–10:35, Mens, Alexander Serebrenik, and Anthony Cleve, 297–326, (August 2011). Springer Berlin Heidelberg, (2014). [4] Jakob Axelsson and Mats Skoglund, ‘Quality assurance in [21] Tom Mens and Philippe Grosjean, ‘The ecology of software software ecosystems: A systematic literature mapping and re- ecosystems’, IEEE Computer, 48(10), 85–87, (2015). search agenda’, Journal of Systems and Software, 114, 69 – [22] David G. Messerschmitt and Clemens Szyperski, Software 81, (2016). Ecosystem: Understanding an Indispensable Technology and [5] Justin M. Beaver and Guy A. Schiavone, ‘The effects of de- Industry, MIT Press, Cambridge, MA, USA, 2003. velopment team skill on software product quality’, SIGSOFT [23] Melanie Mitchell, An Introduction to Genetic Algorithms, Softw. Eng. Notes, 31(3), 1–5, (May 2006). MIT Press, Cambridge, MA, USA, 1998. [6] Christian Bird, David Pattison, Raissa D’Souza, Vladimir [24] Daniel Schall, ‘Expertise ranking using activity and contex- Filkov, and Premkumar Devanbu, ‘Latent social structure in tual link measures’, Data Knowl. Eng., 71(1), 92–113, (2012). open source projects’, in Proceedings of the 16th ACM SIG- [25] Daniel Schall, Service Oriented Crowdsourcing: Architecture, SOFT International Symposium on Foundations of Software Protocols and Algorithms, Springer Briefs in Computer Sci- Engineering, SIGSOFT ’08/FSE-16, pp. 24–35, New York, ence, Springer New York, New York, NY, USA, 2012. NY, USA, (2008). ACM. [26] Daniel Schall, ‘Formation and interaction patterns in social [7] Barry W. Boehm, ‘Improving software productivity’, Com- crowdsourcing environments’, Int. J. Commun. Netw. Dis- puter, 43–47, (1987). trib. Syst., 11(1), 42–58, (June 2013). [8] Jan Bosch, ‘From software product lines to software ecosys- [27] Daniel Schall, ‘Measuring contextual partner importance in tems’, in Proceedings of the 13th International Software Prod- scientific collaboration networks’, Journal of Informetrics, uct Line Conference, SPLC ’09, pp. 111–119, Pittsburgh, PA, 7(3), 730 – 736, (2013). USA, (2009). Carnegie Mellon University. [28] Daniel Schall, Social Network-Based Recommender Systems, [9] M. Claes, T. Mens, and P. Grosjean, ‘On the maintainability Springer International Publishing, 2015. of cran packages’, in Software Maintenance, Reengineering [29] M. Srinivas and L.M. Patnaik, ‘Genetic algorithms: a survey’, and Reverse Engineering, 2014 Software Evolution Week, pp. Computer, 27(6), 17–26, (June 1994). 308–312, (Feb 2014). [30] Cédric Teyton, Marc Palyart, Jean-Rémy Falleri, Floréal [10] Massimiliano Di Penta, Mark Harman, and Giuliano An- Morandat, and Xavier Blanc, ‘Automatic extraction of de- toniol, ‘The use of search-based optimization techniques to veloper expertise’, in 18th International Conference on Eval- schedule and staff software projects: An approach and an em- uation and Assessment in Software Engineering, EASE ’14, pirical study’, Softw. Pract. Exper., 41(5), 495–519, (April pp. 8:1–8:10, New York, NY, USA, (2014). ACM. 2011). [31] Hyeongon Wi, Seungjin Oh, Jungtae Mun, and Mooyoung [11] R.P. dos Santos and C.M.L. Werner, ‘Treating social dimen- Jung, ‘A team formation model based on knowledge and col- sion in software ecosystems through reuseecos approach’, in laboration’, Expert Systems with Applications, 36(5), 9121 – 5 https://www.sfia-online.org 9134, (2009).