Crowdsourcing is basically a type of participative online activity where a person or an organization requests a loosely defined group of people (crowd) to carry out tasks for them using open calls. The crowd undertakes the tasks voluntarily driven by motivation which is not supposed to be financial reasons in all the cases [1]. A new term called Crowdsourced Software Engineering has also emerged to describe the phenomena of using crowdsourcing for various software engineering tasks as it is very popular nowadays [2].
Among many popular crowdsourcing platforms used in software engineering, the Question & Answer (Q&A) website Stack Overflow is used daily to share knowledge globally by millions of software professionals. Therefore, Stack Overflow data can reveal important patterns which will help to get an idea about how software professionals share knowledge in a global scale. Eventually the findings will also help global software companies and crowdsourcing platforms to formulate and reevaluate their strategies and incentive criteria. The aim of this study is to perform data mining on Stack Overflow data to discover patterns of global user distribution and contribution.
Stack Overflow caters wide range of computer programming subjects or topics. In 2015 it has recorded 5.7 billion page views as the number of registered Stack Overflow users was reaching 5 million [3]. The registered users can post questions and answers on the website. All the content is freely available for the public for viewing. It also utilizes a comprehensive reputation management system as Atwood states in one of his blog posts in 2009, that he believes in community moderation [3][4]. Schenk et al. in 2013 in their research has found out that contribution is highest in Europe and North America. Then Asia, which is mostly represented by India; Oceania contributes not as much as Asia, but more than South America and Africa combined. However, they base their research on the transfer of knowledge. Specifically, who (country) raises the question and who (country) answers it [5]. However, it will be beneficial also to perform a comprehensive study on the user distribution across the globe with respect to their contribution and reputation.
Reputation measurement can also be manipulated by users who play around with the gamification methods of Stack Overflow [6]. To tackle this issue, in this research the number of questions and answers posted will be also used to represent the contribution.
When comparing these measurements across users, there is a need of normalization of the figures according to the length of membership for the users. For example, Morrison and Murphy-Hill has used the Reputation per Month without just taking Reputation as the measurement in their research [7]. Similarly, number of answers posted per month and number of questions posted per month can be used in this research in addition to the reputation.
Methodology of this research is based on the following phases specified by Fayyad et al. for discovering knowledge in databases [8].
The public data dump of all user-contributed content on the Stack Exchange Network shared in The Internet Archive is used as the main data source for this research. Following files from Stack Exchange data dump which has been published on 8th December 2017 has been downloaded from The Internet Archive for this study.
• Users.xml (2.36 GB)
• Posts.xml (56.3 GB)
Then the structure of the above xml files were studied to select the most appropriate data items. The Entity Relationship Diagram of the schema is shown in Figure 1.
Data mining tasks could not be performed directly on top of downloaded raw XML files due to large file size, flat structure of XML files and unbreakable nature of XML files. Therefore, raw data had to be loaded into another format which Apache Spark can utilize its in-memory processing and parallelization power. A MySQL relational database is used for this purpose. A Python script has been written for each raw XML file which was then executed using spark-submit script which is loaded in Spark's bin directory. The Table 1 shows the number of records loaded into respective MySQL tables.
Conversion of some of the data into appropriate forms was needed before starting data mining activities which are described below.
Since names of countries/locations have been specified in different formats in raw data, a special Python program was implemented to extract the country name accurately with the help of a free and open-source Python library named geodict (https://github.com/petewarden/geodict). In the end the location of 1,172,495 users were identified and saved in a new database table. This is 15.83% from all users and 80.24% of all the users who have specified a location.
Since tables have millions of data records, Spark with Python API was chosen leveraging the partition aware loading feature. The groupBy function and other built-in aggregate functions like count, avg in Spark were used. All the necessary aggregated data required for the research were generated with the help of Python scripts executed on Spark engine.
The aggregated data were sometimes needed to be merged prior to data mining. Spark's feature to join RDDs is utilized for this purpose.
For the numerical data, descriptive summary statistics were used to understand the distribution of data. Mainly the Spark function describe was used for this purpose.
The descriptive statistics, graphs generated by Oracle Data Visualization Desktop (ODVD) tool and Matplotlib were used to interpret and evaluate the results.
Country names of 1,172,495 users of Stack Overflow (15.83% from total users) and then 205 country names were identified in the subset under analysis. Top 50 countries sorted in the descending order of user count are presented in Table 2. As observed United States and India have marginally very high number of users which is more than 200,000 each. Collectively they represent 40% of total users. They are categorized as countries in Cluster 5. Cluster 4 countries have users between 30,000 and 75,000. UK, Germany, Canada, France and China belong to this category. Even though China has the world's highest population, its participation is not matching with the population. It could be due to language issues. This can be same for Russian Federation. Another notable observation is there are only 78 countries with more than 1000 identified users. Cluster 2 represents countries with more than 3000 and only some of them are in top 50 list. Cluster 1 represents countries with less than 3000 users which is not even included in the Table 2.
Above data has been merged with world population data for year 2015 published by United Nations, Population Division [9]. Then users per 1000 capita figure has been calculated for each country for further analysis.
The map in the Figure 2 displays how users per 1000 capita changes across the globe and the Table 3 presents the top 50 countries with users per 1000 capita in descending order. The main observation compared with user count ranking is United States falling to 17 th position while India does not even qualify in top 50. However, UK shows consistency in both and the biggest (population wise) country having highest participation. Iceland becomes the number one even though it does not even have sufficient users to be listed in the first list. The main conclusion that can be derived is that most European countries have higher participation per capita generally. The countries like New Zealand, Singapore, Israel, Canada, and Australia are also among the high participating countries. To compare contribution levels of average users of countries, the user contributions in the means of average reputation per user, average number of questions posted per user and average number of answers posted per user from each country have been analyzed. The Table 4 summarizes the rankings of countries which fall into top 20 of each category and has more than 500 users along with Russian Federation and India for their significance. The cells in blue background color displays the ranks within top 20 while cells with pink background displays rankings greater than 20 for the respective category.
As reputation and answer ranking relates to knowledge sharing, respectively Switzerland has become top country in both rankings while closely followed by UK and Germany. Sweden, Austria, and Israel are among top 10 of both rankings with most of other European countries. New Zealand, Austria and Canada contribute much as well.
However, India and Russian Federation have less contribution despite their large population. Another important observation is that most of countries who are reputed, and good answer providers are also good at asking questions. However, Italy, Ireland, Latvia, and Lebanon are basically question askers but not answer providers. Meanwhile Finland, Netherlands and Bulgaria have higher reputation and answering rate, but they do not ask many questions. In both user participation and contribution, European countries along with Israel, Australia, Canada, and New Zealand are highlighted from the rest of the world. These findings were cross evaluated by comparing with the ICT Development Indexes of countries provided by United Nations [10]. The major difference found was the underperformance of crowdsourcing activities of countries like South Korea and Japan who have good global ICT rankings. This situation can be further proven by comparing the findings with the IMD World Digital Competitiveness Ranking 2017 [11]. Even though this must be further analyzed, one reason can be the language barrier. Presence of some other popular alternatives to Stack Overflow also can be also another reason. Under presence of China and Russian Federation can be also due to this.
Stack Overflow data reveals important patterns in global crowdsourcing beneficial for software industry. The results on Global User Distribution and Contribution, clearly show that majority of the users are from USA and India. However, in both participation and contribution aspects, European countries along with Australia, Canada and New Zealand have higher rankings. It is also noted the less rankings of Japan, South Korea, Russian Federation, Brazil and China. Since these countries represent huge portion of world population, further studies should be carried out to find factors for this phenomenon.