INTRODUCTION

One of the most important challenges faced by Smart Cities is creating an ecosystem of public and private actors that reuse open data in order to produce IT services and products that both (i) would improve citizens' quality of life and (ii) would contribute to economic growth [32]. However, few open data portals in cities currently track data usage and consider the impact of data on deciding which datasets to maintain or what complementary datasets publish. Cities are not even aware of what kinds of apps are developed, using what data, and how many there are. Answering these questions is a significant research issue [30] that would allow prioritizing which categories of data must be published and maintained with respect to the applications that use them (i.e., impact that a category of open data generates).

To reverse this situation, publishing datasets as open data requires a decision support system to select those categories of datasets that offer higher potential to generate value [12]. Such a system must consider indicators about the impact of the already published open datasets, as well as the strategy of the Smart City. E.g., a small town could provide an open data portal with many high-quality datasets but the portal is rather unknown, and the technological fabric of the city is composed of small IT companies. Therefore, the goal of the city could be to extend the use of the open data portal by prioritizing those datasets that belong to categories that are likely to generate a large number of projects -though simpler ones that involve fewer people. On the other hand, a big city with consolidated open data portals may prefer opening datasets that could be used in complex and mature software applications that involve big teams, since it is more relevant to their specific technological industry context.

Unfortunately, to the best of our knowledge, Smart Cities lack such decision support system, mainly because the process of calculation of those indicators that would use the system is not a trivial task. According to Janssen et al. [14] , "there is no way to predict and calculate the return of investment (ROI) in advance [. . . ]". The main challenge is that open data has no value in itself; it only becomes valuable when used". Therefore, the main problem is that data owners have limited understanding on how open data is reused, thus lacking knowledge about the impact generated by reusing the published open data.

More reasonable indicators of the use of open datasets could help to identify which categories of datasets have more possibilities of being reused and, in this way, generate some type of economic impact to people or enterprises. In this sense, good indicators could come from the reuse of datasets within the open source community. The Tenth Annual Future of Open Source Survey [11] reflects the increasing adoption of pen source and highlights the abundance of organizations participating in the open source community. Concretely, this survey estimates that 65% of companies currently participate in open source projects. Open Source Software (hereon OSS) is considered to encourage the creation of SMEs and jobs, by providing a skills development environment valued by employers and retaining a greater share of generated value locally [8]. Focusing in Europe, a study estimated that the contribution of OSS to its economy was of 450 billion euro per year [7].

Based on these figures, an estimation of the use of the different categories of datasets by the OSS community could be a good indicator of their potential impact. Therefore, when Smart Cities make decisions on which data to publish, they could prioritize publication of data which allows a community of developers to generate impact and effectively release benefits of open data through OSS projects.

In this paper, we present an approach based on the estimation of indicators of the use of open datasets in OSS projects. The goal of this approach is to provide Smart Cities with a Decision Support System which provides an ordered list of categories of datasets most suitable to be published or maintained in their open data portal. To do so, we have carried out a set of actions aimed at estimating useful impact indicators related to the datasets of the same category already published by open data portals of other cities. Concretely, to calculate our proposed indicators we needed two kinds of data sources: (i) already published Smart City datasets (and their metadata) and (ii) OSS projects (together with information about them) which referenced the gathered datasets; i.e., we needed to know which open datasets were being used in which OSS projects. To collect already published open datasets, we chose Socrata [26] because it is one of the most used open data repositories, and notably by some of the most important US cities. We also measured the existence of potential reuses within a community in order to measure open data impact. To do this, we used GitHub [9], because it is the largest web-based distributed revision control and source code repository in the world, and the source of several empirical studies such as in Yu et al. [33].

Using the indicators obtained from these sources, we provide an Analytic Hierarchy Process (hereon AHP)-based [24] tool 1 that allows decision makers weigh these indicators, taking into account the reuse objectives of the city, to offer an ordered list of categories of datasets recommended to publish.

This paper is structured as follows: section 2 describes a new approach to select the most relevant categories of data to be published in a smart city open data portal. Section 3 presents toy samples of two different stereotypical smart cities using our approach and, to finish, section 4 summarizes other work related to the publishing of open data in Smart Cities.

USING REUSE INDICATORS BASED ON DATA FROM OSS PROJECTS IN GITHUB FOR SELECTING DATASETS TO OPEN

This section describes the steps that have been carried out to get an AHP process that allows classifying categories of dataset based on the preferences of the decision-maker. These preferences are applied to a set of useful indicators obtained from data about their reuse in OSS Projects of GitHub repositories. Concretely, these steps 2 are detailed in the following subsections and are summarized below:

(

A proposal of indicators of reuse based on GitHub

Smart Cities should follow a strategy for opening data as described in [17]. This strategy should prioritize publication of data which allows a community of developers to generate impact and effectively release benefits of open data through OSS projects [37]. A Smart City could in fact prioritize publication of open data with more reuse potential depending on the category to which the data belong to. However, due to "open-data by default" idiosyncrasy [23], data is usually published without establishing specific goals and without imposing utilization or authentication restrictions to the infomediaries and end users. As a result, collecting the usage information and measuring impact generated by open datasets may become very complex.

To overcome this situation, our approach is based on considering that the more used an open dataset is by OSS projects, the more impact is generated. Therefore, we borrowed some well-known indicators that measure the success of OSS projects and we have used as starting point to develop our indicators to measure such success when open data is reused. Then, these indicators allow Smart Cities to measure which categories of open data have more reuse potential and decide which data must be released according to the requirements of each city. The following indicators from existing research literature on OSS are considered [27] [28]. First of all, we included (i) number of people who agree to receive information about the project because they find it interesting (subscribers), and (ii) number of people who actually work on the OSS project (developers). On the one hand, subscribers to OSS choose to obtain information on the project and thus reveal a deeper interest in the OSS project. The subscriber indicator not only measures interest within the project but the reputation of the project within the community and the dissemination of the project through the community. On the other hand, the number of developers working on a project is critical to its success, since survival of an OSS project depends on continued contribution from developers [28]. There is another measure for the success of OSS projects [27] as the (iii) age of an active project that is positively related to OSS progress toward completion, as well as the experience of the community of developers.

Based on these three indicators described in the literature about success of OSS projects, we developed a set of three indicators that measure the success of open source projects that reuse open datasets (they are summarized in Table 1). The aim is to compare projects that use different categories of datasets and how successful they are. First of all, we define the reputation among a community of developers of OSS projects that reuse open data from a category. Some projects that reuse open data from some specific categories can be perceived by developers as being highly appealing projects. Smart Cities are interested in opening data that will be reused in these kinds of projects in view of creating a community around open data, thus allowing an open data portal to attract the attention of potential developers. Therefore, the reputation indicator measures how well-known projects reusing data from some specific category are (within the community of developers). Furthermore, the size of the community involved in projects that use data from a category is defined in terms of the size of the community of developers that use open data from a given category. A city needs to adapt the size of the community to the budget and available infrastructure. proposed. Maturity means that the community has been working on the project for some time without the project being abandoned. A Smart City may want to select the datasets that help in promoting fewer projects stretching over longer periods of time, rather than promoting a larger number of short-term projects.

Finally, maturity of projects that use an open data category is

An additional indicator has been developed in order to assess the impact of a dataset category, i.e. the likelihood of datasets from each category of being used. To do so, we defined efficiency of an open data category, as the probability of datasets of one category to be referenced by an OSS project. This indicator determines how relevant a category of datasets is. Smart Cities will use this indicator to know which categories of open data are most likely to be reused. Therefore, in a scenario where the Smart City has the chance of opening a large number of datasets, the efficiency indicator will become secondary to the publishing efforts regarding a wide a variety of datasets.

As aforementioned, these indicators come from well-known indicators from the OSS community, being thus completely generalizable to be used in any OSS repository. It is worth noting that our proposal of indicators is not set in stone, consequently more indicators could be created and checked to be used by Smart Cities according to their requirements.

Search of smart city datasets on Socrata

Once the impact measuring indicators have been established and defined, information should be gathered. This gathering of information focuses on datasets specifically related to the smart cities so as to obtain a more accurate assessment of the collected data.

Socrata is a software company focused "exclusively on democratizing access to public sector data around the world". It provides an Open Data Platform for allowing local, regional or national governments to release data. Socrata is a partner of the USA National League of Cities [22] for the development of open data strategies. Nowadays, the Socrata Open Data Platform is used by some of the most important US cities such as New York, Chicago, San Francisco or Los Angeles. In this respect, Socrata is very useful as a proof-of-concept of our approach, since it is possible to collect precisely open dataset identifiers and their metadata. In this sense, every Socrata dataset has its own endpoint and each is designated by a unique dataset identifier. Every Socrata open data portal provides a list of its published datasets Public transport timetables, access points broadband penetration containing the identifier of every dataset and useful metadata about it, such as the theme or the keyword of the dataset. These metadata of open datasets are important because they are needed to facilitate the categorization step that comes next. To collect the data from Socrata, we followed these steps:

(1) Retrieve data from Socrata on institutions which use its Open Data Platform. 106 institutions were recovered. (2) Gather and filter the identifier and the minimal metadata needed to categorize them (theme or keyword) from every dataset published by US cities using Socrata. 8960 datasets from 32 different US cities met these conditions.

Categorization fo datasets

In this step, we had to choose the taxonomy of dataset categories to be analyzed. There is no common agreement on the best way of classifying Smart City open datasets. However, a 14 high-value data categories is suggested by the G8 Open Data Charter [10]. These categories, together with example datasets for each one, are shown in Table 2. These categories seem to be a good way to classify Smart City datasets, however, some of these categories, such as Global Development and Science and Research, might not be used in the Smart City context. Thus, specific domains which can generate data within a Smart City must be taken into account. In this sense, [21] about Smart City initiatives proposes a classification divided in domains and subdomains show in Table 3 Establishing an exhaustive classification of open data categories for Smart Cities is beyond the scope of this paper. However, this work proposes an initial classification of open data categories for Smart Cities aimed to be as close as possible to the G8 Open Data Charter but incorporating modifications to encompass the aforementioned domains and subdomains proper to Smart Cities. This proposed classification is given in Table4 together with example datasets for each category.

Once the categories were established we had to classify the collected datasets according to such categories. Due to its characteristics, this step requires the participation of experts to execute it adequately. The research groups that have developed this approach includes researchers working in related fields such as open data and knowledge representation. These researchers were responsible for classifying the datasets following the steps described below:

(1) Extracting different themes from US city datasets. In our case, 215 different themes were extracted. (2) Mapping every theme to one of the available categories.

Themes without a clear fit had to be classified as 'Others' in order to be discarded later. When we performed this step, (3) Automatically classifying datasets with a theme according to the mapping in step 2. In our case, 8299 datasets were classified according to the established categories, 11 were categorized as 'Others' and 650 were not categorized due to their lack of theme. (4) Optionally, trying to categorize datasets that have no theme manually, using other metadata such as keywords. This step can be carried out when the number of datasets without a theme is considered high enough to distort the value of the indicators. In our case, although the datasets without a theme represented less than 10 (5) As a result of this process, 8949 datasets were adequately categorized and 11 were discarded due to their unclear fit.

Collecting data from GitHub to calculate indicators

In GitHub has been used for collecting data and calculating indicators related to OSS success in several works such as [3] [19], where GitHub allows researchers to collect several measures regarding open source projects, for example, forks, stars, etc. GitHub has an API that is used to collect all required data from an open source software project. More specifically, the data can be acquired from repositories and from users. A repository is a kind of software project folder that contains all the project files. Valuable data from a repository that can be collected by using the API, apart from the code itself, are as follows: repos-itory_id, user_id, stargazers_count, watchers_count, language, forks_count, subscribers_count, network_count, created_at, up-dated_at, pushed_at, total_contributors, total_contributions. GitHub user data also provide interesting data to be considered, such as followers_user, following_user, public_repos_user, location_user, updated_at_user, created_at_user. The indicators used in our approach are based on these data. We established a process for identifying which OSS projects were using open datasets from Socrata US Cities. Our process consists in the following steps (it was implemented by using the GitHub API within a Pentaho Data Integration [5] process):

(1) Searching every eight-character code from existing Socrata datasets belonging to USA cities (obtained as described in Section 3.3.1) based on code from OSS repositories hosted on GitHub in order to know which projects are reusing open data. When we performed this step, 350644 references were found from 2517 repositories to 5874 of the 8949 categorized datasets. (2) Gathering required data from GitHub on the repositories that reference open datasets to make an estimation of the indicators. In our case we found that 2501 of the 2517 repositories had all the needed data.

After this process, we made an estimation of the indicators in order to be used with AHP. We defined a process consisting in the following steps:

(1) Discarding repositories that do not have all the required data to make an estimation of the indicators. When we performed this step, only 2501 repositories remained. (2) Discarding all repeated references to a specific dataset from a specific repository. When we performed this step, 32551 unrepeated references from 2501 repositories remained.

(3) Making an estimation of the indicators. When we performed this step, we applied the formulas previously presented in Table 1. ( 4) Normalizing the indicators in order to use the ideal mode of AHP. When we applied this step to our case, the indicator of each category was divided by the maximal value obtained by a category in the indicator. Thus, all the indicators of each category were normalized to a 0-1 range.

Use of AHP to weight indicators

The method of decision-making, which our model is based on, is named Analytic Hierarchy Process, hereinafter referred to as AHP [25]. It is a powerful and flexible tool for decision-making in complex multi-criteria problem situations and is useful for comparing several alternatives when several objectives need to be borne in mind at the same time. Following this method, the evaluator can directly assign a normalized weight to a criterion that will indicate the importance which that criterion has with regard to the final objective. Firstly, the AHP method compares the relative importance that each criterion has in relation to all the others; this assessment enables the relative weights of the criteria to be calculated, and finally the method normalizes the weights in order to obtain the measures for the existing alternatives; for this reason, AHP constitutes one of the best options to assist multi-criteria decision making. This method allows people to gather knowledge about a particular problem, to quantify subjective opinions and to force the comparison of alternatives in relation to established criteria. The method consists in the following steps:

(1) Define the problem and the main objective in making the decision. (2) If required, build a hierarchy tree in this way: the root node is the objective of the problem, the intermediate levels are the criteria, and the lowest level contains the alternatives. (3) At each level, build a pairwise comparison matrix with the brothers (sons of the same node). The matrix contains the weights of pairwise comparisons between brother nodes. This provides us with a pairwise comparison matrix (see a simple example in Table 5) for each parent node. (4) For each comparison matrix, an eigenvector must be calculated, using the equation: |A − λI | = 0, where A is the comparison matrix, I is the identity matrix and λ is the eigenvector. This calculus must be performed for each level of the tree. (5) Rate each alternative (leaf nodes) with a previously calculated fixed value for every criteria. The scales for rating alternatives should be established and described in a precise way. ( 6) Determine the value of each alternative using a weighted addition formula, with the weights from the previous steps. These results ascend up the tree to calculate the final value of the objective (root). This final value is used to make a decision about the alternative to choose.

Using this method, as final stage, we have created a Google Spreadsheet based on AHP that uses the reuse indicators as criteria of the process. Concretely, this spreadsheet is composed of three sheets:

(1) 'Indicators'. This sheet provides the normalized indicators that were calculated from GitHub in the previous step. Thus, the use of this tool allows Smart Cities to prioritize datasets in a reasonable way based on the data collected from well-known cities, the indicators taken into account and the open data strategy of the city.

SIMULATING THE BEHAVIOUR OF THE TOOL ON STEREOTYPICAL CITIES

In order to check our proposal according to different motivations in the weighting process, we have simulated the behavior of the tool taking into account the different prospects of two stereotypical cities. We asked three experts to agree on the importance assignment of the indicators, with the assumptions of the two cities.

On one hand, a medium-sized town located in a rural region, with small software companies in its zone rather than big ones, that is starting to develop its own open data portal. On the other hand, a big city with a well-known open data portal and a lot of cutting edge software companies in its area of influence.

In the first case, we have guessed that the town could be interested, mainly, in getting reuses of its different datasets through the development of simple applications by small local enterprises. Hence, the town would assign high weights to efficiency whereas reputation, size of the community and maturity would perform a secondary role. In the second case, we have conjectured that, due to its portal is well-known, it does not search for more reuses, that is, efficiency, but for mature projects with good reputation and bigger communities behind them. The weights applied with this philosophy are shown in Figure 4.

The ranking obtained with these weights is shown in Figure 5 Here, 'Geospatial' changes to third position and 'Welfare' takes the first one. As can be seen, the indicators obtained from GitHub produces that some categories of the ranking tend to have a stable position regardless of the weights assigned with AHP but, even so, different combinations of weights may change this ranking.

RELATED WORK

This section gives a description of (i) some relevant studies about the use of GitHub to measure different indicators about Open Source Software projects, (ii) applications of AHP in Smart Cities as well as (iii) the most relevant studies about how (local) governments publish open data.

Firstly, GitHub is used by individuals, communities and businesses alike to develop software projects. GitHub is free to use for public and OSS projects, and it is profusely used in studies on Software Engineering related to OSS success in several works. Thus, Bissyande et al. uses GitHub [3] to study a possible relation between programming languages and projects success. Marlow et al. [19] analyze metadata projects of GitHub to find how its users decide whom and what to keep track of, or where to contribute next. Sheoran et al. [25] investigate what kind of contributors can be the "watchers" of GitHub. Jarczyk et al. [15] study the relation between popularity of a project in GitHub and its quality. Muthukumaran et al. [20] uses GitHub to propose change metrics that can predict possible bugs. As far as we know, this is the first time GitHub has been used to estimate indicators related to reuse of open data in OSS projects.

Secondly, AHP is a multiple criteria decision making method that has been used in many different applications related to decision making [31]. Some works specifically use AHP in Smart Cities and e-government. In this context, Bartolozzi et al. [2] present a DSS which uses AHP for supporting the decisionmaking process related to Smart City issues. Sultan et al. [29] suggest the use of AHP to decide the most appropriate technology for the development of e-government projects in Smart Cities. Boselli et al. [4] use AHP to rank the factors for innovating a smart-mobility service in the city of Milan. A very interesting use of AHP to evaluate open data portal quality can be found in Kubler et al. [18]. The authors propose considering different dimensions: completeness, openness, addressability and retrievability to assess the quality of 146 open data portals. Although there are several applications of AHP to the domains of Smart Cities and e-governments, they all aim at assessing Smart City strategies and the quality of open data portals. Instead, our approach proposes AHP to recommend the most appropriate datasets to be published.

Finally, with respect to how (local) governments publish open data, Conradie & Choenni [6] explain that data release by local governments is still a novel task, thus knowledge is lacking as to its benefits and barriers. Therefore, they conduct a participatory action research approach to get a better understanding of how internal processes of local governments influence data release. The authors found that the following indicators needed to be addressed by local governments to overcome barriers to releasing public sector information: (i) Data Storage, i.e., is data stored centrally, or is it decentralized?; (ii) Use of data, i.e., the way data is used by the department; (iii) Source of data, i.e., how is a set of data obtained?; and (iv) Suitability of data for release, i.e., are there rules and regulations that determine whether a dataset may be released or not, such as privacy or copyright.

Notwithstanding, these indicators are related to current data but do not address the actual use of the data and its benefits. For example, Hossain et al. [13] show that benefits associated with opening data are ill-understood. In their systematic review of open government data initiatives, Attard et al. [1] explore open data initiatives of a large number of governments, as well as existing tools and approaches. They found that while efforts have focused on developing tools for helping data publishers to open data, there have been no initiatives related to strategies for supporting decisions on which data to release. This means that public entities may end up publishing data with no value, rather than focusing on the relevance of the data they are publishing. Therefore, success in opening data is not a matter of the amount of data published, but of understanding how data is reused. As highlighted by Zuiderwijk & Janssen [34], since providers of open data are not concerned with needs of open data users, they do not know how their data are reused, and business related issues (such as creation of added-value services or products based on open data) are not widely used as a decision criterion. Furthermore, Zuiderwijk et al. [36] argue that the publication of open data is often cumbersome so standard procedures and processes for opening data are required. They found a series of barriers preventing easy and low-cost publication of open data, leading them to propose a set of five design principles for improving the open data publishing process of public organizations: (i) start thinking about the opening of data at the beginning of the process; (ii) develop guidelines, especially about privacy and policy sensitivity of data; (iii) provide decision support by integrating insights into the activities of other actors involved in the publishing process; (iv) make data publication an integral, well-defined and standardized part of daily procedures and routines; and (v) monitor how the published data are reused. Our approach is related to principle (iii) since we provide a decision support framework based on activities of data consumers. We also contribute to principle (v) since our approach is useful for monitoring how datasets are being reused in OSS applications. Additionally, Jetzek et al. [16] Furthermore, Zuiderwijk & Janssen [35] state that different types of users of open data are often interested in different types of data, therefore, publication of data can be improved by taking into account preferences for certain types of data for certain open data users.

Therefore, there are several methods that support opening data, but to the best of our knowledge no approaches focus on supporting Smart Cities in selecting and prioritizing which datasets should be open according to their preferences and the context of the city they work for. To fill this gap, we presented our approach based on obtaining useful indicators from Socrata and GitHub and use them with AHP.

CONCLUSIONS

Smart Cities usually have a limited budget and insufficient time to release and maintain all available open data. In this paper, we have presented an approach whose goal is to provide an AHP tool that allows weighting different indicators of reuse, calculated using Socrata and GitHub as sources of information, in order to combine them taking into account objective criteria. This approach is characterized by: This approach is completely functional and reproducible. We provide a public repository containing the data obtained from Socrata and GitHub, the scripts to collect and analyze the information and the AHP tool in order to users can use or modify these processes. So, Smart Cities or any other public institution can reuse and adapt them to their concrete requirements regardless of whether they work in a Smart City or in any other type of institution. In this sense, further alternative applications of our approach that can be considered as a continuation of this research may include: In summary, a successful publication of open datasets should be based on the proper combination of the objectives of the open data portal and the analysis of the impact of already available open datasets. This approach provides a useful method for Smart City decision makers to carry out this task in an objective and analytic way.