Publishing and maintaining open data is a costly task for public institutions, that becomes even more challenging in the context of Smart Cities, where large amounts of varied data are generated from diferent domains. To optimize resources, they should prioritize the publication and maintenance of datasets most likely to generate social and economic impact. However, there is currently a lack of decision-support tools to help public sector data publishers to evaluate datasets on the light of their particular reuse goals. In this paper, we propose to suggest to data publishers the dataset categories with most potential impact, based on the impact of already published datasets of the same category. To measure impact, we propose a set of indicators based on the amount and quality of Open Source Software projects that use datasets. To aggregate indicators according to specific reuse goals, we provide an Analytic-Hierarchy-Process based tool.
One of the most important challenges faced by Smart Cities is
creating an ecosystem of public and private actors that reuse open
data in order to produce IT services and products that both (i)
would improve citizens’ quality of life and (ii) would contribute
to economic growth [
To reverse this situation, publishing datasets as open data
requires a decision support system to select those categories of
datasets that ofer higher potential to generate value [
Unfortunately, to the best of our knowledge, Smart Cities
lack such decision support system, mainly because the process
of calculation of those indicators that would use the system is
not a trivial task. According to Janssen et al. [
More reasonable indicators of the use of open datasets could
help to identify which categories of datasets have more
possibilities of being reused and, in this way, generate some type of
economic impact to people or enterprises. In this sense, good
indicators could come from the reuse of datasets within the open
source community. The Tenth Annual Future of Open Source
Survey [
Based on these figures, an estimation of the use of the diferent categories of datasets by the OSS community could be a good indicator of their potential impact. Therefore, when Smart Cities make decisions on which data to publish, they could prioritize publication of data which allows a community of developers to generate impact and efectively release benefits of open data through OSS projects.
In this paper, we present an approach based on the estimation
of indicators of the use of open datasets in OSS projects. The
goal of this approach is to provide Smart Cities with a Decision
Support System which provides an ordered list of categories of
datasets most suitable to be published or maintained in their open
data portal. To do so, we have carried out a set of actions aimed
at estimating useful impact indicators related to the datasets of
the same category already published by open data portals of
other cities. Concretely, to calculate our proposed indicators we
needed two kinds of data sources: (i) already published Smart City
datasets (and their metadata) and (ii) OSS projects (together with
information about them) which referenced the gathered datasets;
i.e., we needed to know which open datasets were being used in
which OSS projects. To collect already published open datasets,
we chose Socrata [
Using the indicators obtained from these sources, we provide
an Analytic Hierarchy Process (hereon AHP)-based [
This paper is structured as follows: section 2 describes a new approach to select the most relevant categories of data to be published in a smart city open data portal. Section 3 presents toy samples of two diferent stereotypical smart cities using our approach and, to finish, section 4 summarizes other work related to the publishing of open data in Smart Cities. 2
This section describes the steps that have been carried out to get an AHP process that allows classifying categories of dataset based on the preferences of the decision-maker. These preferences are applied to a set of useful indicators obtained from data about their reuse in OSS Projects of GitHub repositories. Concretely, these steps2 are detailed in the following subsections and are summarized below: (1) From GitHub repositories, studying the characteristics of OSS projects that use open datasets. This information was analyzed to establish a set of reuse indicators. (2) Gathering datasets from 32 cities of the United States (such as San Francisco, Chicago or New York) which use Socrata as an open data repository. With respect to this point, it should also be noted that, although these cities are from the same country, United States, they have diferent cultural, social and economic characteristics that make us consider that the results obtained from their data are enough scalable to other Smart Cities located in diferent countries. (3) Classifying the datasets according to a set of categories specifically designed for Smart Cities. (4) Searching for references to the datasets obtained from
Socrata in GitHub to calculate the indicators. (5) With the reuse indicators established in step 1 as criteria, and the values from step 4, we have created a Google Spreadsheet [w3] based on AHP that allows decision makers to prioritize the most relevant categories of datasets that must be published in a smart city open data portal. 1https://goo.gl/HcUc1e 2A repository containing all the scripts and detailed instructions needed to carry out a functional application of our approach is available at GitHub https://goo.gl/TDp1xi 2.1
Smart Cities should follow a strategy for opening data as
described in [
To overcome this situation, our approach is based on
considering that the more used an open dataset is by OSS projects,
the more impact is generated. Therefore, we borrowed some
well-known indicators that measure the success of OSS projects
and we have used as starting point to develop our indicators
to measure such success when open data is reused. Then, these
indicators allow Smart Cities to measure which categories of
open data have more reuse potential and decide which data must
be released according to the requirements of each city. The
following indicators from existing research literature on OSS are
considered [
Based on these three indicators described in the literature about success of OSS projects, we developed a set of three indicators that measure the success of open source projects that reuse open datasets (they are summarized in Table 1). The aim is to compare projects that use diferent categories of datasets and how successful they are. First of all, we define the reputation among a community of developers of OSS projects that reuse open data from a category. Some projects that reuse open data from some specific categories can be perceived by developers as being highly appealing projects. Smart Cities are interested in opening data that will be reused in these kinds of projects in view of creating a community around open data, thus allowing an open data portal to attract the attention of potential developers. Therefore, the reputation indicator measures how well-known projects reusing data from some specific category are (within the community of developers). Furthermore, the size of the community involved in projects that use data from a category is defined in terms of the size of the community of developers that use open data from a given category. A city needs to adapt the size of the community to the budget and available infrastructure. Finally, maturity of projects that use an open data category is proposed. Maturity means that the community has been working on the project for some time without the project being abandoned. A Smart City may want to select the datasets that help in promoting fewer projects stretching over longer periods of time, rather than promoting a larger number of short-term projects.
An additional indicator has been developed in order to assess the impact of a dataset category, i.e. the likelihood of datasets from each category of being used. To do so, we defined eficiency of an open data category, as the probability of datasets of one category to be referenced by an OSS project. This indicator determines how relevant a category of datasets is. Smart Cities will use this indicator to know which categories of open data are most likely to be reused. Therefore, in a scenario where the Smart City has the chance of opening a large number of datasets, the eficiency indicator will become secondary to the publishing eforts regarding a wide a variety of datasets.
As aforementioned, these indicators come from well-known indicators from the OSS community, being thus completely generalizable to be used in any OSS repository. It is worth noting that our proposal of indicators is not set in stone, consequently more indicators could be created and checked to be used by Smart Cities according to their requirements. 2.2
Once the impact measuring indicators have been established and defined, information should be gathered. This gathering of information focuses on datasets specifically related to the smart cities so as to obtain a more accurate assessment of the collected data.
Socrata is a software company focused “exclusively on
democratizing access to public sector data around the world”. It
provides an Open Data Platform for allowing local, regional or
national governments to release data. Socrata is a partner of the
USA National League of Cities [
These categories seem to be a good way to classify Smart
City datasets, however, some of these categories, such as Global
Development and Science and Research, might not be used in the
Smart City context. Thus, specific domains which can generate
data within a Smart City must be taken into account. In this sense,
Id
A
B
C
D
E
F
10.-Facility management
11.-Building services
12.-Housing quality
13.-Entertainment
14.-Hospitality
15.-Pollution control
16.-Public safety
17.-Healthcare
18.-Welfare and social inclusion
19.-Culture
20.-Public spaces management
21.-E-government
22.-E-democracy
23.-Procurement
24.-Transparency
25.-Innovation and
entrepreneurship
26.-Cultural heritage
management
27.-Digital Education
28.-Human capital management
a survey [
Establishing an exhaustive classification of open data categories for Smart Cities is beyond the scope of this paper. However, this work proposes an initial classification of open data categories for Smart Cities aimed to be as close as possible to the G8 Open Data Charter but incorporating modifications to encompass the aforementioned domains and subdomains proper to Smart Cities. This proposed classification is given in Table4 together with example datasets for each category.
Once the categories were established we had to classify the collected datasets according to such categories. Due to its characteristics, this step requires the participation of experts to execute it adequately. The research groups that have developed this approach includes researchers working in related fields such as open data and knowledge representation. These researchers were responsible for classifying the datasets following the steps described below: (1) Extracting diferent themes from US city datasets. In our case, 215 diferent themes were extracted. (2) Mapping every theme to one of the available categories.
Themes without a clear fit had to be classified as ‘Others’ in order to be discarded later. When we performed this step, Id 1 2 3 4 5 6 7 8 9 10 11 12 13 14 211 themes could be mapped to the established categories and 4 were classified as ‘Others’. (3) Automatically classifying datasets with a theme according to the mapping in step 2. In our case, 8299 datasets were classified according to the established categories, 11 were categorized as ‘Others’ and 650 were not categorized due to their lack of theme. (4) Optionally, trying to categorize datasets that have no theme manually, using other metadata such as keywords. This step can be carried out when the number of datasets without a theme is considered high enough to distort the value of the indicators. In our case, although the datasets without a theme represented less than 10 (5) As a result of this process, 8949 datasets were adequately categorized and 11 were discarded due to their unclear fit. 2.4
In order to calculate the above-described indicators on the success of OSS projects that reuse open data, we decided to collect data from GitHub. GitHub, as mentioned previously, is a platform for collaborative development of software based on a Git repository. It is used by individuals, communities and businesses alike to develop software projects. GitHub is free to use for public and open source projects, and it is profusely used in studies on Software Engineering. Therefore, it ofers useful data about open source software projects, including information on whether they are using open data.
GitHub has been used for collecting data and calculating
indicators related to OSS success in several works such as [
After this process, we made an estimation of the indicators in order to be used with AHP. We defined a process consisting in the following steps: (1) Discarding repositories that do not have all the required data to make an estimation of the indicators. When we performed this step, only 2501 repositories remained. (2) Discarding all repeated references to a specific dataset from a specific repository. When we performed this step, 32551 unrepeated references from 2501 repositories remained. (3) Making an estimation of the indicators. When we performed this step, we applied the formulas previously presented in Table 1. (4) Normalizing the indicators in order to use the ideal mode of AHP. When we applied this step to our case, the indicator of each category was divided by the maximal value obtained by a category in the indicator. Thus, all the indicators of each category were normalized to a 0-1 range. 2.5
The method of decision-making, which our model is based on,
is named Analytic Hierarchy Process, hereinafter referred to as
AHP [
Following this method, the evaluator can directly assign a normalized weight to a criterion that will indicate the importance which that criterion has with regard to the final objective. Firstly, the AHP method compares the relative importance that each criterion has in relation to all the others; this assessment enables the relative weights of the criteria to be calculated, and finally the method normalizes the weights in order to obtain the measures for the existing alternatives; for this reason, AHP constitutes one of the best options to assist multi-criteria decision making. This method allows people to gather knowledge about a particular problem, to quantify subjective opinions and to force the comparison of alternatives in relation to established criteria. The method consists in the following steps: (1) Define the problem and the main objective in making the decision. (2) If required, build a hierarchy tree in this way: the root node is the objective of the problem, the intermediate levels are the criteria, and the lowest level contains the alternatives. (3) At each level, build a pairwise comparison matrix with the brothers (sons of the same node). The matrix contains the weights of pairwise comparisons between brother nodes. This provides us with a pairwise comparison matrix (see a simple example in Table 5) for each parent node. (4) For each comparison matrix, an eigenvector must be calculated, using the equation: |A − λI | = 0, where A is the comparison matrix, I is the identity matrix and λ is the eigenvector. This calculus must be performed for each level of the tree. (5) Rate each alternative (leaf nodes) with a previously calculated fixed value for every criteria. The scales for rating alternatives should be established and described in a precise way. (6) Determine the value of each alternative using a weighted addition formula, with the weights from the previous steps. These results ascend up the tree to calculate the final value of the objective (root). This final value is used to make a decision about the alternative to choose.
Using this method, as final stage, we have created a Google Spreadsheet based on AHP that uses the reuse indicators as criteria of the process. Concretely, this spreadsheet is composed of three sheets: (1) ‘Indicators’. This sheet provides the normalized indicators that were calculated from GitHub in the previous step. (2) ‘AHP Criterion Pair Comparison’. This sheet allows assessing the relative importance between pairs of indicators using AHP. Thereby, a decision maker could weigh the importance of the indicators set out in the previous steps, taking into account the characteristics and objectives of the city. These weights can be assigned according to the institution’s strategic reuse objectives. Thus, diferent Smart Cities may have diferent objectives, strategies and target audiences when deciding which datasets should have priority of publication. Each city has its own idiosyncrasy defining what is most important or of particular interest, and it is unlikely two cities share the same priorities with regard to their respective reuse objectives. Cities can be characterized by their size, the importance of the tourism sector, or its residential, commercial or industrial sectors, etc. And also, cities may have diferent priorities for publishing datasets depending on the type of reuse they want to promote. The result of this step will be the eigenvectors of each matrix, meaning the relative importance of the established indicators. (3) Finally, the ‘AHP Direct Results’ shows a suitability ranking list of dataset categories to publish according to the weights introduced in the second sheet and the indicators calculated from GitHub shown in the first sheet. That is, the value used to elaborate such ranking is the result of multiplying the relative importance of each indicator, calculated in the second sheet, by the values of the indicators in the corresponding categories shown in the first sheet. Thus, the use of this tool allows Smart Cities to prioritize datasets in a reasonable way based on the data collected from well-known cities, the indicators taken into account and the open data strategy of the city. 3
In order to check our proposal according to diferent motivations in the weighting process, we have simulated the behavior of the tool taking into account the diferent prospects of two stereotypical cities. We asked three experts to agree on the importance assignment of the indicators, with the assumptions of the two cities.
On one hand, a medium-sized town located in a rural region, with small software companies in its zone rather than big ones, that is starting to develop its own open data portal. On the other hand, a big city with a well-known open data portal and a lot of cutting edge software companies in its area of influence.
In the first case, we have guessed that the town could be interested, mainly, in getting reuses of its diferent datasets through the development of simple applications by small local enterprises. Hence, the town would assign high weights to eficiency whereas reputation, size of the community and maturity would perform a secondary role.
The weights applied with this philosophy are shown in Figure 1, and the resulting in the ranking shown in Figure 2. The ifrst position of ‘Geospatial’ does not change with respect to the default ranking (same weights for all the indicators) shown in Figure 3 but the rest of the ranking sufers some variations.
In the second case, we have conjectured that, due to its portal is well-known, it does not search for more reuses, that is, eficiency, but for mature projects with good reputation and bigger communities behind them. The weights applied with this philosophy are shown in Figure 4.
The ranking obtained with these weights is shown in Figure 5 Here, ‘Geospatial’ changes to third position and ‘Welfare’ takes the first one. As can be seen, the indicators obtained from GitHub produces that some categories of the ranking tend to have a stable position regardless of the weights assigned with AHP but, even so, diferent combinations of weights may change this ranking. 4
This section gives a description of (i) some relevant studies about the use of GitHub to measure diferent indicators about Open Source Software projects, (ii) applications of AHP in Smart Cities as well as (iii) the most relevant studies about how (local) governments publish open data.
Firstly, GitHub is used by individuals, communities and
businesses alike to develop software projects. GitHub is free to use
for public and OSS projects, and it is profusely used in studies
on Software Engineering related to OSS success in several works.
Thus, Bissyande et al. uses GitHub [
Secondly, AHP is a multiple criteria decision making method
that has been used in many diferent applications related to
decision making [
Finally, with respect to how (local) governments publish open
data, Conradie & Choenni [
Notwithstanding, these indicators are related to current data
but do not address the actual use of the data and its benefits.
For example, Hossain et al. [
Furthermore, Zuiderwijk et al. [
Furthermore, Zuiderwijk & Janssen [
Therefore, there are several methods that support opening data, but to the best of our knowledge no approaches focus on supporting Smart Cities in selecting and prioritizing which datasets should be open according to their preferences and the context of the city they work for. To fill this gap, we presented our approach based on obtaining useful indicators from Socrata and GitHub and use them with AHP. 5
Smart Cities usually have a limited budget and insuficient time to release and maintain all available open data. In this paper, we have presented an approach whose goal is to provide an AHP tool that allows weighting diferent indicators of reuse, calculated using Socrata and GitHub as sources of information, in order to combine them taking into account objective criteria. This approach is characterized by: (1) A classification of 14 categories for Smart City open datasets based on the G8 Open Data Charter and the Smart City domain. (2) A definition of 4 indicators based on the reuse of datasets in OSS projects. (3) Almost 9000 open located datasets of many of the most important US cities. (4) A catalogue of these US city datasets classified according to the proposed categories. (5) Around 32000 distinct references from 2500 diferent GitHub projects referencing two thirds of the categorized datasets found, based on a search performed over all OSS projects in GitHub. (6) An estimation of the defined indicators of reuse of every
Smart City dataset category. (7) An AHP-based Decision Support System to recommend Smart City dataset categories to prioritize, taking into account the estimated indicators and the importance of each indicator for the cities.
This approach is completely functional and reproducible. We provide a public repository containing the data obtained from Socrata and GitHub, the scripts to collect and analyze the information and the AHP tool in order to users can use or modify these processes. So, Smart Cities or any other public institution can reuse and adapt them to their concrete requirements regardless of whether they work in a Smart City or in any other type of institution. In this sense, further alternative applications of our approach that can be considered as a continuation of this research may include: (1) Searching and categorizing open datasets of diferent cities, regions, countries, companies or any other kind of institutions in order to get more data. (2) Developing semantic-based software tools for automatic classification of datasets. (3) Analyzing the reuse of open datasets in proprietary software projects, for instance, by developing an app web repository where developers could register their applications that use open data and indicating which particular datasets are reused. (4) Analyzing the impact of open datasets in mass media, social media, blogs, etc. by searching the references to the datasets in these sites. (5) A set of controlled experiments to demonstrate the efectiveness of our approach in diferent scenarios.
In summary, a successful publication of open datasets should be based on the proper combination of the objectives of the open data portal and the analysis of the impact of already available open datasets. This approach provides a useful method for Smart City decision makers to carry out this task in an objective and analytic way. 6
We would like to thank GitHub that allowed us to use its API without limitations and Socrata that provides a way to collect precisely all the datasets published using its tools. This work has been developed with the support of (i) TIN2015-69957-R and TIN2016-78103-C2-2-R (MINECO/ERDF, EU) project, (ii) POCTEP 4IE project (0045-4IE-4-P), (iii) Consejería de Economía e Infraestructuras/Junta de Extremadura (Spain) - European Regional Development Fund (ERDF)- GR15098 project and IB16055 project, and (iv) Consejería de Educación y Empleo/Junta de Extremadura (Spain) - Becas de Movilidad al Personal Docente e Investigador Curso 2016/2017.