The concentration of people, companies, research organizations and other activities in urban areas is a key process in the development of economies and societies. In order to investigate how these urban systems function, the OECD (Organization for Economic Co-operation and Development) in collaboration with EC (European Commission) and Eurostat have introduced the concept of Functional Urban Areas (FUAs). FUAs consider a preliminary set of socio-economic and environmental factors and provide a basis for an agreed definition for measuring development of metropolitan areas. However, because FUAs are predefined they do not meet the need for designing policies and research questions involving different types of urban areas that are defined by weighting some factors more than others or by using additional factors. Therefore, providing an adaptive approach for dynamic and multi-faceted delineation of FUAs, rather than merely relying on a rigid schema with a fixed list of FUAs per country, allows to more flexibly reflect the socio-economic geography of where people live and work. This adaptive definition of FUAs demands integration of data from multiple up-to-date linked data sources. In this paper, we describe an approach and implementation for a Linked Open Geo-Data space, which combines openly available spatial and non-spatial resources on the Web to classify urban areas with the aim to more flexibly monitor and research urban development.
The concentration of people, companies, research organizations and other
activities in urban areas is a key process in the development of economies and
societies. How urban systems function is crucial to future economic prosperity
and better quality of life for more than three billion people[
Tackling the challenges of data integration on a dynamic environment such as WWW has been the mission of Linked (Open) Data technologies since the introduction of Semantic Web in 2001. Providing a Linked Open Data space, which brings together structured and interlinked geospatial data on the Web, facilitates delineating of FUAs. Currently, access to the micro data used by OECD for calculating different indicators of FUAs is limited, and negotiation with OECD is required to retrieve detailed data for regeneration of OECD FUAs.3 On the other hand, openly available geospatial datasets on the Web such as OpenStreetMap4 are already interlinked with existing structured data published on the Linked Open Data (LOD) cloud and provide the opportunity to reproduce a more flexible and dynamic list of FUAs. For example, the public sector in Europe creates lots of statistical data on different levels of administrative boundaries such as NUTS (Nomenclature of Units for Territorial Statistics), LAU (Local Administrative Unit), HASC (Hierarchical Administrative Subdivision Codes) and ISO 3166 country codes which could be utilized to dynamically identify FUAs adapted to the context of the corresponding policy studies.
In this paper we propose a Linked Data approach and implementation which
combines openly available spatial and non-spatial resources on the Web to more
flexibly classify urban areas. To achieve this goal, we followed methodology
described in the LOD Lifecycle [
2
As first step (step ’search/browse/exploration ’ in LOD Lifecycle [
Database of Global Administrative Areas (GADM). GADM12 provides a curated database of the administrative areas in the world. GADM provides some properties of these administrative areas such as name, variant names and “spatial features” about the location of the areas. Administrative areas in this database 5 http://datahub.io 6 http://lotus.lodlaundromat.org 7 https://open-data.europa.eu/en/data/ 8 for some specific countries level 11 is also defined. Check http://wiki.openstreetmap.org/ wiki/Template:Admin_level_11 9 http://nominatim.openstreetmap.org/ 10 http://overpass-api.de/api/ 11 http://wiki.openstreetmap.org/wiki/Planet.osm 12 http://www.gadm.org
include up to 6 levels of details starting from level 0 which refers to countries. Level 1 to 5 cover lower level subdivisions such as provinces, departments, counties, etc. depending on the size and availability of data for the underlying country. The GADM data are publicly available for download by country or the whole world in different formats such as shapefile, ESRI geodatabase, RData, and Google Earth kmz format.
Flickr Shapefiles Dataset. Flickr Shapefiles Public Dataset13 provides data from 190M geo-tagged photos on Flickr. The shapefiles are generated by plotting all the geotagged photos associated with a particular place and by generating a mostly accurate contour of that place. Flickr offers 6 levels of boundaries identified by so called Where On Earth (WOE) IDs. The levels range from country (level 1), region (level 2) county (level 3), locality (level 4) to neighborhood (level 5). The dataset is publicly available for download in GeoJSON format. Published Shapefiles for Individual Countries. In addition to crowdsourced and curated datasets on global administrative boundaries, local administrative offices or geo-related research centres in specific countries provide shapefiles and other properties related to the administrative units in that country. For example, Centraal Bureau voor de Statistiek (CBS) or Bundesamt für Kartographie und Geodäsie (BKG) provide shapefiles of administrative boundaries for the Netherlands and Germany respectively.
Published Geospatial RDF Datasets. There are already several efforts to
implement a spatial dimension on the Web of Data (a.k.a. Semantic Web).
GeoKnow[
The existing diverse landscape of standards for spatial data on the Web makes
the task of data extration and conversion very cumbersome and time-consuming.
The left side of Figure 1 depicts our implemented approach to deal with spatial
data extraction, processing and conversion. This corresponds to step ’extraction’
in the LOD Lifecycle [
PBF osmosis OSM XML
osmtogeojson GeoJSON
Mapping Configurations
ESRI shapes mapshaper triplify Enrichment Functions
FUAs (G28A8,D66M7) 71,724 (S2Fh7l6aic,p9k7er5s) 76,366 (344,269) 58,561
OpenStreetMap Administrative Boundaries 162,059 25,440
DBpedia WikiData Statistical Data on the Web the conversion to RDF. The Flickr Shapefiles Dataset was already available in GeoJSON format. Additional processing and conversion to GeoJSON format was needed for some of the other collected data. We downloaded the OSM dataset in PBF (Protocolbuffer Binary Format) which provides a more compressed format comparing to the XML format. We then used the Osmosis18 tool to process the data and to only extract the data about administrative boundaries in OSM format. The OSMtoGeoJSON19 tool was then applied on the extracted subset which resulted in GeoJSON version of data. We used MapShaper20 to convert the OECD shapefiles and GADM dataset from ESRI format to GeoJSON format.
We utilized a set of Mapping Configurations and Enrichment Functions to convert spatial data encoded in GeoJSON to RDF format. The Mapping configurations provided a mapping between the given properties of data in original dataset and their best matching RDF properties expressed in the Linked Open Data cloud. Linked Open Vocabularies21 were used to produce suggestions from existing vocabularies on the Web. In case no existing RDF properties are available, a new propriatery RDF property is created and defined as part of our proposed vocabulary.
Enrichment functions were defined to clean up, standardize, and enrich the property values. For example, we added the ISO 3166 code of the countries by processing the given country names and converted the given Wikipedia URLs to their corresponding DBpedia URIs. We also set the right data types for the converted literal values. The convertor scripts are available as separate repositories 18 http://wiki.openstreetmap.org/wiki/Osmosis 19 https://github.com/tyrasd/osmtogeojson 20 https://github.com/mbloch/mapshaper 21 http://lov.okfn.org/
on Github22. In addition to the spatial data, we extracted tabular data about the OECD list of municipalities and FUAs23, as well as metadata on different OSM levels provided as HTML tables on Wikipedia24, and converted this data to RDF. The final RDF dataset consisted of 344,269 administrative boundaries from OSM, 288,668 from GADM and, 276,975 from Flickr. 4
We used Openlink Virtuoso triple store for storing the generated RDF data.
The main reason for using Virtuoso was its extensive support for geometry
data types and spatial indexing25. At the time of conversion to RDF, we
adapted all the GeoJSON shapes coordinates to WKT (Well-Known Text)
Polygon and MultiPolygon representations. Virtuoso’s stored procedures such
as st_intersects, st_contains and st_within were then used to test whether
two geometries overlap in different ways. For example to find all the
adminstrative boundaries which contain a certain point. This step corresponds to step
’Storage/Querying’ in the LOD Lifecycle [
In order to exploit the power of Linked Data, we established links between the
converted RDF datasets and other open datasets available on Linked Open Data
cloud. This corresponds to step ’Interlinking/Fusing ’ in the LOD Lifecycle [
In order to compute direct links between OSM, GADM and Flickr, we followed a hybrid approach combining string similarity with the geometric overlapping of administrative boundaries. We first created a mapping between different levels of boundaries provided in OSM, GADM and Flickr by comparing the granularity of divisions in different countries. We took into account the provided OSM metadata per country for each administrative boundary level. Figure 2 shows a sample of extracted administrative boundaries for the Netherlands which reflects the possible mappings at different levels for a specific address (top-right of Figure 2). Secondly, we checked the overlaps of areas at the similar level, and for the matching areas we applied string matching to make sure that they refer to the same administrative boundary. Code 1.1 brings an example of CONSTRUCT queries used to create linksets between the OSM and GADM datasets. To showcase the output of our query, for Amsterdam in the Netherlands, the approach will result in the following linked entities: oecd:NL002, gadm:158-9-266, hasc:NL-NH-AD, osm:relation_47811, flickr:727232, dbpedia:Amsterdam, wikidata:Q9899 and geonames:2759794. 22 https://github.com/ali1k/osm-rdf https://github.com/ali1k/gadm-rdf https://github.com/ali1k/flickrshapes-rdf 23 http://www.oecd.org/gov/regional-policy/List-municipalities.xls 24 http://wiki.openstreetmap.org/wiki/Template:Admin_level_10 25 http://docs.openlinksw.com/virtuoso/sqlrefgeospatial.html
In addition to a SPARQL endpoint26 provided for Semantic Web users, we also
exposed a set of predefined SPARQL query templates as RESTful Web services
to facilitate use of the interlinked data by developers who are unfamiliar with the
SPARQL query language. The Web services also allow for better management
of data access (in case authentication and authorization are needed) whilst
monitoring the data usage to optimize the queries and to provide load balancing
on the services infrastructure (due to reasons of data size and performance of
the respective geospatial queries, scalability of Linked Geo Data platforms is
a critical issue [
Code 1.1. An example of making links between OSM and GADM datasets.
Invoking the services will result in executing the SPARQL query templates filled in with the given input.
An important benefit of exposing data as service is the ability to combine one or
more services with other existing services and applications to build novel and
innovative applications. With regards to our domains of interest, we created several
applications to better demonstrate the value of the provided services29. Utilizing
GoogleMap and Mapbox APIs to explore a dataset based on the extracted
boundaries was one example of these applications. For instance, Figure 3 shows
our geo-boundaries faceted browser[
Another practical application we built for batch processing of addressess was a Google spreadsheet add-on, depicted in Figure 4, which chains Google Geocoding API with our PointToAdmin and AdminToFUA services (see Section 6). Given addresses in a spreadsheet are enriched with different levels of administrative boundaries and FUAs. The users are then able to export the extracted boundaries and process them in geodata analysis tools such as CartoDB 30.
We applied the provided services and applications of linked geo-boundaries to several use cases within the context of the RISIS project. RISIS EU project31 29 http://sms.risis.eu/demos 30 https://carto.com/ 31 http://risis.eu
aims to build a distributed infrastructure on data relevant for research and
innovation dynamics and policies[
We conducted a concrete case study in this direction which is both exploratory, to gain insight into how a researcher in the science & technology domain (a co-author of this paper) can create an S&T map, and descriptive in nature, to
illustrate what results (delineation in an S&T map based on different attributes) are achieved and how these results can be intepreted. We investigated the effect of socio-economic and structural properties of the urban areas on innovative activities, as stimulated by recent RTD32 policies in the Netherlands. This policy is oriented at the ’top sectors’ of the economy, which were selected in a consultation of policy makers, representatives of the research system and entrepreneurs in the country. After selecting these ’top sectors’, a large part of public research funding was devoted to this new policy. Consortia can apply for funding, and they should exist of companies and research organizations (such as universities) with a company as main applicant. Because of this context, the funded projects can be considered as a useful representation of RTD collaboration for innovation.
In this use case we were interested in the geographical properties of these collaboration networks. In order to investigate this we needed data about the projects, and statistical data about the characteristics of the geographical units. These data are openly available on The Dutch data portal33. In this case, we employed the following open datasets: – RVO dataset 34 provides a list of R&D projects that have received subsidies and financial support from the Netherlands Enterprise Agency35. Projects information includes companies and research institues which are collaborating on the project together with the geographical coordinates of the projects. 32 Research and Technology Development 33 https://data.overheid.nl 34 http://www.rvo.nl/open-data-van-rvonl 35 Rijksdienst voor Ondernemend Nederland: RVO.nl
People
Businesses
Hybrid
OECD FUAs – CBS dataset 36 published by the statistics office of the Netherlands37 provides different types of statistical information on dimensions such as labour, income, economy, society and regional aspects of regions in the Netherlands.
As we did not know ex ante what the level of geographical organization of the consortia was, we needed to define these in different granularities. This enabled us to find out at what geo-level the consortia were organized. We could then identify the characteristics of these geographical ’containers’ of the projects. To realize that, we first calculated different sets of Urban Areas based on different statistics provided by the CBS dataset and different levels of open administrative boundaries. Figure 5 shows the delineation of these Urban Areas through population, business establishment, and combinations of these two indicators in the municipality level. Boundaries typically differ when defined by different characteristics. When compared to the OECD FUAs38 (right map 36 https://www.cbs.nl/en-gb/our-services/open-data 37 Het Centraal Bureau voor de Statistiek: CBS.nl 38 We used the shapefile from the Eurostat used in the large study named urban audit: http://ec.europa.eu/eurostat/web/cities/data/database
in Figure 5), the adaptive Urban Areas take into account additional regions (administrative boundaries) and enable the user to put different weights for the delineated boundaries which could be used for focused analysis of specific factors.
Our open data linking (step 4 in Section 5) allowed us to then map geographical coordinates of RVO projects to these FUAs (as a baseline for analysis of different S&T indicators) using SPARQL queries, to analyse the correlation of projects to the designated socio-economic factors. Figure 6 shows the result of the mapping where frequency of the projects on different factors are highlighted: the darker the color, the higher the number of awarded projects. As can be seen when comparing Figure 6 and Figure 7, by far not all (Functional) Urban Areas have projects. But more importantly, the different ways the Urban Areas are defined leads to different outcomes. Using the OECD FUAs (right map), or the population density based FUA (left map) would miss some of the relevant areas39. 9
With more than half the world’s population now living in urban areas, defining a metropolitan area is critical to reflect the reality of where people live and work as well as the connections between surrounding cities, educational institutes, and businesses. The issue of comparability of metropolitan areas needs an indepth, dynamic and multi-faceted analysis of administrative boundaries bringing together data about all the influencing factors. The OECD has already defined functional urban areas to address factors beyond the predefined city boundaries, and to better reflect the economic geography of where people live and work.
In this work we report an approach, implementation, and case study on the use of Semantic Web and Linked Data technologies to establish an open data space to more flexibly delineate FUAs by intergrating spatial and non-spatial data from the openly available data sources on the Web. We describe how our approach allows dynamic recreation of FUAs using linked open data and we illustrate our implementation in case studies involving researchers in the science & technology domain. In addition to better geographical coverage, our integration of Flickr, OpenStreetMap, and GADM open boundaries enables researchers and government policy makers to have different views on urban areas40. GADM as a curated dataset focuses mainly on formal administrative boundaries while OpenStreetMap and Flickr boundaries as social crowdsourced datsets provide more details and flexibility in defining boundaries.
As future work, we envisage to 1) analyze the quality of data by applying the services to several real-world scenarios defined in the RISIS41 project; 2) create more connections to the relevant datasets on Linked Open Data cloud; 3) design intuitive user interfaces for end-users to explore FUAs while combining several indicators. 39 in this case South-West Friesland is missing in population based indiators because it is less populated but still hosts a large set of businesses. 40 We write about urban areas because we reconstructed FUAs, however, our approach and implementation supports delineation of any functional area. 41 http://risis.eu