In the present effort, we present a knowledge tier platform to collect information from cities in a form of graphs. This platform enables people to share the information of the area where they live allowing them to inform about pollution, crime levels, traffic jams, streets topology, commerces, markets, etc. The main objective is to provide information, stored in Elastic about a city to find spatio-temporal patterns using Graph Mining techniques based on Apache Spark GraphX.
In the last years, we have seen the explosion of
data from on-line activity, user content generated,
health, scientific computing, mobile phones
activity, etc. This data increments due to the daily
transaction of people in urban centers and still
grows. By 2030, 60% of the worldwide
population will live in cities appearing 27 megacities
greater than 10 million inhabitants
In the present effort, we present a knowledge tire platform to collect information on cities in a form of graphs. This platform enables people to share the knowledge of the area where they live allowing them to inform about pollution, crime levels, traffic jams, streets topology, commerces, markets, etc. The primary objective is to provide information about the city to find spatio-temporal patterns using Graph Mining techniques.
The present paper is organized as follows. Section 2 introduce some basic concepts, while Section 3 describes the platform architecture. Sections 4 and 5 show some preliminary results and present the discussion about the platform. Finally, Section 6 concludes the paper and presents future works. 2
In the current section, we introduce some basic concepts, such as graph, knowledge tiers and Spark for describing the platform. 2.1
A graph is a mathematical structure composed of vertices, nodes or points, which are connected through edges, lines or arcs as depicted in Figure 1. A graph (G = (V, E)) is composed of a set of V vertices and E edges. in our context this structure allows us to represent street intersections as geo-referenced nodes and roads as edges. 2.2
The Haversine distance
dlon dlat a c = d = = lon2 lon1 = lat2 lat1 = (sin( dl2at ))2 + cos(lat1)⇥ cos(lat2) ⇥ (sin( dlon ))2
2 2 ⇥ atan2(p a, p 1 a) R ⇥ c (1) Since we are able to model street network of a city in the form of a graph. Note that each node and edge could have a weight representing different phenomena of a city, such as: (1) congestion, (2) crime, (3) pollution, (4) population density, (5) urban transportation, (6) subway network, etc. Thus, for each phenomenon, we have a graph modeling this particular fact. Finally, we can stack each node as depicted in Figure 2 to have a knowledge stack. Apache Spark is an open source cluster developed by the University of Berkeley. Then, the code was maintained by Apache Software Foundation. Apache provides distributed computation taking charge of task dispatching, scheduling, and basic I/O functionalities. These functionalities are available through Java, Python, Scala and R interfaces.
1Fereshteh ASGARI, Inferring User Multimodal Trajectories from Cellular Network Metadata in Metropolitan Areas
As shown in Figure 3 Apache Spark provides at the top of its framework a tool for graph mining call GraphX2. This API allows parallel graph computation and integrates tools for extraction, transformation and load. More detail about the architecture as well as the capabilities of Spark is given in the next section. 3
System Overview In the current section, we describe the architecture of our platform. As illustrated in Figure 4, our platform allows collecting data from Open Street Maps 3 (OSM) to build the graph representing streets and intersections in the form of a comma separated values CSV files. Then, these CSV files are stored in a NoSQL database. We use Elasticsearch4 as NoSQL database due to its scalable, flexible and performant search and analytics engine (c.f., Figure 5).
Once data is saved in the NoSQL database, we are able to analyze the knowledge tiers represented and combined in form of graphs trough Spark 2GraphX: http://spark.apache.org/graphx/ 3OSM: https://www.openstreetmap.org/ 4Elasticsearch :https://www.elastic. co/guide/en/elasticsearch/reference/ current/index.html GraphX as depicted in Figure 5. For instance, with this platform, we could optimize supply chain in cities minimizing cost, avoiding traffic jams and passing over low crime rate zones. We can also discover spatial patterns to understand common features of high crime rates areas in a city. All these analytics could be performed using programming languages such as: Scala5, Java6, Python7 or R8.
Finally, we implement a Python script to visualize the result of the pattern mining process using Google Maps9. In the next section, we present some preliminary visualization of graphs stored in the platform. 4
In this section, we present some preliminary results, of the Knowledge Tier Platform, about data gathering, and visualizations.
Concerning the data collection, we have done two campaigns to collect data from streets and tweets in Lima, Peru. The former campaign was performed in the month of May collecting 1˜00 000 and 4˜20 000 nodes and vertices, respectively. The latter campaign was carried on between the months of April to Jun obtaining 7˜,1 millions of geolocated tweets.
About visualization, the platform allows to plot a graph over a cartography, where the nodes are placed in the intersections of the streets and the 5Scala: www.scala-lang.org 6Java: www.java.com 7Python: www.python.org 8R: www.r-project.org 9Google Maps: /maps.google.com edges model the streets connecting nodes or intersections as shown in Figure 6.
Another possibility of visualization are Heatmaps. In our case, Heatmaps are generated based on nodes weight. For example, Figure 7 presents a Heatmap of collected tweets in the platform. It is worth noting that tweets are affected to the nearest node relying on latitude and longitude of both nodes and tweets. We use as distance function the Haversine function (c.f., Subsection 2.2). In the next section, we argue about the platform, and we present our vision of its application to research on Smart Cities. 5
We firmly believe in the potential of this project as
the cornerstone to enable new research directions.
Graphs have been widely used to model different
kinds of phenomena ranging from: urban street
network
When cities have more nanostores than retailers, it is more complicated to transport products to small spare stores.
Multi-modal transportation is a problem in urban context where individuals need to optimize their movements within a city by using different massive transportation mode.
Crime patterns could be extracted by combining different features from the graph model. Pollution dispersion could be modeled by a street and intersection models to represent and forecast particles of matter dynamic in a city. Social network activity levels could be represented in the urban graph to detect social activity for extracting the hot spots in a city. Privacy perception to understand how people consider privacy and what are the real dangers and risks.
The aforementioned list of possible research directions is not limited to these topics. There are many issues related to smart cities still opened. 6
Conclusions In the present work, we have detailed the architecture of the Knowledge tier platform. The novelty of this platform is to gather diverse kind of data from different knowledge layers to extract spatiotemporal patterns for smart cities applications. We have shown the potential of this platform as the stone corner for many research question in the near future.