There is a vast amount of linked open data published nowadays on the web, ranging from user-generated data to government public data. This data needs visualization and exploration tools which people can use to make sense of the data and turn it into useful and used information. Several such tools have been proposed in the past, but they are designed mostly for specialists. Augmented reality has recently emerged as an interactive medium for exploring information in the real world and is well-suited for non-specialists. We propose a demo of a mobile augmented reality application that runs in the browser and consumes and displays linked open data in a friendly manner on top of the surroundings of the user.
Due to the vast amount of data that was published in the latest years, linked data has
become a popular research field. The Linking Open Data cloud1 has grown from 12
datasets in 2007 to 295 datasets in 2011. One of the major research areas is related to
the consumption of linked data by browsers designed specifically for this task. These
browsing and visualization tools are required by tech users and lay users equally to
easily retrieve information from the Web of Data. A recent survey [
Mobile augmented reality applications have recently emerged as interactive and
usable tools for exploration of the surrounding world of a user [
2 http://dev.cm.upt.ro/ar/
The application implements the crawling/data warehousing pattern. Figure 1 highlights the steps needed to build an augmented reality application that uses data from multiple sources.
In the first step, a list of data sources is identified with the purpose of being used in an augmented reality application. The data sources have to contain information regarding places with geographic coordinates, with labels and descriptions, ideally in multiple languages, and with mapping links and specific categories.
In the second step, the data from these multiple data sources is collected and mapped to a consistent vocabulary. The identity resolution is resolved for different URIs addressing the same resource, a data quality assessment is provided, the merged data is stored into a RDF store and an API is provided for the AR application to obtain the desired information in JSON format.
The mobile augmented reality application, the third step, is browser based so there is no need for the user to download a standalone application from the store. Given the detected geolocation of the user, the application displays a set of Points of Interest (POIs) in the immediate vicinity of the user. The information about the POIs is consumed from the triple store, via the above-named API.
The first two steps are described in section three and the AR application in section four. 3
In order to build a dataset usable in an augmented reality application, first of all we identified possible data sources that satisfied the requirements described above. The datasets chosen for this demo application are extracted from DBpedia.org, LinkedGeoData.org and the Romanian Government Open Data portal3.
In order to collect, map and integrate the chosen datasets we used the powerful
Linked Data Integration Framework (LDIF) [
Data translation was carried out to obtain a consistent list of categories and to build geometry values, if missing. Identity resolution was employed using the Levenshtein distance with a threshold of 1 on labels and comparing the distance between POIs with a threshold of 100 meters. For quality assessment the metric was time closeness; this implies that only the most recent data is used in the data fusion step.
The resulted dataset was outputted by LDIF in an OpenRDF Sesame RDFS store and contained 25,000 statements for the city of Timisoara. To allow the AR application to query the data, an API was built that can query triples from the RDF store and return them in JSON format. 4
The augmented solution chosen is a browser-based one. Modern web technologies (HTML, CSS, Javascript) combined with the current capabilities of mobile devices that are available in the browser (geolocation, camera, WebGL) have given rise to such solutions. Our demo uses the recently launched awe.js library, 4 which builds an augmented reality layer on top of the three.js library, a 3D library working in the browser. Awe.js library is advertised to work both with location and marker based AR, on the latest versions of Chrome and Firefox on Android, as well as on devices such as Oculus Rift, Leap Motion and Google Glass. We successfully tested our web application so far on Chrome v35, Firefox v30 and Opera v22 on a Nexus 4 smartphone with Android 4.4.
The web application queries asynchronously the Sesame server (through a proxy, to overcome the same-origin policy) to get the required POIs, based on the location of the user, an area around it, where to search the points, and the desired category of interest. It then processes the list of retrieved points to place 3D pinpoints into the space, based on the category the POI belongs to and on the distance from the user to that POI. On touching a certain POI, the user is presented with a short snippet of information, which he can expand to read more about that place. Figure 2 shows some screenshots of the application.
In this demo paper we proposed a mobile augmented reality application that runs in the browser and consumes and displays linked open data. To accomplish this we used three open datasets, a powerful linked data integration framework, LDIF, an OpenRDF Sesame RDFS store and we built an API for extracting data from the store and delivering it to the mobile augmented reality browser application built on top of awe.js. As future work we intend to include: submenus (to be able to further refine the selection of the POIs), UI improvement, additional datasets, improved data quality. Acknowledgements. This work was partially supported by the strategic grant POSDRU/159/1.5/S/137070 (2014) of the Ministry of National Education, Romania, co-financed by the European Social Fund – Investing in People, within the Sectoral Operational Programme Human Resources Development 2007-2013.