Data lakes are repositories of data stored in natural/raw format. A data lake may include structured data from relational databases, semi-structured data (i.e., JSON, CSV), unstructured data (i.e., text data), or binary data (i.e., images, audio, video). It is usually built on top of cost-eficient infrastructures such as Hadoop, Amazon S3, MongoDB, ElasticSearch, etc. Several organisations rely on big data lakes for crucial tasks such as reporting, visualisation, advanced analytics, machine learning, and business intelligence. A major limitation of this solution is that without descriptive metadata and a mechanism to maintain it, such data tend to be noisy, making their management and analysis complex and time-consuming. Therefore, there is the need to add a semantic layer based on a formal ontology to describe the data and eficient mechanism to represent them as a knowledge graph. In this paper, we present a methodology to add a semantic layer to a data lake and thus obtain a knowledge graph that can support structured queries and advanced data exploration. We describe a practical implementation of a methodology applied to a data lake consisting of text data describing the online marketplace for lodging and tourism activities. We report statistics about the data lake and the resulting knowledge graph.
The term “data lake" was introduced by James Dixon, Chief Technology Oficer of Pentaho, in a blog post in 20101. Data lakes are data repositories for storing large and heterogeneous sets of raw data. They have quickly become a common data management solution for organizations that desire to own a holistic and large repository for their data. Data lakes allow users to access and explore data without the need to move them into another system. Insights and reporting performed from a data lake typically occur on an ad-hoc basis. However, users might apply a schema and a certain degree of automation to the data to make it possible to duplicate a report when needed.
Data in a data lake are stored in their raw format and are not transformed until they are
needed for analysis. Also in such a case, a schema is somehow applied so that they can be
analyzed. This way of working is called “schema on read" [
Several attempts have been done to provide a semantic layer to data lakes and each of them
has targeted a particular domain of application [
The remainder of this paper is organised as it follows. Section 2 focuses on the previous works on semantic layers for data lakes. Section 3 describes our methodology and presents the implementation in the tourism domain. We provide also statistics and information about the resulting knowledge graph. Section 4 ends the paper with conclusions and future works.
A knowledge graph [
Dibowski et al. [
Diferently from the approaches above, we propose a methodology to extend a data lake containing data extracted from touristic platforms with a semantic layer and produce a knowledge graph. To this end, we engineered an ontology in the touristic domain integrating already existing ontologies and extending them with our classes. However, the focus of this manuscript is not on the ontology but on the extracted knowledge graph and the steps we performed to transform the data from the data lake to the knowledge graph.
In this section, we describe our methodology for enriching a data lake by creating a domain ontology and generating a knowledge graph that will extend the data lake with a sophisticated representation of knowledge. This approach is articulated in five steps: i) analysis of the data sources; ii) definition of the use cases; iii) creation of the ontology; iv) data transformation; and v) generation of the knowledge graph. In the following we will briefly describe each phase.
The data lake we have used comes as a result of the Data Lake Turismo6 project whose aim was to create a digital platform of tourism data. The data lake was developed by Linkalab7 through Amazon Web Services (AWS) cloud computing technologies including S3 where the
data was stored. The data lake collected data from various sources using an Extraction, Load and Transformation (ELT) approach. A crawling system was developed to identify and extract data related to the London Region8 area from various sources including Booking.com9 and AirBnB10.
The data lake is organised in three tiers: i) intake tier, where the raw data is collected, ii) curated tier, where each transformed/cleaned version of the data is stored, and iii) consumption tier, where the data is exposed to business analysts in many formats such as reports, dashboards, APIs. In our specific scenario, the intake tier contains HTML files extracted from Booking.com, and JSON files extracted using APIs exposed by AirBnB systems; the curated tier contains JSON data extracted from the Booking.com HTML files; the consumption tier contains the knowledge graph as a set of RDF triples.
The data lake is built on AWS serverless technologies: Amazon S311 object storage is used to store the files, AWS Lambda 12 and AWS Fargate13 are used to execute the crawling and the data processing, Amazon Athena14 is used to query the data stored in JSON files using SQL language while all technical metadata is managed using AWS Glue catalog15.
The data lake describes three kinds of entities: • lodging facilities i.e., any hotel, holiday house or other quarters that provide temporary sleeping facilities open to the public16, which are described by specific properties like name, address, geolocation, average user rating, textual description, pictures, related amenities; • accommodation ofers i.e., a specific place that can accommodate persons (e.g. a hotel room, a camping pitch or an entire apartment) that is part of a lodging facility and is ofered for lease under specific conditions; these ofers are characterised by specific properties like number and type of beds, max and min occupancy, related amenities, price; 8The London Region area is an administrative area including the 32 London boroughs and the City of London. 9https://www.booking.com/ 10https://www.airbnb.com/ 11See https://aws.amazon.com/s3/ 12See https://aws.amazon.com/lambda/ 13See https://aws.amazon.com/fargate/ 14Athena is a query engine based on PrestoDB. See https://aws.amazon.com/athena/ and https://prestodb.io/ 15See https://aws.amazon.com/glue/ 16Source: Law Insider, see https://www.lawinsider.com/dictionary/lodging-facilities • user reviews about the lodging facility that are characterised by a rating value and a text.
Table 1 reports an overview of the number of business entities stored for both sources (Booking.com and AirBnb). For AirBnb we have the same amount of lodging facilities and accommodation ofers, because AirBnB associates each ofer to a unique lodging facility. Conversely, in Booking.com a lodging facility (e.g., hotel) can ofer multiple accommodations (e.g., rooms).
Table 2 summarize of the storage space used in the data lake. The main diference between Booking.com and AirBnB is that the first is crawled exporting HTML pages that are then used to extract the data whereas the latter is accessed using APIs to retrieve the data itself already in JSON format.
The purpose of the creation of the Data Lake Turismo project was to analyse the supply and demand side of tourist destinations. During the project development the following use cases have been identified in collaboration with the analysts of Linkalab:
2. Identify the topics of interest in the text presentations of lodging businesses ofers; 3. Detect the sentiment [18] of tourists toward a certain lodging business or destinations; 4. Classify tourist destinations according to what they ofer and according to the tourist opinions.
To better support these use cases the data lake has been extended with a semantic layer supported by an ontology. The resulting knowledge graph includes both data and metadata, hence enhancing the support for developing dedicated services.
A crucial step is the creation of a domain ontology that could support the use cases. For this purpose is possible to rely on standard ontology engineering frameworks and evaluation methodologies [19].
In our implementation, the ontology has to satisfy both functional and non functional requirements. As far as functional requirements are concerned, the ontology has to include classes for lodging businesses (e.g., hotels, hostels, apartments), accommodations ofered by them (e.g., rooms, suite), amenities for tourists, tourist attractions and points of interest, inter-relations among entities (e.g., geographic relations, composition/inclusion), tourist reviews, tourist destinations and taxonomies to support all of them. As far as the non functional requirements are concerned the ontology must be defined in OWL, and be based on Schema.org17 and GoodRelations18.
To drive the creation of the ontology, we designed a set of competency questions and identified a set of existing ontologies that have been used as support. The entire ontology creation is not discussed in this manuscript because out of the scope of the paper which focuses on the methodology for the creation of a knowledge graph to support a data lake.
The data transformation depends on the source data structures and on the desired output. The steps needed to transform the data are: i) extraction of relevant structured data and texts from the original sources; ii) data cleaning; iii) ontology mappings, to represent the entities in the structured data according to the ontology; iv) language detection, to identify the source of the language; v) identification and extraction of entities within the text.
The last step is very crucial to obtain a good representation of the data, since many important information are only expressed in natural language, especially in the text regarding the description of the lodging facilities and reviews. To this purpose, we used DBpedia Spotlight entity linking approach for extracting common entities such as activities, events, places, and food.
We then integrated this information in the knowledge graph by linking DBpedia entities with the relevant lodging facilities. This allows our system to support advanced queries such as retrieving all the accommodations that are close to touristic attractions, those that ofer a specific amenity or propose a special kind of food, but also looking for what places or events users cite most frequently in their reviews.
The last step takes in input the refined data and the ontology and produced the knowledge
graph. To this purpose it is possible to rely on several languages and tools for the the automatic
generation of triples [20, 21]. In our implementation, we adopted the RDF Mapping Language
(RML) [
Triples are generated for each of the triples maps of the RML mapping. In our prototype, we
used RMLMapper [
We ingest new data from the original sources into the data lake every two months. We then recreate the knowledge graph from scratch by repeating all the data transformations steps described in Section 3.4.
Table 3 reports some metrics about the last version of the knowledge graph: i) total statements refers to the overall number of triples stored in the triplestore (both explicit and inferred), ii) explicit statements refers to the number of raw triples created in the triplestore, iii) inferred statements refers to the number of triples inferred by the reasoner from the explicit statements, iv) expansion ratio represents the percentage of triples added using the inference. The other metrics are self explanatory.
In this paper we have presented a general methodology for extending a data lake with a knowledge graph. In particular, we have focused our analysis to the tourism domain by considering a data lake containing structured and unstructured data crawled from Booking.com and AirBnB. The knowledge graph thus obtained has been stored into a triplestore which can be accessed online.
We can conclude that the semantic layer provided by the knowledge graph brought many advantages to Linkalab’s data lake platform: i) it treats data and metadata in a unified way, ii) it has a flexible schema that can support the data variety and evolution, iii) it supports algorithms and applications development and data science activities based on the data lake; iv) it embeds information in its graph structure that can be leveraged by graph analytics [22, 23] and representation learning [24] algorithms; v) it incorporates knowledge extracted from texts along with structured and semi-structured data typically found in the data lake; vi) it can be used to expand the data lake information context through connections with open knowledge graphs like DBpedia.
In future work, we aim to expand the pipeline for producing the knowledge graph by developing new solutions for entity extraction and to further improve the ontology. We also plan to develop a tool that will take advantage of the knowledge graph for analysing and comparing accommodations and generating explainable recommendations.