A Knowledge Graph (KG) is a form of structured human knowledge depicting relations between entities, destined to reflect cognition and human-level intelligence. Large and openly available knowledge graphs (KGs) like DBpedia, YAGO, WikiData are universal cross-domain knowledge bases and are also accessible within the Linked Open Data (LOD) cloud, according to the FAIR principles that make data findable, accessible, interoperable and reusable. This work aims at proposing a methodological approach to construct domain-oriented knowledge graphs by parsing natural language content to extract simple triple-based sentences that summarize the analyzed text. The triples coded in RDF are in the form of subject, predicate, and object. The goal is to generate a KG that, through the main identified concepts, can be navigable and linked to the existing KGs to be automatically found and usable on the Web LOD cloud.
Due to the complex nature of real-world data, defining a data knowledge model within the specification of a domain is not a trivial task. Semantic Web Technologies (SWT) promise formal paradigms to structure data in a machine-processable way by connecting resources defining semantic relations among them. The result is a rich information representation model whose information is well defined, uniquely identifiable, and accessible as a knowledge base. However, the knowledge representation (KR) task is becoming more challenging every day due to the increasing need to improve, extend, and re-adapt that knowledge to the real-world evolution.
Knowledge Graphs (KGs) provide a way of represent knowledge at diferent levels of
abstraction and granularity; generally it is described [
Thanks to the W3C Resource Description Framework (RDF) standard1, knowledge can be expressed as a network of atomic assertions, also called facts, described as triples composed of Subject (S), Predicate (P) and Object (O), = (, , ). In the light of the traditional definition of heterogeneous graph = (, , ), a KB can be described by mapping ≡ ( ∪ ) and ≡ as the semantic relation between nodes, leaving the description of vertices as ∀ ∈ : = (, , ). On the other hand, the KG Reasoning System (KRS) can be interpreted as the combination of one or more computing or reasoning (r) techniques to read or write the the statements in the KB, and it can be expressed as = (, ) or = (, (, , )) where = {1, 2 . . . } and ≥ 1 e.g.: : KGE() = (, , ). In a nutshell, a Knowledge Graph (KG) is composed of a Knowledge Base (KB) and a Knowledge Reasoning System (KRS) = ({1, 2 . . . }, (, , )).
The FAIR [
This work proposes a methodological approach implemented as a library to build domainspecific knowledge graphs compliant with FAIR principles from an automated perspective without using knowledge-based semantic models. The method leverages traditional Natural Language Processing (NLP) techniques to generate simple triples as a starting point and then enhance them using external OpenKG, e.g., DBpedia, to annotate, normalize, and consolidate an RDF-based KG. As a case study, novels in literature were explored to validate the proposed approach by exploiting the features from the narrative construction and relationships between characters, places and events in the story.
The remainder of this work is structured as follows: Section 2 provides an overview of relevant approaches, knowledge base construction foundations, and KG construction approaches. Section 3 presents the proposed methodological approach at a high abstraction level. Then, Section 4 describes details of the developed tool and the case study results. Experimental evidence and evaluation is reported in Section 5. Finally, the conclusions and the future work are highlighted in Section 6.
The Knowledge Base (KB) lifecycle [
A knowledge base construction (KBC) can be achieved by performing the first two stages of the
KB life cycle, populating a KB [
The KB is often a Commonsense Knowledge (CSK) since it refers to generic concepts from
daily life; classifying and getting more specific information can enhance the CSK by adding new
statements annotated semantically. In [
Ontological reasoning, on the other hand, needs synergistic integration: in [16], for example, lexical-syntactic patterns are extracted from sentences in the text, exploiting an SVM (Support Vector Machines) algorithm and then finding matches with DBpedia; the resulting matches are then integrated by a Statistical Type Inference (STI) algorithm to create a homogeneous RDF-based KB. Furthermore, Open Information Extraction (Open IE) provides atomic units of information, with individual references (URI) to simplify conceivable queries on external databases, such as DBpedia [17].
The KB construction has evolved along with the knowledge representation requirement for data ingestion as well as human consumption of data [18]. This evolution has increased the complexity in terms of data dimensionality, number of sources, and data type diversity [19]. For these additional challenges [20], semantic tools [21] can enhance the collected data by providing cross-context, allowing inference from native data relationships or external data sources, such as status reports or supply chain data.
Once an initial KB has been deployed, the Knowledge Completion can be accomplished, leveraging on the open-world assumption (OWA) also by using masking functions [22] to connect unseen entities to the KG. Specifically, a masking model uses text features with the structure = (, , ?) to learn embeddings of entities and relations via high-dimensional semantic representations based on the known Knowledge Graph Embedding (GKE) models such as TransE neural network [23]. This approach can also be used for link prediction by exploiting existing GKE models such as DistMult, ComplEX, to perform inference using the masked form = (, ?, ).
Knowledge Graph Embeddings (KGE), also called Knowledge Graph Representation Learning, has become a powerful tool for link prediction, triple classification, over an existing KG. These approaches encode entities and relations of a knowledge graph into a low-dimensional embedding vector space; these networks are based on Deep Learning techniques arriving at a score by minimizing a loss function ℒ.
KGE provides a comprehensive framework to perform operations [24] over a KB, usually provided in the KB deployment phase. TransE [23], being one of the most studied KGE models, = −‖ + − ‖, computes the similarity between the translated by the embedding of the predicate and the object . TransE model is often exploited for applications such as a knowledge validation model and achieves state-of-the-art prediction performance.
TorusE is a TransE-based embedding method that, combined with the use of Lie Groups knowledge base (KGLG) [25], is used to validate an inferred KB; it has been tested using standard de-facto datasets to evaluate the multidimensional distance between spaces of computed embeddings. Other KGE methods such as DistMult [26] = {, , } use a trilinear dot product to capture and extract semantic relations, while the method ComplEx [27] = ({, , }) uses a Hermitian dot product.
The proposed approach provides an incremental unsupervised methodology, compliant with the FAIR principles, to build a knowledge graph on a specific domain, without exploiting any ontology schema, but achieving the discovery, construction, and integration of triples = (, , ), extracted from an unstructured text, with zero initial knowledge. The approach is described by the pipeline of Figure 1 shows the interaction of the main components of the proposed framework.
From a high-level perspective, the input is represented by the natural language text sources and feeds the Triple Construction component, in charge of extracting triples from the analysis of sentences in the text. The text parsing and the triples extraction are tailored to exploit the specific prosaic text features, starting from the narrative properties and the implicit relations created by the recurring entities described in the text. The facts extracted from textual data sources are plain triples < , , > that, are incorporated as formal statements in the KB. The triple elements are annotated semantically, by querying external semantic knowledge bases such as DBpedia or Wikidata. This task is accomplished by the Triple Enrichment component, generating a RDF-compliant KB. Afterward, this triple set is input to Embedding Training component responsible for the KB completion. It accomplishes the reasoning on the KB, exploiting a KGE model, to infer and validate the collected KB.
Next subsections provides additional details about the introduced components of the framework pipeline of Figure 1.
NLP-based parsing tasks are applied to extract plain triple-based sentences. The Triple Construction component initially carries out the Basic POS annotation on the input raw text, consisting of traditional preliminary text analysis, namely, the tokenization, stop-word removal and -tagging routines to identify and annotate the basic part of speech entities.
Once the Basic POS annotation is accomplished, parallel activities, namely Information Extraction and Chunking Rules and Tagging Pattern Execution are carried out, as shown in Figure 2. 3.1.1. Information Extraction: This block is composed of three basic tasks necessary to get an further processing of the tagged text yield by the Basic POS Annotation. The text is indeed given to the Named Entity Recognizer (NER) which identifies named entities according to some specific entity types: person, organization, location, event. Then a Dependency Parsing task allow locating named entities into subject or object and finally, the Relation Extraction task is in charge of the triple composition, starting by the detected entities. Let us remark that the named entities has a crucial role in the triple identification and generation. 3.1.2. Tagging Pattern: This task accomplish ad-hoc text processing, considering the nature of the text from a linguistic perspective. Linguistic patterns were defined specifically for processing the prosaic narrative and for this reason, for example, some stop words are not discarded. Those patterns can be implemented to discriminate additional triple-based statement to feed the KB; in particular, a finite-state machine is proposed, implementing three evaluation steps, one for each triple component = (, , ). Figure 3 sketches a few of the implemented state transitions used in the development of this methodology. 3.1.3. Chunking Rules: This task achieve a parallel shallow syntactic analysis where certain sub-sequences of words are identified as forming special phrases. In particular, a regular-expression has been defined over the sequence of word tags to select and extract basic triple for enriching the KB, such as the following tag string : {< >? < * >< >? < * >< >? < * >< >}. This regular expression pattern says that a triple pattern TP chunk should be formed by the subject chunker which finds an optional determiner (DT) followed by any number of adjectives (JJ) and then a noun (NN); then the predicate-chunker composed of the verb (in its tenses) an finally the object-chunker that is similar to the subject chunker, except that it could be a pronoun besides the noun. Subject and object chunkers are typical noun phrases. Some rule example application is shown in Figure 4.
As final result of the Triple Construction component, all the generated data are integrated into a set of plain triples, ≡ ∪ ∪ ℎ, where the represents resulting triples of Information Extraction task, the , the result of Tagging Pattern and ℎ is the Chunking Rules result, once redundant and duplicated are discarded. Finally, the resulting triple set is stored into a plain format file.
Once the construction of the triple has been completed and stored, the Triple Enrichment component (Figure 1) is in charge of annotating semantically the triple elements. First, each element of triples is a candidate to be enriched semantically, by assigning a URI to it; for this purpose, SPARQL queries whose arguments are the individual subject and object of the triples are submitted to OpenKGs such as DBpedia or Wikidata, to retrieve the relative URIs, when they are available. Similarly, the predicate is also looked up in the semantically enhanced lexical database WordNet, once the verb has been transformed into its infinitive form.
1: for all [, , ] : do 2: for (S and O) as Entity do 3: [URI, Label] = QueryOpenKG(Entity) 4: triples.replace( Entity, URI ) 5: triples.append( (URI, rdfs:label, label) ) 6: end for 7: O = QueryWordNet( tokenization(O) ) 8: end for 9: return RDF Triples
The general process of the Triple Enrichment component is described in Algorithm 1. It takes the triple collection as input and for each triple subject and object, search for the corresponding URI in a OpenKG; and then, once retrieved, replaced the URI in the triple, along with the associated rdfs:label. Let us notice that not only the external OpenKG URI is extracted from the query, but also the rdfs:label for each entity added to the KB. A similar query is carried out on WordNet, for the predicate: a simple disambiguation task is accomplished considering the other triple elements, by similarity between terms in the WordNet definition and triple words. Otherwise, the first sense is retrieved for default. Future development are taken into account to select the right sense after a more accurate disambiguation task. The output of the Triple Enrichment component is a semtic version of the collected triples: − ≡ − ∪ − ∪ − ℎ, where − , − , − ℎ are the RDF-annotated triples coming from the corresponding tasks of the Triple Construction component.
Last component of the described pipeline is the Embedding Training that is responsible to train the KGE model on the processed triples. The KGE model provides a mechanism to perform operations with the discovered KB, such as fact checking, question answering, link prediction and entity linking.
According to the close-world knowledge assumption (CWA) [22] of the knowledge graph, in general no new statements (triples) can be added to an existing KG, except that predicting relationships between existing, well-connected entities. In the light of this consideration, it is essential to integrate some reasoning mechanism (KRS) that allows the KG to perform operation within the statements inside the KB, to reach the so called closed-world Knowledge Graph Completion. At the same time, once all the possible statements are saturated, additional new statement could be added by considering external information sources, in order to enrich the KB.
The approach has been implemented and validated, considering a case study relative to the narrative domain, specifically books and article synopses describing short stories, fairy tales, adventure stories. This section describes the methodology at a technical level to identify the main characters, capture the relationships among them, places and events where they are involved by exploiting linguistic features and qualities of the narrative prose.
An enlarged view of the proposed pipeline of Figure 1 is shown in Figure 5 where the whole high-level framework is described. A synergy between diferent methodologies and technologies has been achieved: NLP activities are integrated with the Machine Learning models, designed according to the principles of Information Retrieval and FAIR, through the Semantic Web technologies (RDF, Turtle, OWL and SPARQL).
The implementation is based on Python language and cover the core system shown in Figure 5. Input textual resources are downloaded from Kaggle narrative books, available in plain text format; at the same time, the synoptic version of the book was retrieved from Wikipedia. Our case study is applied to the book The Lord of the Rings. The text was extracted using a simple web crawler that retrieve the content of the URL page; the data is cleaned using a standard HTML/XML parser (Data Extraction).
As shown in Figure 5, the Triple Construction component achieves the traditional POS-tagging activities (Figure 2). The information extraction is performed by the Stanford Open-IE server tool combined with Spacy library (en_core_web pre-trained models in English) for the extraction of triples.
The Tagging Pattern and Chunking Rules tasks from Triple Construction are ad-hoc designed; these tasks also were implemented by exploiting WordNet. To give an example of pattern extraction described in Figure 3 the text "But Frodo is still wounded in body and spirit" is analyzed according to the regular expression = ( * , * , * ), where the wildcard character is a placeholder for possible variations of that POS-tags. The generated triple is = ( , , ).
The triple extraction is also aimed at generating the widest set of triples from the sentence analysis. Along with the definition of regular expression patterns for discovering simple phrase chunks, the co-reference is also achieved: primary sentence with the same subject in secondary sentences or simply sentences with pronoun are re-arranged to get triples with noun instead of pronoun. The rule ( , , ) in Figure 4 allows us to obtain the pronoun PRP that will be replaced by the right a noun ( in the previous sentence NN = ℎ), to get the triple 2 = (ℎ, , ).
Once the Triple Construction process is concluded, the generated triples are rearranged, discarding duplicates (coming from the parallel tasks). The remaining triples are ordered according to the same subject and then predicate and then stored in a plain CSV file for further use.
The Triple Enrichment component uses the CSV-generated file to carry out Algorithm 1. An initial Named Entity Recognition task allows discriminating important entities among subject and object, in the triples in order to discard irrelevant triples (i.e., triples with no named entity) Algorithm 2 SPARQL Query example to URI and Label extraction
SELECT ∗ WHERE { {
{ { } } d b r : { word } r d f : t y p e dbo : F i c t i o n a l C h a r a c t e r .
d b r : { word } dbp : name ? l a b e l . } } UNION { { { { d b r : { word } dbo : w i k i P a g e R e d i r e c t s ? URI } } UNION { { d b r : { word } dbo : w i k i P a g e D i s a m b i g u a t e s ? URI } } . ? URI r d f : t y p e dbo : F i c t i o n a l C h a r a c t e r .
? URI r d f s : l a b e l ? l a b e l } }
FILTER ( la ng ( ? l a b e l ) = ' en ' ) and more important, to use that entities as parameter for the query, according to Algorithm 1. The individual entities are looked at DBpedia by submitting SPARQL queries, to retrieve the semantic annotation (URI and label) corresponding to that entity. The goal is to make the subject or object on each triple accessible resources, according to the Linked Data principles.
The query shown in Algorithm 2 was submitted to the public SPARQL end-point 4 by implementing the SPARQLWrapper to retrieve the DBpedia URI of the selected entities and the rdfs:label. Let us notice that when a URI of an entity is not available, i.e., no result is returned by the query, a customized URI is generated. In a similar way, the predicate is processed using WordNet after reducing it to the infinitive form, e.g., the several tenses kills, killed, killing become the infinite form kill. After these activities, a semantic knowledge base (called RDF KB) should be available.
The resulting RDF KB is passed to the KGE models (Embeddings Training in Figure 5 to get the embeddings on the RDF triple-based KB. Embedding models such as ComplEX, TransE, and DistMult were trained to compare and evaluate the results across this implementation. The training was achieved and evaluated by using the evaluation proposed by the KG Embeddings of the framework Ampligraph [28]. 4.0.1. Triple Storage: An important role was assumed by the storage component, that allows to store all the intermediate version of the triples generated. In a nutshell, triples generated in the Triple Construction component (Section 3.1) are stored in CSV format (to reuse them later); then, after the the Triple Enrichment (Section 3.2), the resulting triples are stored semantically, namely in RDF/Turtle languages. RDF-based triples can be e imported into the NEO4J5 by the neosemantics plugin and then the resulting knowledge graph can be graphically displayed into the GUI This last
storage task is crucial for reusing the KB for later validation, charts generation, query execution, and further operations. 4.0.2. Source Code: A demo for the implementation of this project is available on GitHub. The usage instructions are consigned on the repository6 and the produced results (KBs) should be published under the CC-BY-SA license.
The KB validation is performed by implementing a cross-graph representation learning approach [29] by embedding triples based on the semantic meaning of the KB; the representation is done by estimating a confident score in each statement based on a degree of correctness. This estimation is calculated by exploiting the KGE models and the corresponding evaluation metrics over the discovered KB.
First, the validation is accomplished by considering the output of each tasks of the Triple Construction component and setting a pair-wise comparison among − , − , − ℎ through the trained KGE models, with the purpose of measure the correspondence between the discovered knowledge. Then, to assess the generated knowledge base, new triples are added to the KGE models once trained the trustworthiness (i.e., how they are true) of these triples. These new triples are called "synthetic", because they are manually generated, are used to perform link predictions queries of the form = (?, , ?). The synthetic triples are divided into two groups, positive ( , existing) and negatives (, not existing, or unlikely to be true). Known learning-to-rank metrics to assess the performance of KGE models are as follows.
1 ∑|︁| 1 MRR = (1) HITS@N = | ∈ : < | (2)
|| =1 ||
The Mean Reciprocal Rank (MRR) is a statistical measure for assessing the quality of a list of possible answers to a sample of queries, sorted by probability of correctness. MRR assume value equal to 1 if a relevant answer was retrieved at rank 1; the intention is to measure the correctness of each of the proposed scenarios using to validate source knowledge against the validation dataset. The @ measures the percent of triples that were correctly ranked in the top . Besides, the combination with , with two diferent coeficients = 5 and = 3 to measure the ranking of the entities part of the statement (S, O) in the validation dataset at the time of link prediction.
For the validation of the KBC process, let us focus on the KB generated by the Triple Construction component described in Section 3.1. After the basic POS tagging tasks, the three sub-tasks,
Information Extraction, Chunking Rules and Tagging Pattern Execution generates statements for the graph construction, the statements are grouped in , , ℎ, respectively.
The triples generated by using the Information Extraction ( ) are approximately 75% of the discovered triples. The remaining 25% can be split between the triples (about 10% of the total KB generated) coming from the Chunking Rules (ℎ) and those (around 15%) from Tagging Pattern components ( ). Overlapping statements represent 1% of the triples discovered.
.
The results suggest that the validation of the synthetic triples produces high values (about 0.95) of for the positive triples and 0 for the negative triples. The synthetic statements appear to accurately reflect the knowledge discovered by the methodology and the correspondence implementation, and the positive statements are potential candidates to feed into the existing knowledge base.
This paper proposes unsupervised knowledge construction from domain-specific knowledge given a natural language text. In particular, a case study is designed on a narrative text, exploiting features unique to prosaic text. The text was parsed and basic triples, representing the core part (subject, predicate and object) of the main sentences are extracted. By Semantic Web technologies, the triple are semantically annotated to be linked and navigable into the Linked Data cloud. At that point, Knowledge graph embedding models have been exploited to extend the potential of the collected triples by verifying and validating the so generated knowledge. Performance is assessed by cross-validation, scoring functions and accuracy on unseen triples. 0.53 0.40 0.42 0.28 0.40 0.28 TransE 0.92 0.01
MRR
MRR 0.63 0.48 0.52 0.30 0.53 0.33 MRR
The work provides a general methodological approach to automatically build a knowledge base, on a preferred domain, given in textual resources. It allows the creation of diferent types of KBs. It can be leveraged to build ad-hoc knowledge bases, tailored to a specific domain. Question-answering systems can act as validation systems when verification of the veracity and reliability (degree of truth) of a given statement is required, for example, for the discovery of fake news or just for fact-checking.
As future work, the generation and enrichment of triples by querying existing Open Knowledge bases will be improved, along with the more challenging goal of enhancing the degree of confidence in the KB by implementing an automatic validation method.