Representation⋆

-

CEUR ceur-ws.org RDF/O WL

1. Introduction

Transforming natural language text into logically sound Knowledge Graphs (KGs) supports at-scale information retrieval from collections of texts.

Natural Language Processing (NLP) and Semantic Web (SW) communities dedicated significant efort to text-to-KG pipelines. The NLP community exploited the progress of Machine Learning (ML) and Neural Networks (NN) to improve semantic parsing. Graph-based semantic parsing has gained attention due to the potential of general-purpose representations, such as CEUR Workshop Proceedings parsers struggle with making the extracted knowledge interoperable and exploitable due to its formalisms’ balkanisation [11] and lax logic.

The SW provides means to formally represent the extracted knowledge according to interoperable ontologies, therefore favouring knowledge augmentation with heterogeneous Knowledge Bases (KBs) and alignment with other ontologies. The SW machine reader FRED [6] encodes the extracted information using Semantic Web (SW) standards. The resulting KGs enable the exploration and retrieval of facts extracted from heterogeneous text corpora through structured queries, as well as their augmentation through alignment to other KGs. This alignment supports the disclosure of explicit knowledge that would otherwise remain hidden in texts. However, FRED relies on cumbersome NLP pipelines, hard to maintain and unsuitable to scale in multilingual scenarios.

To overcome such limitations, this paper presents Text2AMR2FRED1, a revised architecture of FRED’s text-to-KG pipeline. It exploits pre-trained end-to-end text-to-AMR parsers, which mitigate error propagation typical of component-based pipelines. In fact, thanks to AMR’s generalization of lexical and syntactic variations, it allows a more abstract and robust representation of text, without employing ad hoc data augmentation strategies, such as lexical substitution [9, 8]. Furthermore, the SotA AMR parser’s multilingual capabilities enhance the scalability of our application, expanding its reach beyond the original English-only input restriction of FRED’s NLP pipeline.

2. Text2AMR2FRED at work

Text2AMR2FRED implements a pipeline to produce KGs automatically from unstructured text. These KGs are event-centric, as they rely on PropBank Frames2 [13]. The process for generating a KG from an input text relies on two modules: (1) the text-to-AMR parsing module, which takes natural language sentences as input and transforms them into AMR graphs. Sentences in English 1https://arco.istc.cnr.it/txt-amr-fred/ 2PropBank Frames are the core lexicon of the PropBank paradigm and consist of predicate-argument structures named “rolesets”. A complete list of PropBank frames can be found at http://propbank.github.io/v3.4.0/frames/ are parsed by SPRING3 [ 2 ]. Sentences in other languages are parsed by USeA4 [12], which takes input in 100 languages. (2) the AMR-to-FRED translation, which extends the AMR2FRED5 tool [10] to transform AMR graphs into OWL-compliant RDF KGs, following FRED’s theoretical model [6]. The integration of the two modules is eased and streamlined by the APIs provided by both tools. Specifically, the AMR graph produced by the text-to-AMR parsers from the input text can be directly used as input for the AMR2FRED tool to get a corresponding KG. This makes it possible for tools such as the Machine Reading suite6 to query both components through the Text-to-AMR-to-FRED APIs7 and generate RDF named graphs from input text sentences or paragraphs in batches. Text2AMR2FRED is also released to the public via a user-friendly web app1.

The AMR-to-FRED translation facilitates KGs enrichment, which can be achieved by employing Framester [ 5 ]. Thanks to Framester, additional relevant knowledge missing in the text (e.g., common sense knowledge) can be recovered from other KBs such as WordNet8, DBPedia9, DOLCE-Zero10. For example, the output KGs are enriched through Word Sense Disambiguation (WSD) based on the RDF version of WordNet, included in Framester. The WSD process is applied to AMR elements (usually nouns and adjectives) that miss links to lexical resources. Figure 1 shows the AMR graph corresponding to the sentence “Apple unveils revolutionary watch”. The reader may notice that the text-to-AMR parser associates predicates in AMR graphs with PropBank word senses and Named Entities with their corresponding entities in Wikipedia. The node z3 / watch instead is missing a link to lexical resources. Therefore, we disambiguate it against Framester. The WSD process consists of submitting the original sentence to EWISER11, a WSD system well-suited for multilingual scenarios due to its SotA performance in both all-words English WSD and multilingual WSD tasks. As Figure 2 shows, we associate the result of WSD (WordNet’s synsets) with the AMR nodes missing links to any external source and whose label 3http://nlp.uniroma1.it/spring/ 4https://github.com/SapienzaNLP/usea 5https://github.com/infovillasimius/amr2Fred/tree/master 6https://github.com/anuzzolese/machine-reading 7http://framester.istc.cnr.it/txt-amr-fred/api/docs 8https://wordnet.princeton.edu 9https://www.dbpedia.org 10http://www.ontologydesignpatterns.org/ont/d0.owl 11https://github.com/SapienzaNLP/ewiser corresponds to the lemma of the input sentence. This association is implemented through the owl:equivalentClass property between the identified node and the selected WordNet’s synset in Framester. For the example above, we use EWISER and keep the information for the lemma ”watch”. For the same entities (those not linked with external information sources), we further exploit Framester to generate alignments to two top-level ontologies: WordNet ”supersenses” (through the rdfs:subclassOf property) and a subset of DOLCE+DnS Ultra Lite (DUL) classes.

3. Conclusions and Future Work

Text2AMR2FRED is a tool that mitigates the issues of existing NLP semantic parsers and machine readers, adhering to Semantic Web standards to ensure interoperable knowledge extraction. It enhances the informativeness of KGs by aligning them with domain-specific ontologies, enabling interrogation through structured queries. This approach uncovers implicit knowledge from text, enabling the output of KGs with external KBs.

Future work will focus on creating resources for the evaluation of the tool. The AMR parsers employed in our tool can be leveraged to perform AMR-to-text tasks and compare the original textual excerpts to the automatically generated ones via a back-translation [16] approach. This allows the calculation of similarity metrics, such as BLEURT [15] or others, between the original and generated texts. Under the hypothesis that generated sentences with (relatively) high similarity scores correspond to high-quality AMR graphs, automatic filters can be designed and applied to prevent lower-quality AMR graphs’ transformation into RDF/OWL KGs. Our evaluation method will be completed by the analysis of Motifs, basic logical patterns employed in SW, defined in [ 7], in the output KGs. The Motifs-based validation will permit a cross-tools knowledge extraction tasks comparative evaluation, following the method outlined in [ 4 ].

References

[1] L. Banarescu , C. Bonial , S. Cai , M. Georgescu , K. Grifitt , U. Hermjakob , K. Knight , P. Koehn , M. Palmer , and N. Schneider . Abstract Meaning Representation for Sembanking . In Proc. of the 7th Linguistic Annotation Workshop and Interoperability with Discourse , pages 178 - 186 , Sofia, Bulgaria, August 2013 . ACL.

[2] M. Bevilacqua , R. Blloshmi , and R. Navigli . One SPRING to Rule Them Both: Symmetric AMR Semantic Parsing and Generation without a Complex Pipeline . Proc. of the AAAI Conference on Artificial Intelligence , 35 ( 14 ): 12564 - 12573 , May 2021 .

[3] R. Blloshmi , R. Tripodi , and R. Navigli . XL-AMR: Enabling Cross-Lingual AMR Parsing with Transfer Learning Techniques . In Proc. of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 2487 - 2500 , Online, November 2020 . ACL.

[4] A. Gangemi . A Comparison of Knowledge Extraction Tools for the Semantic Web . In The Semantic Web: Semantics and Big Data , pages 351 - 366 , Berlin, Heidelberg, 2013 . Springer Berlin Heidelberg.

[5] A. Gangemi , M. Alam , L. Asprino , V. Presutti , and D. R. Recupero . Framester: A Wide Coverage Linguistic Linked Data Hub . In EKAW 2016 , pages 239 - 254 , Bologna, Italy, 2016 . Springer International Publishing.