We present an end-to-end approach that takes unstructured textual input and generates structured output compliant with a given vocabulary. We treat the triples within a given knowledge graph as an independent graph language and propose an encoder-decoder framework with an attention mechanism that leverages knowledge graph embeddings. Our model learns the mapping from natural language text to triple representation in the form of subject-predicate-object using the selected knowledge graph vocabulary. Experiments on three di erent data sets show that we achieve competitive F1-Measures over the baselines using our simple yet e ective approach. A demo video is included.
Converting free text into usable structured knowledge for downstream
applications usually requires expert human curators, or relies on the ability of machines
to accurately parse natural language based on the meanings in the knowledge
graph (KG) vocabulary. Despite many advances in text extraction and
semantic technologies, there is yet to be a simple system that generates RDF triples
from free text given a chosen KG vocabulary in just one step, which we consider
an end-to-end system. We aim to automate the process of translating a natural
language sentence into a structured triple representation de ned in the form of
subject-predicate-object, s-p-o for short, and build an end-to-end model
based on an encoder-decoder architecture that learns the semantic parsing
process from text to triple without tedious feature engineering and intermediate
steps. We evaluate our approach on three di erent datasets and achieve
competitive F1-measures outperforming our proposed baselines, respectively. The
system, data set and demo are publicly available12.
Inspired by the sequence-to-sequence model[
Lake George is at the southeast base
of the Adirondack Mountains Bi-directional LSTM
Concatenate Encoder
<Start_of_Triple> dbr:Lake_George_(New_York)
dbr:George_Lake dbr:Lake_George_(Florida) dbr:Lake_George_(New_South_Wales) Decoder dbo:country dbo:birthplace dbo:location dbo:isPartOf
dbr:Adirondacks
dbr:Adirondack_Mountains
yago:Mountain109359803
dbr:Whiteface_Mountain
As shown in Figure 1, the model consists of an encoder taking in a natural
language sentence as sequence input and a decoder generating the target RDF
triple. The model pursues the maximized conditional probability
p(Y jX) =
3
Y p(yjy<td ; X);
td=1
(1)
Both encoder and decoder are recurrent neural networks3 with Long Short Term
Memory (LSTM) cells. We apply the attention mechanism that forces the model
to learn to focus on speci c parts of the input sequence when decoding, instead
3 We use tf.contrib.seq2seq.sequence loss which is a weighted cross-entropy loss
for a sequence of logits. We concatenate the last hidden output of forward and
backward LSTM networks, the concatenated vector comes with xed dimensions
of relying only on the last hidden state of the encoder. Furthermore, in order
to capture the semantics of the entities and relations within our training data,
we apply domain speci c resources[
Data Sets We ran experiments on two public datasets NYT4[
Text Berlin is the capital city of Germany.
Triple dbr:Germany dbo:capital dbr:Berlin
Evaluation Metrics We consider pipeline-based approaches that combine
Entity Linking (EL) and Relation Classi cation (RC) as state of the art. We
propose several baselines with combined outputs from state-of-the-art EL7 and RC
for evaluation. We use F1-measure to evaluate triple generation (an output is
considered correct only if s-p-o are all correct) in comparison with the baselines.
Baselines We implement multiple baselines including a classical supervised
learning using simple Lexical features, a state-of-the-art recurrent neural
network (RNN) approach with LSTM [
We achieve the best F1 Measure of 84.3 on the triple generation from Table 2. Note that the baseline approaches that we implemented are pipeline-based, and thus they are very likely to propagate errors to downstream components. However, our model merges the two di erent tasks of EL and RC into one during the 4 New York Times articles: https://github.com/shanzhenren/CoType 5 Adverse drug events: https://sites.google.com/site/adecorpus 6 http://deepdive.stanford.edu/distant_supervision 7 Stanford, Domain speci c NER decoding, which composes a major advantage over pipeline-based approaches that usually apply separate models on EL and RC. The most common errors are caused by Out of vocabulary and Noise from overlapping relations in text. As we do not cover all rare entity names or consider multiple triple situations, these errors are valid in some sense. F1-Measure F1-Measure F1-Measure 36.8 58.7 59.8 64.2 71.4
We present an end-end system for translating a natural language sentence to its triple representation. Our system performs competitively on three di erent datasets and our assumption on enhancing the model with pre-trained KG embeddings improves performance across the board. It is easy to replicate our work and use our system following the demonstration. In the future, we plan to redesign the decoder and enable the generation of multiple triples per sentence. Acknowledgement This work was partially supported by the NIEHS Award 0255-0236-4609 / 1U2CES026555-01.