A massive amount of news is being shared online by individuals and news agencies, making it dificult to take advantage of these news and analyse them in traditional ways. In view of this, there is an urgent need to use recent technologies to analyse all news relevant information that is being shared in natural language and convert it into forms that can be more easily and precisely processed by computers. Knowledge Graphs (KGs) ofer ofer a good solution for such processing. Natural Language Processing (NLP) ofers the possibility for mining and lifting natural language texts to knowledge graphs allowing to exploit its semantic capabilities, facilitating new possibilities for news analysis and understanding. However, the current available techniques are still away from perfect. Many approaches and frameworks have been proposed to track and analyse news in the last few years. The shortcomings of those systems are that they are static and not updateable, are not designed for largescale data volumes, did not support real-time processing, dealt with limited data resources, used traditional lifting pipelines and supported limited tasks, or have neglected the use of knowledge graphs to represent news into a computer-processable form. Therefore, there is a need to better support lifting natural language into a KG. With the continuous development of NLP techniques, the design of new dynamic NLP lifters that can cope with all the previous shortcomings is required. This paper introduces a general NLP lifting architecture for automatically lifting and processing news reports in real-time based on the recent development of the NLP methods.
about news being shared on the web and social media
networks. JKPs have become crucial for press
indusFor several years we have seen how the traditional try. Yet, many works have proposed to process the
news press has moved to online content and new news texts in many diferent ways in order to apply
online press has appeared, publishing more online diferent JKP processes.
content than ever. Social networks enhanced that Our group have been developing a series of JKP
prophenomenon facilitating real-time interactions and totypes called News Hunter [
This paper presents an NLP lifting architecture Diferently from previous works, our introduced
component of the Journalistic Knowledge Platforms NLP subsystem’s architecture for News Hunter aims
(JKP) for lifting natural language news text into knowl- to lift all processed news into a semantic knowledge
edge graphs. JKP is a system intended for analysing, graph in real-time. Moreover, two Natural Language
lifting, and representing news using knowledge graphs Processing (NLP) lifting tracks could be chosen: the
to support journalists exploiting knowledge from and traditional pipeline and the end-to-end which
follows the state-of-the-art (SOTA) development of deep
Proceedings of the CIKM 2020 Workshops,
October 19-20, Galway, Ireland. neural network. That would avoid some limitations
email: Tareq.Al-Moslmi@uib.no (T. Al-Moslmi); reported in previous lifting tasks [
The NEWS project [
The process of recognizing events is a relevant fea- Many approaches for lifting natural language to
ture of such systems, which is approached in many knowledge graphs are based on previous-generation
diferent ways. For example, Event Registry [12] uses NER techniques, and new lifting approaches that add
clustering algorithms to detect and group similar ar- disambiguation and linking to recent best-of-breed
ticles which represent the same event. Following the NE recognisers are needed . There is also a lack of
central idea of events, NewsReader project [13, 18] standards for comparing lifting approaches[
The NewsReader NLP pipeline performs language recognition-disambiguation-linking pipeline. For
exspecific NER and NEL, event and semantic role
de1https://www.exfo.com/en/ontology/
2www.docker.com
or multiple sources of interest. Due to the high
amount of news items and their velocity of
production, the harvested items are represented using
standard lightweight formats like JSON, in order to
facilitate its parsing, execution, transfer, sharing between
Figure 1: News Hunter architecture [
News items can be collected from multiple sources: it into a proper semantic format that will then be inonline news (e.g., RSS feeds), social media (e.g., Face- serted to the KG. book, Twitter), archives or daily produced in-house news (e.g., agency daily news activity). The news crawler is oriented to harvest news from any source 3https://cassandra.apache.org 4https://hbase.apache.org 5https://kafka.apache.org 6https://www.rabbitmq.com
ing, and structural parsing. Recent works indicate that
robust lifting systems require accurate tuning of
sevThis section describes the NLP lifter for news natural eral steps, especially tokenization and semantic
similanguage texts to knowledge graphs. The NLP lifter larity [21]. Recently, deep neural networks, especially
which is a component of the JKP architecture consists end-to-end methods, have reduced the need for
preof the main NLP lifting tasks as well as some addi- processing steps. Moreover, using deep neural
nettional related tasks. Diferently from others proposed works for pre-processing tasks such as tokenization
systems, our proposed NLP lifter is docker-based and has recently produced promising results [22]. The
procontains the most possible tasks (traditional and re- posed NLP lifter could include as many pre-processing
cently developed ones) as shown in Figure 2. This steps as possible, which will be in separate dockers, so
allow the development of the platform and ensure us- the user can choose all suitable ones for the target data.
ing the most recent technology all the time. There will
be two main NLP tracks: the traditional pipeline that 4.2. Named entity recognition
is updated by recent technologies and the end-to-end
track which is the SOTA in many tasks. In addition, Named entity recognition is the task that identifies
there is the ensemble service that could combine more the named entities contained in the text like
perthan one lifter to produce better results. The purpose sons, locations, organizations, time, date, money, etc.
and advantage of this is that the user can choose to NER approaches could be categorised into three main
use the most suitable track for his case and data as groups: knowledge-based approaches, learning-based
well as the most recent techniques. In the traditional methods, and feature-inferring neural network
methpipeline the tasks like NER, NED, and NEL are imple- ods. Despite the existence of recent SOTA NER
remented separately and mostly using the of-the-shelf sults (especially recent deep NN approaches) such
software. The of-the-shelf systems are usually based as [23, 24, 25, 26], these approaches have not been
ution old approaches and their performance is not the lized and exploited in the process of lifting natural
lanSOTA. Moreover, traditional lifting methods neglect guage to knowledge graphs as mentioned earlier. This
the relations between entity types and entity context. paper aims to implement those SOTA NER methods in
However, there will be a possibility in our introduced docker-based components to tackle this shortcoming.
architecture to ensure the using of the most updated
ones or using newest systems by just replacing or 4.3. Named entity linking
adding their dockers to the related component. The
news item annotation ontology that has already been
designed by[
tifier of its corresponding entity that is described in a KB in the LOD cloud. Our paper has defined NEL as a wider task that includes NED as one of its processes.
Many NEL approaches are utilizing of-the-shelf systems for NER task. It is, however, a challenging task to choose which particular model to use for those systems. That is because it requires to estimate the similarity level between the system’s training datasets and the dataset that needs to be processed in which we strive to accurately recognize entities, according to [27]. Most recent SOTA systems on AIDA-CoNLL 4.1. Pre-processing dataset includes [28, 29, 30, 31]. There is no perfect NEL model for all datasets and one model might be the The quality of the data plays a key role in determin- best on one dataset but perform poorly on others. Acing the suitable pre-processing techniques. Since we cordingly, having the top N best SOTA implemented are dealing with the real-time streaming, the cleaning in dockers will allow the user to pick the most suitable and normalization are required to remove unnecessary model for his data and/or replace or update them at or noisy terms (like ASCII codes, currency symbols, any time when needed. hashtags, and so forth). The most frequently used preprocessing techniques are tokenization and POS tagging [19, 20]. Other common steps are sentence splitting, lemmatisation, chunking and dependency pars4.4. End-to-end track named entities and concepts) and reported the SOTA results. Similar to previous components, the proposed lifter will implement those methods and include them as optional tasks as many others for the user.
The majority of previous studies were mostly
assuming the availability of mentions and entities and
focused on the disambiguation process only. However,
leveraging mutual dependency between mentions and
their entities is neglected. Moreover, it is not a practi- 4.6. User-oriented tasks
cal idea in a real-world application. Diferent from that User-oriented tasks include those tasks specific and
and to overcome those shortcomings, end-to-end deals personalised for the project where the NLP lifting
with row text and aims to extract all mentions and link architecture is implemented. Apart from including
them to their entities in the knowledge base. End-to- SOTA NLP tasks like the previously described, the
end entity linking has been recently proposed and is NLP lifting architecture takes into account purpose
receiving increasing attention. Few studies have been specific tasks such as news angles detection, event
published which claiming the application of the end- detection, IPTC media codes annotation, rumours
deto-end approach [
Moreover, the constructed knowledge graph could Acknowledgments be used to implement more operations like question answering, knowledge graph-based sentence autocompletion, storytelling, fact-checking and so forth using semantic news analysis.
Lifting high-volume streams of news texts involves representing their content in machine-understandable formats. KGs is one such formats that has received much attention recently. NLP lifters are an important prerequisite for making the abundance of natural language news on the internet available as computerprocessable knowledge graphs. Thus, the presented NLP lifting pipeline provides with an structured and formalised process for transforming natural language text into computer-processable knowledge graphs.
The presented pipeline can incorporate any NLP technique like traditional or end-to-end approaches and combining its results or expand them with specificpurpose NLP method like sentiment analysis. Moreover, the introduced NLP lifter is designed to simplify its components replaceability by making use of docker technology, facilitating e.g., the update of all tasks and methods to SOTA approaches. Although the proposed JKP is designed mainly to help journalists, it could be used and customized for the public. The presented NLP lifting architecture aims to be used as reference for developers and researchers of JKP interested in real-time NEL. News organisations may need to adapt their systems, replace components, add new SOTA technologies, or integrate it with other JKP, thus having such NLP lifting pipeline as reference facilitate its management and understanding. Furthermore, it is not restricted to news text and could be used to lift other types of texts.
In our future work, we plan to validate the results
of our proposed NLP lifter by using both a manually
collected and annotated corpus of news and
goldstandards, and compare the results of our proposed
lifter with current NEL systems such as ADEL, SpaCy
lifter, NewsReader, Stanford CoreNLP or DBpedia
Spotlight. Besides, we want to explore the
possibilities that validations tools like GERBIL [
Event registry: learning about world events from 168–176. news, in: Proceedings of the 23rd WWW, ACM, [22] T. Boros, S. D. Dumitrescu, R. Burtica, NLP-cube: 2014, pp. 107–110. End-to-end raw text processing with neural net[13] P. Vossen, R. Agerri, I. Aldabe, A. Cybulska, works, in: Proceedings of the CoNLL 2018, ACL, M. van Erp, A. Fokkens, E. Laparra, A.-L. Mi- 2018, pp. 171–179. nard, A. P. Aprosio, G. Rigau, M. Rospocher, [23] A. Baevski, S. Edunov, Y. Liu, L. Zettlemoyer, R. Segers, Newsreader: Using knowledge re- M. Auli, Cloze-driven pretraining of selfsources in a cross-lingual reading machine to attention networks, in: Proceedings of the 2019 generate more knowledge from massive streams Conference on EMNLP and the 9th IJCNLP, ACL, of news, Knowledge-Based Systems 110 (2016) 2019, pp. 5359–5368.
60 – 85. [24] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, [14] M. Kattenberg, Z. Beloki, A. Soroa, X. Artola, Bert: Pre-training of deep bidirectional transA. Fokkens, P. Huygen, K. Verstoep, Two formers for language understanding, 2018. architectures for parallel processing of huge arXiv:1810.04805. amounts of text, in: Proceedings of the Tenth [25] M. Peters, M. Neumann, M. Iyyer, M. Gardner, LREC’16), European Language Resources Asso- C. Clark, K. Lee, L. Zettlemoyer, Deep contextuciation (ELRA), 2016, pp. 4513–4519. alized word representations, in: Proceedings of [15] U. Germann, P. v. d. Kreeft, G. Barzdins, A. Birch, the 2018 Conference of the NAACL, ACL, 2018, The summa platform: Scalable understanding of pp. 2227–2237. multilingual media, in: Proceedings of the 21st [26] L. Liu, X. Ren, J. Shang, X. Gu, J. Peng, J. Han, EfAnnual Conference of the European Association ifcient contextualized representation: Language for Machine Translation, 2018. model pruning for sequence labeling, in: Pro[16] C. Rudnik, T. Ehrhart, O. Ferret, D. Teyssou, ceedings of the 2018 Conference on EMNLP, R. Troncy, X. Tannier, Searching news articles ACL, 2018, pp. 1215–1225. using an event knowledge graph leveraged by [27] J. Plu, G. Rizzo, R. Troncy, Enhancing entity Wikidata, in: 30th WWW Conference, 13-17 May linking by combining ner models, in: H. Sack, 2019, 2019. S. Dietze, A. Tordai, C. Lange (Eds.), Semantic [17] N. Fernández, J. M. Blázquez, J. A. Fisteus, Web Challenges, Springer International PublishL. Sánchez, M. Sintek, A. Bernardi, M. Fuentes, ing, Cham, 2016, pp. 17–32.
A. Marrara, Z. Ben-Asher, News: Bringing [28] J. Raiman, O. Raiman, Deeptype: Multilingual semantic web technologies into news agencies, entity linking by neural type system evolution, in: I. Cruz, S. Decker, D. Allemang, C. Preist, 2018. arXiv:1802.01021.
D. Schwabe, P. Mika, M. Uschold, L. M. Aroyo [29] I. Yamada, H. Shindo, Pre-training of deep (Eds.), The Semantic Web - ISWC 2006, Springer contextualized embeddings of words and enBerlin Heidelberg, Berlin, Heidelberg, 2006, pp. tities for named entity disambiguation, 2019. 778–791. arXiv:1909.00426. [18] M. Rospocher, M. van Erp, P. Vossen, A. Fokkens, [30] Z. Fang, Y. Cao, Q. Li, D. Zhang, Z. Zhang, I. Aldabe, G. Rigau, A. Soroa, T. Ploeger, T. Bo- Y. Liu, Joint entity linking with deep reinforcegaard, Building event-centric knowledge graphs ment learning, in: The WWW Conference, from news, Journal of Web Semantics 37-38 WWW ’19, ACM, 2019, p. 438–447. (2016) 132 – 151. [31] A. Luo, S. Gao, Y. Xu, Deep semantic match [19] G. Zhu, C. A. Iglesias, Exploiting semantic simi- model for entity linking using knowledge graph larity for named entity disambiguation in knowl- and text, Procedia Computer Science 129 (2018) edge graphs, Expert Systems with Applications 110 – 114. 2017 International Conference on 101 (2018) 8 – 24. Identification, Information and Knowledge in the [20] M. Fossati, E. Dorigatti, C. Giuliano, N-ary rela- Internet of Things.
tion extraction for simultaneous t-box and a-box [32] A. Moro, A. Raganato, R. Navigli, Entity linkknowledge base augmentation, Semantic Web 9 ing meets word sense disambiguation: a unified (2018) 413–439. approach, Transactions of the ACL 2 (2014) 231– [21] M. Conover, M. Hayes, S. Blackburn, P. Sko- 244. arXiv:10.1162/tacl_a_00179.