Temporal information provides important and precise indications on facts and events. Yet, when automatically processing unstructured documents, it may be complicated to extract temporality-related information. Besides, such tools are not available for many languages. We propose to adapt an existing tool for the automatic detection and annotation of temporal information in Ukrainian. The tool is rule-based. It permits to detects temporal expressions, be they absolute or relative, and to normalize them. We test the adapted tool on two corpora from different genres and evaluate the results.
Unstructured documents are the most common source of information, and they may
represent the majority of information available on a particular question and domain.
For instance, in the biomedical area, several documents are unstructured, such as
clinical discharge summaries, scientific literature, patient brochures, informed
consents, or clinical trial protocols. The situation is similar in other areas (law, energy,
economics, politics, history, etc.). Hence, when working with unstructured narrative
texts, the process is very demanding on automatic methods for detecting, extracting,
formalizing and organizing information contained in these documents. Information
extraction (IE), which is part of Natural Language Processing (NLP), proposes such
methods and aims at detecting and extracting relevant pieces of information from
textual data. Different types of information can be searched, such as (1) entities and
events, which are traditionally detected thanks to the exploitation of terminological
resources and thesauri, when available. The process is dedicated to the recognition of
terms and is a very challenging issue related to the computing of variants of these
terms in documents [
In our work, we propose to contribute to this research and to concentrate on the description of temporal information and on its automatic detection and annotation in Ukrainian. This implies that we have to design suitable methods, resources and tools for this language.
In what follows, we first present some related work (Sec. 2). We then precise our objectives (Sec. 3), introduce the material used (Sec. 4) and the proposed method (Sec. 5). Our results and their discussion are presented in Section 6. Finally, we conclude with some directions for future work (Sec. 7). 2
Work on temporal information relies on three important steps when processing
unstructured narrative documents: identification of linguistic expressions that are
indicative of the temporality and their normalization [
TimeML1[
TimeML offers the possibility to encode several types of temporal information and expressions (i.e. TIMEX3 tags): 1. Expressions of dates, time, durations or sets (attribute types). Dates and time are represented according to the ISO-8601 norm. Examples below present these types of temporal information: (1) Корабель Аполлон-11 стартував 16 липня 1969 о 13 годинi 32 хвилини за 1 http://www.timeml.org 2 http://timexportal.wikidot.com/ Грiнвiчем. (The Apollo-11 ship took off at 1:32 pm GMT on 7/16/1969.) (date and time) (2) Протягом трьох годин, поки налагоджували зв’язок iз Москвою, Гагарiн давав iнтерв’ю i фотографувався. (During three hours, while establishing communication with Moscow, Gagarin was interviewed and photographed.) (duration) (3) Корейська вiйна – збройний конфлiкт мiж Корейською НародноДемократичною Республiкою та Пiвденною Кореєю, який тривав з 25 червня 1950 року до 27 липня 1953 р. (Korean war is an armed conflict between Democratic People's Republic of Korea and South Korea, which lasted from 25th of June 1950 up to 27th of July 1953.) (duration) (4) В екваторiальному та тропiчному поясi припливи i вiдпливи здебільшого повторюються двiчi на добу. (In the equatorial and tropical areas, high and low tides mostly occur twice a day.) (set) (5) Тривали 118 рокiв, з примиренням. (Lasted for 118 years, including armistices.) (duration) (6) До середини 260-х до н.е. Римська республіка остаточно пiдпорядкувала собі Апеннiнський пiвострiв. (By the mid of 260 BC, the Roman Republic had gained control of the Italian peninsula.) (date) (7) Основним джерелом з історії греко-перських воєн є «Iсторiя» Геродота, що містить опис подiй до 478 до н.е. включно. ("The Histories" by Herodotus, which contains description of events up to 478 BC, is the main source on history of the Greco-Persian Wars.) (date) 2. ISO-normalized forms of the expressions (attribute value), such as in (from examples above): ─ 16 липня 1969 о 13 годинi 32 хвилини 1969-07-16T 13:32:00 ─ трьох годин P3H ─ двiчi на добу P1D 3. Quantity and frequency of the set expressions (attributes quant or freq), such as in this expression of frequency: ─ двiчi на добу 2X 4. Begin and end anchors for durations (beginpoint and endpoint attributes). For instance, in Example (3), the begin anchor is 25th of June 1950 and the end anchor is 27th of July 1953. The implicit duration is 3 years, 1 month and 2 days, which is normalized in P3Y1M2D. 5. Temporal modifiers, which have been introduced in order to annotate changed or clarified temporal expressions. For instance, in Example (6), the date 260 до н.е. is changed by середини, which is the date modifier attribute MID.
In addition to the annotation of temporal expressions, TimeML also allows to describe events as well as relations between temporal expressions and/or events. In this paper, we only focus on the annotation of temporal expressions (TIMEX3) related to dates and durations. Description and detection of other temporal information will addressed in later work.
The purpose of our work is to automatically detect and annotate temporal expressions
in corpora in Ukrainian language. We aim particularly at the description of dates and
durations, such as in Examples (1)-(3) and (5)-(7). During a preliminary study, we
tested several existing systems for identification of temporal expressions and found
out that HeidelTime [
We use two types of texts from two different genres: newspaper and encyclopedic articles. Both of them have the potential to contain temporal information. 4.1
Newspaper articles are obtained from the online news journal from Ukraine Українська правда3 (Ukrainian truth). This journal covers any news related to the events which happen in Ukraine and also to the events which happen in other places but which may be important to Ukraine. The journal has been founded in 2010 and shows good popularity and objectivity. The main interest in using this type of articles is that they typically contain dates associated to events. We use 40 articles (over 31,000 word occurrences) for the development of the system and 40 articles (over 35,000 word occurrences) for its tests and evaluation. 4.2
Encyclopedic articles are obtained from the Wikipedia resource4, which is a free and
collaborative resource. This encyclopedia contains information on a great variety of
topics. We have chosen to work with the articles related to wars, as part of the
WikiWars corpus5 [
4 https://uk.wikipedia.org 5 http://timexportal.wikidot.com/wikiwars articles (66,479 word occurrences) for the evaluation of the automatic system. 5
The methods are composed of several steps: pre-processing of texts, adaptation of HeidelTime to Ukrainian, and evaluation of the automatic annotations. 5.1
All source documents are in the html format because they are obtained from online resources. The documents are converted in the text format. The characters are encoded with the UTF-8 characterset. 5.2
HeidelTime is a cross-domain temporal tagger that extracts temporal expressions from
documents and normalizes them according to the TIMEX3 annotation standard, which
is part of the markup language TimeML [
Hence, adaptation of the HeidelTime resources to Ukrainian is the main step of the current work. The detection and normalization of temporal information by HeidelTime relies on three kinds of resources: ─ linguistic patterns, which describe linguistic elements of the temporality (days of the week, months, numbers, etc.). This type of resources is used for the detection of temporality in texts; ─ normalization resources, which are created to permit the normalization of the detected elements. In this way, all the detected units are normalized. Thanks to these resources, normalization can be performed for absolute (Example (8)) and relative (Example (9)) dates, durations and sets. Thus, the normalized values of Examples (8) and (9) are 2015-05-07 and 2017-05-09, respectively if we consider that these two dates are related; ─ rules for composing more sophisticated detection of temporality, such as periods,
intervals and specific expressions. (8) 7 травня 2015 року. (May 7th, 2015.) (9) Через два днi. (Two days later.) 5.3
A subset of the corpus (22 newspaper articles) is used for the development and tuning
of HeidelTime. The rest of the corpus is used for the evaluation. For all the processed
document, we used the default parameters, namely no postagger and the type of
document are considered as narrative which leads to solve relative dates regarding the
previous absolute date. The results generated are evaluated manually with two
classical evaluation measures [
In Table 1, we present the evaluation results obtained on the two processed corpora. The results are indicated in terms of true positives TP and precision P. We also indicate the total number of temporal expressions occurring in each corpus (total).
The system was adapted to Ukrainian on a subset of newspaper articles. We can
see that the results obtained on the two subsets of newspaper corpus are comparable
and close to 90% precision. This is a very good performance for the first version of
the system. Besides, when working with newspaper articles, both detection and
normalization show good results. Transposition of the system on another genre,
encyclopedia articles, permits to test the same system on different data. As we can see, the
results are slightly lower, especially for the normalization. For the detection, we keep
the precision values high (86%), while the normalization process is more complicated:
it shows 78% precision. By comparison with similar work in other languages [
Examples below illustrate successful annotation of temporal values and of their normalization. In these examples, the TIMEX3 values are annotated in the XML format, which is the native format of HeidelTime. We present here examples for dates and durations and explain them. Our main interest is to show the results obtained when normalizing relative, ambiguous or imprecise expressions: ─ <TIMEX3 type = "DATE" value = "2017-03-01"> 1 березня 2017 </TIMEX3> ... У <TIMEX3 type = "DATE" value = "2016-09"> вереснi </TIMEX3> керiвник Спецiалiзованої антикорупцiйної прокуратури Назар Холодницький заявив, що Чаус перебуває в анексованому Росiєю Криму...<TIMEX3 type = "DATE" value = "2016-11-11"> 11 листопада </TIMEX3> Iнтерпол оголосив Чауса в мiжнародний розшук. In this example, the starting date 2017-03-01 is first recorded by the system. Then, the next two dates (2016-09 and 2016-11-11) are positioned during the previous year, which is correct. This example illustrates the possibility of the system to disambiguate the chronology of event seven if the year soft he events are not indicated precisely. ─ S&P прогнозує зростання ВВП Українина 1,9% <TIMEX3 type = "DATE" value = "2017"> цього року </TIMEX3>. Similarly to previous example, the system records that the current year is 2017 and can disambiguate expression цього року (this year) through its correct normalization. ─ <TIMEX3 type = "DATE" value = "XXXX-XX-XX"> Середа </TIMEX3>, <TIMEX3 type = "DATE" value = "2017-03-08"> 8 березня 2017 </TIMEX3>. ...Про це повiдомив генпрокурор Юрiй Луценко у <TIMEX3 type = "DATE" value = "2017-03-08"> середу </TIMEX3> в Facebook. In this example, the system can normalize the date by referring it to the day of week Середа (Wednesday). ─ S&P: за <TIMEX3 type = "DURATION" value = "P3Y"> три роки </TIMEX3> Україна повинна вiддати 20 мiльярдiв доларiв боргiв. This sentence provides an example of the detection and normalization of durations. ─ запобiжний захiд у виглядi арешту на <TIMEX3 type = "DURATION" value = "P60D"> 60 дiб </TIMEX3>. This is another example of the detection and normalization of durations.
We have also found several cases in which the system is not successful, such as those presented below: ─ радник президента США з нацiональної безпеки Майкл Флiнн подав у вiдставку <TIMEX3 type = "TIME" value = "2017-02-24TEV"> ввечерi </TIMEX3> <TIMEX3 type = "DATE" value = "2017-02-13"> 13 лютого </TIMEX3>. This is a typical example of temporal expressions in which the normalization process may fail. Here, the right normalization value is 2017-02-13. Yet, for the normalization of the first part of the expression ввечерi (in the evening), the systems exploits the last date recorded, which is 2017-02-24. Such temporal expressions are very frequent in the encyclopedia corpus since several military actions are happening in the evening. For mending this kind of errors, more sophisticated patterns and rules will be created. ─ У <TIMEX3 type = "TIME" value = "1776-12-07TNI"> нiч </TIMEX3> з 25 на <TIMEX3 type = "DATE" value = "1776-12-26"> 26 грудня </TIMEX3> крадькома перетнув Делавер I розбив британський загiн у битвi бiля Трентона, захопивши майже 1000 здивованих I неукрiплених гессенських найманцiв. This example is similar to the previous example, but the situation is more complicated because we have in addition the interval of dates з 25 на 26 грудня (from December 25th to 26th). As we can see, currently we did not fit the system to the detection of intervals and usually only the last date is detected. This is the main source of current errors as the intervals are very frequent in presentation of historical and political information. They frequently occur in both corpora processed. Description and encoding of intervals will be added to the system in future. ─ Уряд планує додати до пенсiй вiд 200 до <TIMEX3 type = "DATE" value = "1000"> 1000 </TIMEX3> гривень. Here, the system detects wrong information because of the preposition до (up to), which can also have temporal meaning, followed by numbers 1000, yet not related to the temporality. This kind of errors can be reduced with the creation of specific exception patterns and rules. ─ У спробах полегшити тиск з <TIMEX3 type = "TIME" value = "1967-06-12T 24:00"> пiвночi </TIMEX3>, <TIMEX3 type = "DATE" value = "1967-08-09"> 9 серпня </TIMEX3> мобiльна бригада армiї Бiафри у складi 3000 осiб за пiдтримки артилерiї та бронемашин переправилася на захiдний берег Нiгера. In this example, we can see that the word пiвночi is ambiguous as it can mean midnight and north. This temporality marker causes several errors in the processed corpora. Its disambiguation will need additional analysis of texts, as pre-processing or post-processing step.
These are certainly the main detection and normalization errors which we can find in the corpora processed. Another difficulty which we currently face is related to very specific temporal expressions. The system does not take them into account, which causes several silences (false negatives) in the output. They will be described and encoded in our future work. Here are some example: ─ Relative temporal expressions like в той же день, цього ж дня (the same day, that day); ─ Specific temporal expressions like За минулий з тих пiр мiсяць (during the month that passed since); ─ Specific forms for expressing the temporality (combination of numbers and characters) like 21-го столiття, на початку 1990-х рокiв (XXI century, at the beginning of 1990s); ─ Specific calendars like the one introduced during the French revolution. На вимогу Робесп’єра 14 фрiмера другого року (<TIMEX3 type = "DATE" value ="179312-04"> 4 грудня 1793 </TIMEX3>) був органiзований уряд iз винятковими повноваженнями. In this example, the system correctly detects and normalizes 4 грудня 1793, but is not sensible to 14 фрiмера другого року (14 of Frimaire of the second year). For such cases, additional resources must be created, so that their detection, conversion and normalization become possible. 7
We presented our work on creation of automatic system for the detection and normalization of temporal information in Ukrainian. In information extraction applications, temporal information is indeed important. For performing this task, we proposed to use an existing tool HeidelTime. This system has undergone automatic adaptation of its resources to Ukrainian but this showed to be not efficient: the system produced no results with these resources. Hence, the main purpose of our work was to create the suitable resources.
We used two corpora from two genres (newspaper and encyclopedia). 22 newspaper articles were used to develop the system. The rest of our data was used for the tests. Encyclopedia articles correspond to the WikiWars project. The Ukrainian WikiWars corpus has been compiled for our study. The evaluation of the system shows that up to 90% of temporal units in newspaper articles are detected and normalized correctly. In encyclopedia articles, the detection shows 86% precision and the normalization 78% precision. We also present and discuss some examples of the current failures of the system.
In future, the system will be further developed and fitted to Ukrainian, so that it detects and normalizes other temporal expressions. It will be made freely available to the research community. Besides, the two used corpora will be fully annotated with temporal expressions and also made freely available to the research community.