=Paper=
{{Paper
|id=Vol-2309/05
|storemode=property
|title=Events in the legal domain: first impressions
|pdfUrl=https://ceur-ws.org/Vol-2309/05.pdf
|volume=Vol-2309
|authors=María Navas-Loro,Cristiana Santos
|dblpUrl=https://dblp.org/rec/conf/jurix/Navas-LoroS18
}}
==Events in the legal domain: first impressions==
https://ceur-ws.org/Vol-2309/05.pdf
Events in the legal domain: first impressions
Marı́a Navas-Loro1[0000−0003−1011−5023] , Cristiana Santos2
1
Ontology Engineering Group
Universidad Politécnica de Madrid, Madrid, España,
mnavas@fi.upm.es,
http://marianavas.oeg-upm.net/
2
University of Toulouse
Toulouse, France,
cristiana.teixeirasantos@gmail.com
Abstract. Dealing with events is a challenging task in Artificial Intel-
ligence; just trying to define what is an event and deciding what in-
formation should be considered relevant to it is a difficult and domain
dependent endeavour. Despite this task has been already tackled in the
legal domain, no consensual definition nor standard representation for
legal events has been established yet. In this paper, we analyze current
approaches to event representation and extraction in the legal domain,
and we review generic approaches as well. We expose our first impres-
sions derived therefrom, and we offer a first round of event annotations
of judgments of the European Court of Justice. Aspects such as the con-
cept of relevance are discussed, along with choices for evaluation. Possible
strategies for extraction of legal events are described. Finally, a roadmap
for a formal, complete definition and delimitation of events in the legal
domain is presented.
Keywords: legal events, court decisions, relevance, European Court of
Justice
1 Introduction
Extracting legal relevant events from a sheer scale of legal corpora is challenging
because of the fine grained information contained therein. For example, case-
law3 can be long (and rambling) and information related to important legal
events of the case can be scattered. Moreover, there is no settled way a factual
element can be depicted, and the information is not complete as presented, in
the sense that inferences must be made to determine a definite value. By locating
3
Research on legal information-seeking behavior [1] and legal information retrieval
[2] posit that lawyers search for useful legal documents that are relevant for their
task at hand, to enable decision-making or problem solving (“situational relevance”).
Haruna et al. [3] assert that the most common information needs that lawyers have
refer to the latest higher court decisions and the most recent legislation. Therefore,
in this work we refer to the judgments of the European Court of Justice (ECJ),
identifying therein the legal events.
45
textual regions likely to compact relevant legal events will avoid incurring the
expense of reading entire texts. Also, common event-based constituents [4] eases
the detection of events: the legal history of a claim (what give raise to the dispute
and occurrence of events time-line, among others); parties (who are the people
involved in the dispute); final decision, etc. Perusal and a defined consensual
definition of legal events will enable further harvesting thereof.
Events can also be found in other kind of legal documents, such as contracts
and regulations, so they could be used, for instance, to check compliance among
different jurisdictions based on the information surrounding a same event.
In this paper, we describe the preliminary steps we have taken towards ex-
traction of legal events. We elaborate on the qualification and scope of legal
events, having as a use-case the ECJ judgments. The research question of this
paper consists in identifying which linguistic and legal information is required
to reliably identify legal events.
The remainder of the paper is organized as follows. Section 2 reviews different
proposals in the literature both for extraction and representation, from generic
to legal specific ones. Section 3 analyses how the previous approaches addressed
the evaluation of event extraction. Section 4 further elaborates on the varying
criteria to which legal events are bounded to, and introduces our first impressions
on the task. Finally, Section 5 presents the conclusions and the future work.
2 Concept of event and previous approaches
For the tasks of event extraction and representation, different approaches have
been proposed in the literature. Most of them provide for their own definition
and formalization of the concept of event. In this section we will review different
proposals, both for extraction and representation, from generic to legal specific
ones.
2.1 General approaches
Regarding representation, there are several ontologies available dealing with
events. The W3C Time Ontology4 , focusing on temporal expressions, is usually
combined with the PROV Ontology5 to represent events stressing their temporal
information. The Event Ontology of Yves Raymond6 allows representing a basic
concept of event, considering just as surrounding information where it happens,
when it happens, and who intervenes in that event, as well as the factor and the
product of it. Also sub events can be declared. Similarly, the Simple Event Model
(SEM)7 also deals with that information, but adding concepts such as Role, Au-
thority or EventType; SEM has also been later expanded8 to its use in EventKG
4
https://www.w3.org/TR/owl-time/
5
https://www.w3.org/TR/prov-o/
6
http://motools.sourceforge.net/event/event.html
7
https://semanticweb.cs.vu.nl/2009/11/sem/
8
http://eventkg.l3s.uni-hannover.de/schema model.html
46
[5], a event-centric knowledge graph. Besides ontologies, frames, scripts or case
frames [6] are also used to represent events; Ontology Design Patterns9 suggest
how to represent events for different kind of situations.
Different annotation schemas are also often used for event extraction. One of
the most used mark up language in the temporal domain is TimeML10 . TimeML
allows to annotate temporal expressions (such as dates or times), events and
temporal relations among them. TimeML events can be of several types (like
reporting, aspectual or state); by using the tag MAKEINSTANCE, several in-
stances can be linked to a given event and also attributes like polarity or aspect
can be expressed. Also the ACE model [7] has been widely-used in previous liter-
ature, and has its own definition of event11 . The ACE project guidelines focus on
different types of events, among which we find JUSTICE, with its own subtypes
(namely, arrest-jail, release-parole, trial-hearing, charge-indict, sue, convict, sen-
tence, fine, execute, extradite, acquit, pardon and appeal ). Several studies were
conducted to compare these approaches [8, 9]. Finally, Chambers et al. [6, 10]
consider that verbs sharing coreferring arguments are semantically connected,
what they call narrative coherence; they use this information and Semantic Role
Labeling to learn new events in an unsupervised way.
Nevertheless, regarding extraction, most approaches develop their own defi-
nition and strategies. Hagege et al. [11] claim the difficulty of defining an event
from a concept perspective, so they decide to consider an event any verb (state or
action), any deverbal noun, any noun argument of the preposition during, or any
time span noun. Carpet et al. [12] developed their own ad hoc way to represent
events, consisting on some templates with the core of the event and some coor-
dinates (agents, other participants, places and time). The work of Hogenboom
et al. [13] gives an overview on different event extraction methods.
Existing proposals lack therefore of a common definition of an event, usually
using ad hoc representations for very specific type of event extraction. Annota-
tion formats are on the other hand too generic, and uneasy to adapt to the legal
domain needs. The ontologies available for representation are also very similar,
having as a basis the same information (what, who and where).
2.2 Approaches in the legal domain
Regarding the legal approaches to event extraction and representation, we have
divided previous literature in three different communities: Legal Information
Retrieval (LIR) proposals, Events in Legal Requirements Engineering and Events
in Legal Knowledge Representation. An overview of the different approaches
analyzed can be found below in Table 1. Therein we mention, in relation to each
paper or contribution, their objective, if whether and which there is a definition
of legal event, the analyzed corpus (if any), the methodology and the level of
implementation readiness.
9
http://ontologydesignpatterns.org/wiki/Community:Event Processing
10
http://www.timeml.org/
11
“An event is a specific occurrence involving participants. An Event is something that
happens. An Event can frequently be described as a change of state.”
47
Table 1: Different approaches to event extraction and representation in the legal domain.
Name Objective Event Corpus Methodology Implementation
Readiness
Legal Information Retrieval
Evaluation Applied Any eventuality (event, 18 sentences Use of SRL (Semantic Role Preliminary
of semantic event ex- state or attribute) re- from the Labelling). From the cor- study
events for traction for lated to expressions in Canadian pus, 150 events were re-
legal case Legal case legal texts, and by its Supreme viewed to compare with the
retrieval [14] retrieval compositionality. Can be court. The automatic Event extraction
decomposed and composed authors claim performed on 2 cases.
into great or lesser events. to develop a a
500,000 word
legal corpus
as future work
Event Ex- Legal case Events are viewed as tem- NLP semi automatic ap- Event-based
traction for building porally bounded objects proach to enable the use of abstrac-
Legal Case and rea- that have entities impor- entity related information tions: role-
Building and soning for tant within the application corresponding to the rela- based events,
Reasoning litigation domain (e.g. persons and tions among the key players interaction-
[15] organizations) as partici- of a case, extracted in the based, refer-
pants. form of events. ence events,
cognitive
events.
System to
support lit-
igation case
construction
Event Ex- Finding No proposed definition; in 300 docu- Use of patterns from le- Work in
traction from patterns essence is litigation-based ments down- gal text. The description progress, us-
Legal Doc- in Legal information loaded from of events usually follows ing human
uments in Language the Mexican a regular pattern involving evaluation
Spanish [16] in Mexican ‘Instituto at least two elements: the
Spanish Federal de- action, determined by the
in order Telecomu- main verb, and the date on
to extract nicaciones’ which the event occurred.
events in (IFT) An analysis was made of
writs of the verbs that occur in the
‘amparo’. writings of amparo, as well
as the direct objects of each
verb.
Cognitive Improve Legal events understood as Recognize possible legal Manual,
Linguistic Brazilian the cognitive connections event structures to be de- no imple-
Represen- courts in- that specialists make when scribed in legal documents; mentation.
tation of formation they are reading a legal use of semantic frames – Still work
Legal Events. retrieval document e.g. “Lawsuit frame” has in progress.
Towards a systems as participants and props Legal events
semantic- ‘Type of Action’ which in- are described
based legal dicates the type of lawsuit as semantic
information that was filled against a frames; they
retrieval [17] defendant (administrative, have identi-
criminal, familiar), ‘Au- fied 10 legal
thor’, who is the person frames
that goes to the court with
a request, ‘Defendant’, who
is the person that is been
suited, and ‘Concrete case’,
which is the legal base that
gives the author the right
to make a legal request.
Events in Legal Requirements Engineering
Automating Automatic The concept of event is not U.S. HIPAA Elicit “events” as one legal Automatic
the Extrac- extraction defined per se, but framed Privacy Rule concept, and also date and tool
tion of Rights of legal as “event, date and infor- and the information. Other concept
and Obli- require- mation”. They state that Italian ac- elicitation is observed, e.g.
gations for ments they ‘‘(...) found other terms cessibility cross-references, actors,
Regulatory from legal that we could generalize into a law policies.
Compliance text common, abstract type, includ-
[18] ing event, date, and informa-
tion. Thus, on the basis of the
definition section, we derived a
list of hyponyms for the basic
concepts: actor and policy as
well as event, date and informa-
tion.”
Nomos Automatic No consideration of “event” Nomos models are built Modelling
framework legal per se, but other core con- around 4 core concepts: language
[19] metadata cepts related to events (sit- roles (the holder of benefi- (Nomos 3)
extraction uations, roles) ciary of provisions); norms implemented
(either duties or rights); sit-
uations (describing the past,
actual or future state of the
word); and associations (de-
scribing how a provisions
affects a given situation).
48
Table 1: Different approaches to event extraction and representation in the legal domain.
Name Objective Event Corpus Methodology Implementation
Readiness
12
Legal Knowledge Representation
LKIF [20] Rule mod- Events are considered It includes At a phrase level, “events” Upper ontol-
eling lan- changes (“Changes events more than 200 are represented, and it ogy
guage occur against this canvas of classes further provides for the an-
temporal and spatial positions”) tecedents and consequences
of events. Other important
concepts herein are: actors,
objects, time, locations,
trades and transactions,
among others. Also state-
ments are classified into
facts and norms.
LegalRuleML For ex- It does not model events per The concept of event is On the way to
13 pressing se, but only temporal di- introduced at the level of be a OASIS
and in- mensions of the norms. phrases. Other concepts Standard
ferencing are: participants, time,
over legal locations, jurisdictions,
knowl- artifacts, and compliance.
edge. Participants may be de-
Gandon signed as agents, bearers
et al. [21] or third parties, who may
proposed have roles and be part of
an exten- an authority.
sion of
the Legal-
RuleML
that sup-
ports
modeling
of nor-
mative
rules.
Akoma Ntoso An XML Events are considered “Ac- It is an OASIS
14 markup tions and occurrences”, al- standard.
schema though they are not specif-
for de- ically targeted and are con-
scribing sidered “other concepts”15
legal re-
sources
of various
types, for
example,
laws, reg-
ulations
and court
decisions.
There has not been, at a representative scale, an instantiation of LKIF neither
of LegalRuleML. It was neither used for formalizing or annotating the content of
a legal corpora, either automatically or manually, to understand how events in
the legal domain are ascertained. In general, event in the legal domain is used as
a fine grained structure, and also combined with others (entity, relation, event,
temporal expression, value).
3 Evaluation
Since there are several interpretations and formalizations on what is an event,
evaluation is also a controverse issue. Different concepts imply different represen-
tations and attributes, needing therefore of different metrics to measure relevant
aspects. An example of this difference becomes evident when comparing TimeML
12
Some of these concepts have correspondences with RE literature on Legal Require-
ments
13
https://www.oasis-open.org/committees/tc home.php?wg abbrev=legalruleml
14
http://www.akomantoso.org
15
http://www.akomantoso.org/?page id=47
49
concept of event and the one by Ji et al [22]. While the first tackles events from
a pure temporal point of view, marking them up as temporal entities with at-
tributes and linking them just to other temporal expressions, the second focuses
in argument detection, centering its evaluation on Named Entities and tempo-
ral ordering. It must be noted that the latest deals with cross-document event
extraction, where common entities are a useful lead to connect related events
and ordering tends to be more tricky than in a single (and usually chronologi-
cal) document. In this section we will analyze the different evaluation methods
proposed in previous literature, trying to derive the one that fits the most to our
aim in the legal domain.
Time-related tasks in SemEval have been for many years the reference venue
where different temporal tools demonstrated their capabilities. While first edi-
tions of TempEval [23, 24] made use of the TimeML standard for dataset anno-
tation, evaluation consisted in using the figures Precision, Recall and F-measure
on the extent and the type of event (among the nine classes defined in TimeML
guidelines), and optionally other attributes (tense, aspect, polarity, and modal-
ity). Additionally, Temporal relations were also extracted in another task, and
could be considered additional information of an event, but not a part of its for-
malization. Nevertheless, in SemEval-2015 [25] the evaluation changed to tempo-
ral question-answering, prioritizing therefore temporal understanding over sim-
ple annotation in the text, where just differing in the span of the annotated
event led to an error. On the other side, ACE 2005 evaluation plan [26] proposes
the ACE value VDR (Event Detection and Recognition), a metric developed
considering the extent of the annotation, the arguments of the event and the
attributes value and modality.
Other proposals include several levels of evaluation. It is the case of BioNLP’09
shared task on event extraction16 , where different tasks tackled different levels
of detail of event extraction, from just identifying the so called core of the event
(namely, its trigger, type and main argument) to surrounding entities (such as
the location of the action) and its factuality (e.g., if it actually happened or is
negated or introduced as a mere possibility)
One of our main concerns was relevance. A judgment can contain thousands of
events in the broadest sense of the word, but not all of them are relevant from the
legal point of view. How could relevance be measured? Is it related to the agents
intervening in the event? By the appearance of this event in other judgments
from the same court? From the same legal field? Is it possible to extract a
common structure of most legal procedures in a court? A jurisdiction, a country?
Regarding relevance, some efforts have been previously made in literature to
distinguish important events [27], although no specific evaluation metrics on this
aspect were found. Just when these events were further used for an alternative
task (e.g., summarization), some evaluation was performed using metrics of the
final format [28].
16
http://www.nactem.ac.uk/tsujii/GENIA/SharedTask/
50
4 Our first analysis
In this section we introduce the results of a first study of events on the legal
domain. First, with some general observations on the concept of legal events;
then, with more specific considerations derived from the first annotation of a
corpus of judgments from the ECJ.
4.1 General Observations
Legal events (and the annotations thereof) may vary according to different cri-
teria contingent to the legal realm. We present possible differential criteria in a
general perspective:
Multi-jurisdictionality and multilinguism: legal events fluctuate according to
common law or civil law jurisdictions, and to the languages in which they are
expressed in or translated to.
Document-dependency: qualification of legal events can vary according to the
heterogeneity of the embedding document: legal/corporate documents, e.g. con-
tracts, parliamentary documents, doctrine books, court decisions (landmark cases
vs commonplace cases), legislation (primary, secondary, domestic, European,
international). Moreover, both the jurisdiction and the underlying domain of
the document (civil, criminal environmental, taxes, business, etc.) are pertinent
when determining the relevance of an event. For instance, comprising the do-
mestic judicial hierarchy and their procedural rules, and the procedural rules
of the higher Courts, e.g. request for a preliminary ruling, hearings, submitting
observations, opinions, citing case-law, etc.
Level of abstraction: legal events can be considered from a casuistic analysis (a
specific argument of a case), to a general consideration of the facts of a case.
For example, the same event (e.g., someone declares something) can be seen
as a declaration (a reporting event), a procedural event (specifically, part of
the timeline of the proceedings) or as an abstract legal event (an event in the
document level); also, what is being reported could suppose events on their own.
Agents and role: the consideration of legal events can vary according to an agent,
which is a participant in some juridical relationship, e.g., the applicant (a victim
vs perpetrator), or a reputed judge, a notary, a legal scholar. The role of the court
is also influenced by their level of authority (first, second national instances).
Temporal, contextual and spatial features: this quadrant (time, context and space)
can be illustrated by chronological events within court proceedings, such as the
submission of an application in a certain date in the applicant’s national Court;
pleadings; ulterior appeals to a different located court; judgment delivery by
the ECJ (located in a different State) –as the ultimate decision that ends all
proceedings, etc.
Scenario or application-based : annotation of legal events might vary according to
the sought application, purpose or scenario. If one considers predicting judicial
51
decisions, events referring to case-facts will be mostly regarded [29]. If however
the devised application aims to detect arguments for legal argumentation, then
the party’s claims will be the target annotation [30]. The same holds for consid-
ering the most cited case-law for the purposes of considering the authoritative
and relevant ones [31].
These criteria gives a hint on the different dimensions to take into account
when sketching a first definition of an event. They are also a good starting point
for building a event extractor in the legal domain.
4.2 Lessons learned from our case study
In this section we refer to the composition and specificities of the corpus, and
the adopted methodology on the annotations; then we expound on the lessons
learned and finally we provide an initial scope of legal events.
Corpus, data collection and annotations The current sourced corpus con-
sists of 10 European Court of Justice (ECJ) decisions, dated of 2014, extracted
from the EUR-LEX database. These sources were collected because the ECJ
decisions contain different types of events related to different actors and they
contain a standard (but not fixed) structure in which different legal events are
embedded. The ECJ decisions generally include the following structural sections
(according to the Rules of Procedure of the ECJ):
i. Preamble, stating information on the parties and the main object of the
judgment;
ii. Legal context: listing all the legal instruments used in the judgment;
iii. Background of the case. Two settings are observed therein: it is depicted the
factual background of the case; and an account of the procedural history of
the case (typically) followed at national level, before domestic courts or the
state’s authorities, before the application was lodged by the Court. The ECJ
contains also the questions referred therein to the Court, normally titled as
“the dispute in the main proceedings”;
iv. Considerations of the questions referred: the observations submitted to the
Court by the parties and other actors, such as the Governments of Member
States, and the responses of the Court;
v. Cost attribution; and finally
vi. The ruling, i.e., the final decision and the orders to the parties.
The judgments were manually annotated following a procedure herein de-
scribed. While analyzing the decisions, we considered the subsequent constituent
parts:
i. preamble;
ii. background of the case;
iii. consideration on the questions referred
iv. final ruling, as they contain legal relevant events.
52
However, we discarded ii. the legal context and v. the costs attribution, as
no legal events are put forth.
As the text of the decisions is divided into numbered paragraphs, in selecting
the events, we proceeded as follows. For each paragraph, if events were present,
we labeled (i) the contextual information, (ii) the event, and (iii) related syn-
onyms, along with the natural-language content of the judgment.
Observations From our text analysis, we present the ensuing event-aware ob-
servations:
– Events can cover a great deal of linguistic information, consisting of verbs
(declare), nouns (appeal) and nominal phrases (the facts of the case). Ad-
ditionally, it gets more complex to denote its boundaries in terms of the
extension to actors and also their roles, within the time/space/context axis.
– As usual while processing texts in the legal domain, legal terminology, char-
acterized by synonymy, ambiguity, vagueness, polysemy – suppose an extra
challenge. We captured the following variations:
a. Conventional terms change their meaning in the judicial decision-making;
this is the case, for instance, of the expressions “lodging an application
before the Court”, or “criminal proceedings were instituted against the
applicant”, where the terms before and against do not have the conven-
tional meaning;
b. Terms can have several interpretations. For instance, two verbs (sub-
mit, argue) are indicative expressions of an argument. However, the verb
“submit” can also refer to submitting written observations or pleadings,
which consist in procedural documents lodged before the Court.
– In the preamble we can find different types of event-aware information re-
garding to:
a. Identity-related event on identifiable information related to all the partici-
pants involved in the main proceedings:
i. The referring court, and ECJ (and its internal composition);
ii. The litigant parties: applicant/defendant
iii. Agents (States, European Commission, etc);
b. Location-event and date-related event: it is possible to identify both the
date of the request and the place of the national referring court, e.g.,
“10 September 2014, Request for a preliminary ruling from the Krajsky
Sud v Presove (Slovakia)”.
c. Domain-related event of the judgment: the initial summary of the judg-
ment indicates for the domain at stance, e.g., consumer protection (or
others, illegal migration, genetic modified food, etc.)
– The background of the case is the most interesting part to annotate, as it
includes the relevant events, arguments and facts of the parties, e.g. “On 26
February 2009 , Mrs Kusionova concluded a consumer credit agreement with
SMART Capital for an amount of EUR 10 000”, Case 34/13, paragraph 25.
53
– Events can be subsumed to decisions of the national courts (first-instance
and second-instance courts that refer to the ECJ). Expressions on the text
mentioning “Regional Court”, “District Court”, “national court” illustrate
what are the juristic positions of the former courts according to a legal
problem.
– Interpretative issues were considered when analysing the different versions
alleged by each of the parties in the dispute with regard to the same event,
e.g. the claimant alleges there was illegal use of goods and no smuggling. Each
of the involved parties claim that the other one is at fault, which consists in
an interpretative indicator of a same event.
– Negation and Factuality: it must be noted that some events can be negated or
not actual facts, but “possibilities, intentions or preferences” [32]. Examples
of paragraphs 51 and 82, respectively:
“The Court does not have before it any evidence which might raise
doubts as to the compliance of the legislation at issue in the main
proceedings with that principle”; (...) “In the light of the answer to
the first three questions, it is not necessary to respond to that request
by the Slovak Government.”
– As for event-related relations, we observed that two-way (bi-directional) re-
lationships can be found in the same judgment engaging both parties, e.g.
actions “submitted by”, “brought by”; or “the facts of the case, as submit-
ted by the parties; or “observations submitted by the Government and the
observations in reply submitted by the applicant”.
– Legal related events can be identified at different (internal) structures of the
documents, e.g. at a paragraph level of a court decision, or in the summary,
or in the conclusion thereof.
– Within the ECJ judgments, we considered as legal relevant events the fol-
lowing descriptors/indicative expressions, among:
a. Concerning identification elements: petition type, e.g. preliminary ruling;
composition of the court, date of the judgment; parties involved; topical
content of the case; facts;
b. Quoted case-law; The Courts attempt to ground the decision by reference
to established past case-law;
c. Judgment delivery, where the merit of the case is assigned.
5 Conclusions and future work
This paper presents a theoretical reflexive work on events in the legal domain,
wherein an account of their varied definitions, representations and application-
base uses is offered.
We performed a literature review of events in the legal domain in three
different communities: Legal Information Retrieval (LIR), Legal Requirements
Engineering and Legal Knowledge Representation. Even if events in the legal
domain are slightly considered within known frameworks – as it is the case
54
of LegalRuleML, Akoma Ntoso, LKIF – no consensual definition nor standard
representation for legal events has been established.
Our initial analysis, based on a small corpus of ECJ annotated judgments,
permitted to convey an initial scope of legal events. The qualification of event in
the legal domain seeks to be wide-ranging in scope and facilitating the detection
and extraction thereof, regardless of their applicable domain (criminal, civil),
but customizable/modular for instantiation.
The envisaged usefulness and wide prospective applicability of event extrac-
tion/representation as a supportive tool can be evidenced in legal information
retrieval (LIR), in legal knowledge engineering, in Legal Requirements Engi-
neering, and legal argumentation communities, where events are already ac-
counted. Other applicable scenarios can be contemplated, among the following:
e-discovery; case-based reasoning (CBR); legal argumentation; determining con-
tractual positions and concept definition; contract review; prediction technology;
electronic billing.
As future work, the authors will further prune the definition of legal events by
atomizing their properties, instances, and attributes and set some competency
questions relevant for extracting necessary information from court decisions.
Forth bringing annotations of a corpus of ECHR judgments will proceed for
reviewing, canvassing and consolidating the annotation frames acquired in this
paper. A semantic model will be constructed from these annotations to provide
for structured legal data on the web. Regarding evaluation, assessment criteria
for validating the relevance of an event within a legal text will be developed.
Acknowledgments
This work was supported by a Predoctoral grant from the I+D+i program of
the Universidad Politécnica de Madrid.
References
1. Stephann Makri, Ann Blandford, and Anna L Cox. Investigating the information-
seeking behaviour of academic lawyers: From elliss model to design. Information
Processing & Management, 44(2):613–634, 2008.
2. Marc Van Opijnen and Cristiana Santos. On the concept of relevance in legal
information retrieval. Artificial Intelligence and Law, 25(1):65–87, 2017.
3. Ibrahim Haruna and Iyabo Mabawonku. Information needs and seeking behaviour
of legal practitioners and the challenges to law libraries in lagos, nigeria. The
International Information & Library Review, 33(1):69–87, 2001.
4. Jody J. Daniels and Edwina L. Rissland. Finding legally relevant passages in
case opinions. In Proceedings of the 6th International Conference on Artificial
Intelligence and Law, ICAIL ’97, pages 39–46, New York, NY, USA, 1997. ACM.
5. Simon Gottschalk and Elena Demidova. Eventkg: A multilingual event-centric tem-
poral knowledge graph. In Aldo Gangemi, Roberto Navigli, Maria-Esther Vidal,
Pascal Hitzler, Raphaël Troncy, Laura Hollink, Anna Tordai, and Mehwish Alam,
editors, The Semantic Web, pages 272–287, Cham, 2018. Springer International
Publishing.
55
6. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative event
chains. Proceedings of ACL-08: HLT, pages 789–797, 2008.
7. Ace (automatic content extraction) english annotation guidelines for events.
https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/english-events-
guidelines-v5.4.3.pdf.
8. David Ahn. The stages of event extraction. In Proceedings of the Workshop on
Annotating and Reasoning about Time and Events, pages 1–8. Association for Com-
putational Linguistics, 2006.
9. Jacqueline Aguilar, Charley Beller, Paul McNamee, Benjamin Van Durme,
Stephanie Strassel, Zhiyi Song, and Joe Ellis. A comparison of the events and
relations across ace, ere, tac-kbp, and framenet annotation standards. In Proceed-
ings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and
Representation, pages 45–53, 2014.
10. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative
schemas and their participants. In Proceedings of the Joint Conference of the
47th Annual Meeting of the ACL and the 4th International Joint Conference on
Natural Language Processing of the AFNLP: Volume 2 - Volume 2, ACL ’09, pages
602–610, Stroudsburg, PA, USA, 2009. Association for Computational Linguistics.
11. Caroline Hagege and Xavier Tannier. Xtm: A robust temporal text processor. In
International Conference on Intelligent Text Processing and Computational Lin-
guistics, pages 231–240. Springer, 2008.
12. Philippe Capet, Thomas Delavallade, Takuya Nakamura, Agnes Sandor, Cedric
Tarsitano, and Stavroula Voyatzi. A risk assessment system with automatic ex-
traction of event types. In International Conference on Intelligent Information
Processing, pages 220–229. Springer, 2008.
13. Frederik Hogenboom, Flavius Frasincar, Uzay Kaymak, and Franciska De Jong.
An overview of event extraction from text. In Workshop on Detection, Represen-
tation, and Exploitation of Events in the Semantic Web (DeRiVE 2011) at Tenth
International Semantic Web Conference (ISWC 2011), volume 779, pages 48–57.
Koblenz, Germany: CEURWS. org., 2011.
14. K Tamsin Maxwell, Jon Oberlander, and Victor Lavrenko. Evaluation of semantic
events for legal case retrieval. In Proceedings of the WSDM’09 Workshop on Ex-
ploiting Semantic Annotations in Information Retrieval, pages 39–41. ACM, 2009.
15. Nikolaos Lagos, Frederique Segond, Stefania Castellani, and Jacki ONeill. Event
extraction for legal case building and reasoning. In International Conference on
Intelligent Information Processing, pages 92–101. Springer, 2010.
16. Gerardo Sierra, Gemma Bel-Enguix, Guillermo López-Velarde, Ricardo Saucedo,
and Lucı́a Rivera. Event extraction from legal documents in spanish. In Workshop
on Language Resources and Technologies for the Legal Knowledge Graph, LREC
2018. Miyazaki, Japan, 2018.
17. Anderson Bertoldi, Rove Chishman, Sandro José Rigo, and Thaı́s Domnica
Minghelli. Cognitive linguistic representation of legal events. towards a semantic-
based legal information retrieval. In COGNITIVE 2014 : The Sixth International
Conference on Advanced Cognitive Technologies and Applications, 2014.
18. Nadzeya Kiyavitskaya, Nicola Zeni, Travis D Breaux, Annie I Antón, James R
Cordy, Luisa Mich, and John Mylopoulos. Automating the extraction of rights and
obligations for regulatory compliance. In International Conference on Conceptual
Modeling, pages 154–168. Springer, 2008.
19. Silvia Ingolfo, Ivan Jureta, Alberto Siena, Anna Perini, and Angelo Susi. Nòmos 3:
Legal compliance of roles and requirements. In Eric Yu, Gillian Dobbie, Matthias
56
Jarke, and Sandeep Purao, editors, Conceptual Modeling, pages 275–288, Cham,
2014. Springer International Publishing.
20. Thomas F Gordon. The legal knowledge interchange format (lkif). Estrella deliv-
erable d4, 1, 2008.
21. Fabien Gandon, Guido Governatori, and Serena Villata. Normative requirements
as linked data. In The 30th international conference on Legal Knowledge and
Information Systems (JURIX 2017), 2017.
22. Heng Ji, Ralph Grishman, Zheng Chen, and Prashant Gupta. Cross-document
event extraction and tracking: Task, evaluation, techniques and challenges. In
Proceedings of the International Conference RANLP-2009, pages 166–172, 2009.
23. James Pustejovsky, Marc Verhagen, Xue Nianwen, Robert Gaizauskas, Mark Hep-
ple, Frank Schilder, Graham Katz, Estela Saquete, Tommaso Caselli, et al. Tempe-
val2: Evaluating events, time expressions and temporal relations (in semeval tasks
proposal).
24. Naushad UzZaman, Hector Llorens, Leon Derczynski, James Allen, Marc Verha-
gen, and James Pustejovsky. Semeval-2013 task 1: Tempeval-3: Evaluating time
expressions, events, and temporal relations. In Second Joint Conference on Lexi-
cal and Computational Semantics (* SEM), Volume 2: Proceedings of the Seventh
International Workshop on Semantic Evaluation (SemEval 2013), volume 2, pages
1–9, 2013.
25. Hector Llorens, Nathanael Chambers, Naushad UzZaman, Nasrin Mostafazadeh,
James Allen, and James Pustejovsky. Semeval-2015 task 5: Qa tempeval-evaluating
temporal information understanding with question answering. In Proceedings of
the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages
792–800, 2015.
26. The ace 2005 ( ace 05 ) evaluation plan. evaluation of the detection and recog-
nition of ace entities , values , temporal expressions , relations , and events.
https://www.semanticscholar.org/paper/The-ACE-2005-(-ACE-05-)-Evaluation-
Plan-Evaluation-NTRODUCTION/3a9b136ca1ab91592df36f148ef16095f74d009e.
27. Rachel Chasin. Event and temporal information extraction towards timelines of
wikipedia articles. Simile, pages 1–9, 2010.
28. Radityo Eko Prasojo, Mouna Kacimi, and Werner Nutt. Modeling and summa-
rizing news events using semantic triples. In European Semantic Web Conference,
pages 512–527. Springer, 2018.
29. Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preoţiuc-Pietro, and Vasileios
Lampos. Predicting judicial decisions of the european court of human rights: A
natural language processing perspective. PeerJ Computer Science, 2:e93, 2016.
30. Marco Lippi, Francesca Lagioia, Giuseppe Contissa, Giovanni Sartor, and Paolo
Torroni. Claim detection in judgments of the eu court of justice. In Ugo Pagallo,
Monica Palmirani, Pompeu Casanovas, Giovanni Sartor, and Serena Villata, ed-
itors, AI Approaches to the Complexity of Legal Systems, pages 513–527, Cham,
2018. Springer International Publishing.
31. Marc van Opijnen. Towards a global importance indicator for court decisions.
In Legal Knowledge and Information Systems - JURIX 2016: The Twenty-Ninth
Annual Conference, pages 155–160, 2016.
32. Marı́a Navas-Loro, Ken Satoh, and Vı́ctor Rodrı́guez-Doncel. Contractframes:
Bridging the gap between natural language and logics in contract law. In Pro-
ceedings of theTwelfth International Workshop on Juris-informatics (JURISIN)
(November 2018), 2018.
57