Knowledge extraction systems are strongly demanded in the legal domain, to provide legal actors like judges or lawyers with useful and relevant information to enforce a knowledge-based evaluation and judgement of new cases. In this paper, we present LATO-KM, a three-layer legal knowledge model where terms featuring legal knowledge, both law and case-law, are properly formalized as entities and relationships and they are implemented in the LATO ontology using SKOS. The LATO ontology constitutes the core component of CRIKE (CRIme Knowledge Extraction), a data-science approach and related tool environment conceived to support legal knowledge extraction and enrichment from a corpus of Court Decision documents.
Law is the set of rules which govern human conduct. Law is stated using a
general and abstract terminology, in that it has to be applicable to several cases and
events. On the opposite, Court Decisions (CDs) are written using speci c and
concrete terminology, in that they provide a contextualized, case-oriented
interpretation of law deriving from the way judges/lawyers decide to apply the law
statements to the speci c circumstances/situation of the case at hand. Both law
and case-law (that is, the set of CDs) constitute prominent knowledge sources to
be considered for the knowledge-based evaluation and judgement of a new case,
in that they provide the general legal framework (law) and the speci c
interpretations (case-law) adopted for already processed cases. When a new case is received
for judgement, the knowledge-based evaluation process takes into account
relevant legal knowledge to support CD de nition, that is, knowledge deriving from
i) the law, for understanding the general rules that are relevant/prominent for
the current case [
In this paper, we present LATO-KM (Legal Abstract Term Ontology - Knowledge Model), a three-layer knowledge model where terms featuring legal knowledge, both law and case-law, are properly formalized as entities and relationships and they are implemented in a LATO ontology using SKOS. The LATO-KM and the related LATO ontology constitute a core component of CRIKE (CRIme Knowledge Extraction), a data-science approach and related tool environment conceived to support knowledge extraction and enrichment from a corpus of Court Decision documents. Knowledge extraction in CRIKE is based on multilabel classi cation techniques that aim at associating CD documents with appropriate concepts in the LATO ontology. Knowledge enrichment in CRIKE is based on black-box model explanation techniques that aim at selecting the document features (i.e., terms) candidate for enrichment of the LATO ontology.
The paper is organized as follows. Section 2 presents the LATO ontology formalization. The CRIKE techniques for knowledge extraction and enrichment are described in Section 3. In Section 4, we discuss experimental results on a real CD dataset. Concluding remarks are nally provided in Section 5. 2
The legal knowledge model of LATO captures and formalizes the features and
nature of terminology used in law and case-law documents. A design challenge is
to nd a suitable way of modeling the di erent nature of terms appearing in law
and case law as well as their meaning and roles [
Legend FUNCTIONAL CATEGORY
LEGAL CONCEPT TERM-SET is-a istance-of Drug Minor Offense Act
Drug Trafficking Illinois Controlled Substances Act
Illinois Contr.
Sub. Act, ICSA, …
Deal Deal, Dealing to deal, …
Unit of Measure
Weight Kilo
Kilo, kilogram, kg., … { Legal concept: a legal concept Ci denotes a general rule/fact/element de ned in the law (e.g., Act, Illinois Controlled Substances Act) and it is labelled with the terminology that appears in law texts. Legal concepts constitute the intermediate layer of LATO-KM. { Term-set: a term-set Ti represents the concrete interpretation of a legal concept Ci in form of a set of terms occurrences that can be found in case-law texts. A term in a term-set is a string of characters of the language of the case law texts; also multi-term expressions are considered as terms in LATO (e.g. both Illinois Contr. Sub. Act and the acronym ICSA are terms). Term-sets constitute the bottom layer of LATO-KM. { Functional category: a functional category represents the di erent kinds/roles of legal concepts in the law formulation, namely descriptive, statutory, modi er, and abstract, respectively. A statutory category describes a legal
concept featuring something that is directly or indirectly de ned in the law
speci cation itself (e.g., Act). A descriptive category describes a legal
concept featuring actions, human activities, and any real-life object in the
law speci cation (e.g., Drug Tra cking). A modi er category describes
a legal concept featuring quantitative/qualitative aspects of things/actions
in the law speci cation (e.g., Weight). An abstract category describes a
legal concept featuring something indeterminate that requires a concrete
application for being really de ned (e.g., Drug Minor O ence) [
According to LATO-KM, the concrete meaning of legal concepts is fully de ned by referring to the speci c terminology (i.e., term-set) that appears in real CDs. Moreover, legal concepts are classi ed with respect to the role they play in the law formulation using functional categories. To formalize, a legal concept Ci is de ned as 3-uple of the form:
Ci = hn(Ci); C(Ci); Ti i where: { n(Ci) is the label of the legal concept; { C(Ci) 2 fSC; DC; M C; ACg is the functional category of Ci, either statutory (SC), descriptive (DC), modi er (M C), or abstract (AC). { Ti = ft1; : : : ; tng is the term-set of the concept Ci, namely the language terms concretely used in legal document corpora (i.e., Court Decisions) to refer to Ci. The asterisk symbol (\*") denotes optionality, in that we may have some legal concepts not yet associated with a corresponding term-set. For instance, abstract concepts are not directly associated with a speci c term-set, but rather they are indirectly expressed through the term-sets associated with the legal concepts to which the abstract concept is related.
Intra- and inter-layer relationships are de ned in LATO-KM to capture the semantic relationships that hold between pairs of entities. The following intralayer binary relationships are de ned in LATO-KM: { Term-to-Term: it is a binary relationship between a pair of terms t and t0 in a term-set Ti at the bottom layer, that holds due to either a morphological or a linguistic relationship between terms. Examples of morphological relationships are: paradigm (e.g. to deal - dealt - dealt) conjugation for verb (e.g. dealt - deals - dealing) declension for nouns (e.g. drug - drugs - drug's) abbreviation (e.g. Illinois Contr. Sub. Act - ICSA) string similarity (e.g. Substances Act - substances act - Subs. Act). An example of linguistic relationship is synonymy (e.g., Paragraph - Section). { Concept-to-Concept: it is a binary relationship between two legal concepts Ci and Cj at the intermediate layer, capturing semantic relationships holding between them in the law formulation. In particular, we introduce the kind-of relationship between two concepts to represent a generalization/specialization relationship between them. For example, Drug minor o ence kind-of Minor o ence is de ned to express the fact that the former is a more speci c crime than the latter in the law. Moreover, we introduce the related relationship between two concepts to represent a generic positive relationship between them. For example, Drug Minor O ence related Drug is de ned to express the fact that the crime of drug minor o ence involves detention of drug in some quantity.
The following inter-layer binary relationships are de ned in LATO-KM: { Term-to-Concept: it is a binary relationship between a term t 2 Ti and a legal concept Ci denoting that Ci can be \lexicalized" by t in a CD text. A Term-to-Concept relationship is de ned through the instance-of relationship for each term t 2 Ti and the corresponding legal concept Ci at the intermediate layer of LATO-KM. For example, ICSA instance-of Illinois Controlled Substances Act is de ned to express the term ICSA belongs to the term-set of the concept Illinois Controlled Substances Act. { Concept-to-Category: it is a binary relationship between a legal concept Ci and a functional category C (Ci) expressing the nature of the concept in the law formulation. A Concept-to-Category relationship is de ned through the is-a relationship. Act is-a Statutory is de ned to express that the notion of Act is directly de ned in the law. 2.1
The LATO-KM is implemented in a LATO ontology by using the Simple
Knowledge Organization System (SKOS) [
The legal concepts of the intermediate layer are implemented as SKOS concepts in LATO. Concept-to-Concept relationships are speci ed through a corresponding SKOS relation. In particular, the kind-of relationship of LATO-KM is speci ed through the skos:broader relation. For instance, a skos:broader relation is de ned between the concept Cocaine and the concept Drug. The Related relationship of LATO-KM is speci ed through the skos:related relation. For instance, a skos:related relation is de ned between the concept Drug Minor O ence and the concept Drug.
The term-sets of the bottom layer are implemented using labels of SKOS concepts. In particular, for each SKOS concept i) a skos:prefLabel is de ned to implement the instance-of relationship, and ii) a number of skos:altLabel are dened to implement the various Term-to-Term relationships denoting possible alternative terms for the considered SKOS concept. For instance, a skos:prefLabel
RELATION
EXAMPLE
RELATION NAME
SKOS IMPLEMENTATION Conceptto-Category Conceptto-Concept Term-toConcept Term-toTerm relation is de ned between the Drug LATO concept and the Drug term, while a skos:altLabel relation is de ned between the Drug term and the Narcotics term.
Finally, functional categories of the top layer are implemented as SKOS concepts, too. Concept-to-Category is-a relationships are expressed through the skos:broader relation. For instance, a skos:broader relation is de ned between the Drug LATO concept and the Descriptive category concept. 3
The LATO ontology is a core component of the CRIKE approach to enforce knowledge extraction and enrichment based on a given corpus of Court Decision documents (see Figure 3). The goal of CRIKE is to progressively enrich the knowledge speci ed in a reference LATO ontology by extracting concrete terminology associated with concept applications/interpretations occurring in the considered document corpus. At the beginning, CRIKE relies on an initial version of the LATO ontology where domain experts manually de ne a starting set of legal concepts of interest with associated term-sets. CRIKE is enforced as a cyclic incremental approach where the execution of knowledge extraction and knowledge enrichment tasks produces a new enriched version of the LATO ontology. The enrichment task consists in discovering terms to populate term-sets Corpus of Court Decisions
CRIKE (CRIme Knowledge Extraction)
LATO ontology
LATO-KM knowledge enrichment knowledge extraction
legal actors (e.g., judges, lawyers)
Court Decisions on incoming trials of legal concepts that have recognized in the text of the CD documents. This new ontology version is then exploited to trigger the execution of a new CRIKE cycle to further enrich the LATO ontology. The enforcement of CRIKE cycles is stopped when the enrichment of the LATO ontology is terminated, namely when it is not possible to detect/extract additional terms to insert in the LATO ontology. As a result, the knowledge currently-available in the LATO ontology can be exploited to support legal actors such as judges and lawyers in managing new incoming legal trials and taking appropriate Court Decisions. Knowledge extraction in CRIKE is based on multi-label classi cation techniques where the training set is built by relying on the ontology contents without the need of manual annotation. In other words, CRIKE works as a sort of selftraining scheme that can be considered as a kind of semi-supervised learning approach. Extraction is articulated in three main steps as follows:
Document annotation. For each CD document d, the goal of annotation is to determine the set of associated legal concepts Cd as follows: (
" Cd =
Ci :
X w(t; d) t2Ti # th ) where w(t; d) is the weight of a term t in the document d according to standard information retrieval techniques based on tokenization and tf-idf, while th is a threshold used to set the minimum cumulative weight of all the terms t 2 Ti that is required for associating a corresponding concept Ci with the document d.
Document vector representation. For each document d in the corpus, a
vector-based representation d is derived by exploiting doc2vec techniques [
Document classi cation. A multi-label classi er is employed to generate a model that is capable to predict the association of CD documents with legal concepts. In CRIKE, we employ a Convolutional Neural Network (CNN) with the goal to generalize the terminology of the documents and to enable the association of legal concepts with Court Decisions that actually contain terms other than those already included in LATO. For each document d, the CNN receives the document vector representation d as input and it produces the corresponding concept vector representation cd as output. As a result, a multi-label classi cation model M is generated to map the correspondence between corpus documents and legal concepts in the LATO ontology. In particular, by Ci 2 M (d) we denote that the document d is associated with the legal concept Ci through the model M . 3.2
Knowledge enrichment in CRIKE is based on black-box model explanation techniques that aim at selecting the document features (i.e., terms) that play a major role in determining the decision of the CNN classi er (see above the knowledge extraction step) about the association of concepts with the corpus documents. The selected document features are candidate for the enrichment of the LATO ontology. Enrichment is articulated in three main steps as follows:
Classi cation explanation. Black-box model explanation is enforced by
relying on LIME (Local Interpretable Model-agnostic Explanations) [
TCi = 8 < : t : t 2
9 [ d= d2DCi ; The choice of CNN is due to the positive experimental results we observed in a number of considered case-studies. As a general remark, di erent kinds of multilabel classi er can be employed for enforcing document classi cation, like for example random forest and kNN. Then, we associate each term t 2 TCi with a degree of relevance Ci (t) as follows: Ci (t) =
X
X t2TCi d2DCi (t; d) The set TCi is the set which contains the terms that are candidate to enrich the term-set associated with Ci in LATO.
Expert validation. For each concept Ci, legal experts are involved to validate the terms in the set TCi n Ti. Furthermore, the legal experts de ne the set Ri (TCi n Ti) containing the terms that are relevant for Ci. In the validation step, the degree of relevance Ci (t) is exploited by the experts i) to lter out terms whose association with the concept Ci is poor (i.e., low values of Ci (t)), and ii) to select terms whose association with the concept Ci is strong (i.e., high values of Ci (t)).
Ontology enrichment. According to the results of legal expert validation, the term-set Ti associated with each concept Ci is enriched. Being k the current CRIKE cycle, enrichment is enforced as follows:
Tik+1
T k i [ Ri where Tik is the term-set initially associated with the concept Ci and Tik+1 is the term-set associated with Ci after enrichment.
Example. In Figure 4, we report an example of two court decision fragments, d1 and d2 that are associated with the legal concept Drug.
d1: [...] Paragraph 14 of section 1 of the same act provides: \Narcotic Drugs means coca leaves, opium, cannabis, and each substance neither chemically nor physically distinguishable from them." [...] d2: [...]Defendant, who was charged by indictment with violation of 402 of the Illinois Controlled Substances Act" [...]
The Court Decision d1 is included in the corpus used for training the multilabel classi cation model M . The model classify both d1 and d2 as documents related to the Drug legal concept. In case of d1, the choice of the classi er is trivial, since d1 is classi ed as Drug-related in the training set and it contains the terms narcotic drugs and cannabis. The decision of classifying d2 as a Drug-related is instead less trivial, because d2 does not contain any of the terms provided by experts as part of the term-set of the Drug concept. However, the two documents are semantically similar. As a consequence, the two court decisions are close enough in the feature (i.e., term) space to motivate the classi er decision of associating both with the concept Drug. Thus, we can use LIME to detect the terms of d1 and d2 that have the main impact on the classi er decision, namely the terms that, if deleted from the court decision, may more likely produce a di erent classi cation result. According to the LIME results, we obtain the following terms for the concept Drug: narcotic drugs, controlled substances, cannabis, coca leaves, opium. Among the list, narcotic drugs and cannabis are already present in LATO, while the others (underlined in Figure 4) are validated by the legal experts and included in an enriched version of the term-set layer of LATO. 4
The goal of our experimentation is to assess the capability of our approach to discover new terms for enriching the term-sets of legal concepts in LATO. For the experiments, we select six concepts from our legal ontology, namely drug, drug tra cking, unit of measure, illinois legislation, criminal procedure, and evidence. These concepts are all related to the drug criminal legislation of the State of Illinois. The court decision corpus used for experiments is composed by 14,000,000 sentences taken from about 180,000 decisions of courts of the State of Illinois taken from the Caselaw Access Project (CAP) that provides public access to U.S. law (https: //case.law/bulk/download) digitized from the collection of the Harvard Law Library. Sentences are indexed by exploiting standard techniques for tokenization and compound term detection. The initial term-sets associated with the selected concepts have been manually de ned by a legal expert and they are shown in Table 1.
By using CRIKE, we select a subset of 115,993 court decision sentences that constitutes the training set for the classi cation step. The training set is prepared for classi cation by embedding each document in a 100 dimensions vector using doc2vec to obtain a 115,993 100 corpus matrix. The six concepts selected for the experiment are associated with CD documents with the document annotation process discussed in Section 3. The model M used to train the classi er is a neural network organized in three layers. Between the input and the output layer, we use a convolution lter activated by ReLU. The M1 accuracy obtained by cross-validation is 0.77. M is then used to perform terminology enrichment using LIME. For each legal concept Ci, we obtain a new set of terms TCi , where each term t is associated with the degree of relevance Ci (t). In the experiment, a legal expert validated the top-20 terms in the new term-set Ti of each concept Ci. In particular, the expert associates each term t with a numerical value in j TCi j j T 1 j j T 0 j j T 1 j f 1; 0; 1g, where T 1 denotes the set of terms that were not in LATO and that are not relevant for the concept Ci; T 0 denotes the set of terms that were in LATO (and thus have been already validated as relevant); T 1 denotes the set of terms that were not in LATO but that are relevant for the concept Ci. An overview of the results of knowledge enrichment is shown in Table 2.
The number of relevant terms retrieved through knowledge enrichment (i.e., terms in T 0 or T 1) is equal to the 83% of the total number of new terms validated by the expert (TCi ). The 34% of those terms was not in the term-sets of LATO. As expected, the increment of new relevant terms is higher for the concepts that were associated with small term-sets, such as illinois legislation, criminal procedure, and evidence. The number of irrelevant terms T 1 is limited with the exception of the concept evidence, because the criminal evidences usually consist in common objects that are used in a criminal context. These objects are thus associated with a generic terminology (e.g., garbage, suitcase) that cannot be associated per se to an evidence according to the legal expert. The new relevant terms are nally included in the term-sets of LATO. A new CRIKE cycle has been then executed. The new term-sets are exploited in the knowledge extraction steps and a new training set of 158,398 CD sentences is extracted (+37% with respect to the rst execution). These sentences are then used to train a new model M and to enforce the execution of the knowledge enrichment steps. Finally, the accuracy of M obtained by cross-validation is 0.81 (+5.2%). 5
In this paper, we presented the LATO-KM for automated knowledge extraction from Court Decisions corpora. The CRIKE knowledge extraction and enrichment process is based on black-box models explanation techniques. Preliminary results on a corpus of Court Decision documents show that our approach achieves promising results in e ectively discovering new terminology for enriching the term-sets associated with legal concepts in the LATO ontology. Ongoing work is related to the extension of the LATO knowledge model to enforce rule-based extraction and classi cation techniques. The goal is to improve the accuracy in recognizing the application of abstract legal concepts in CD documents. We aim to exploit reasoning techniques based on ontology rules de ned over legal concepts for detecting concept instances throughout documents where speci c constraints are satis ed.