Terms in the laws of a legislature can be highly contextual: especially for corpora of codified laws and regulations where the reader has to be aware of the correct context when the corpus lacks a single level of hierarchy. The goal of this work is to assist professionals when reading legal text within a codified corpus by finding contextually consistent information units. To achieve this, we combine NLP and data mining techniques to develop novel methodology that can find these information units in an unsupervised manner. Our method draws on expert experience and is modeled to emulate the “contextualization process” of experienced readers of legal content. We experimentally evaluate our method by comparing it to multiple expert-annotated datasets and find that our method achieves near perfect performance on four state corpora and high precision on one federal corpus.
Within a corpus, the same term can have diferent meanings. For instance, in the United States Code (USC) 26 USC § 7701(a)(1) provides that for purposes of Title 26 “The term ‘person’ shall be construed to mean and include an individual, a trust, (...) or corporation.” However, 42 USC § 2000e(a) provides that for purposes of the subchapter it belongs to “The term ‘person’ includes one or more individuals, governments, (...) or receivers.” Thus, the context is not identical across the corpus.
This work’s goal is to inform professionals who read a section of legal text about the continuous and contextually consistent information unit the section belongs to. We coin such a unit “root context” and it commonly represents an individual law on a specific topic. Root context is used where a codified corpus’ hierarchical structure does not designate a single level for individual laws. Our root context method is modeled to emulate the “contextualization process”, where experienced readers use references in the text to help contextualize what they are reading. The contextualization process can be broken down into three steps: 1) The reader notices that definition of ‘person’ is not in the current section. 2) The reader needs to find the definition of ‘person’ that is applicable to the current section. 3) Once the definition for ‘person’ is found ∗This work was done during an internship at Regology. 5 8
5 SC1
C1
8 3 SC2 3 8 T1 8 9 8 SC3 8
C2 9 SC4 9 9 S1 - S15
S20 - S25
S30 - S35 S40 - S49 the reader needs to understand which other sections the definition applies to. For instance, Figure 1 shows Chapter 1 as root context. If the reader is reading Section 12 she might find the mention of the word ‘person’. From our algorithm the reader is informed that the definition of ‘person’ is applicable to Chapter 1, which encompasses Section 1 through Section 25.
One challenge is to optimize the existing hierarchy within legal documents. For instance, the USC has 53 titles that are broken down in lower levels of hierarchy such as chapters, parts, etc. We address the aforementioned challenge by combining the natural language text and the existing document hierarchy to find higher-level logical groupings, which are not present in the original document hierarchy (we call these groupings “root contexts”). To achieve this, our method makes use of NLP techniques to extract hierarchy references from the text and automatically builds an hierarchy-reference graph (an example is depicted in Figure 1 with hierarchy references shown as arrows in the figure). We then create an algorithm that follows the graph references to automatically identify a point in the hierarchy that is a root context.
We evaluate our approach on five U.S. law corpora (one federal and four state corpora). To build our test datasets, we have a domain expert annotate every root context in each of the corpora. In our experiments, we compare these annotations with the predictions of our algorithm and find that our method achieves near perfect performance on our state corpora and high precision on our federal corpus. Summarizing, our work makes the following contributions: (1) We introduce and define the novel problem of root context identification. (2) We combine NLP and data mining techniques to develop novel methodology to identify root contexts. (3) We show the efectiveness of our approach on five corpora.
The identification of information units is common in library and
information sciences [
Definition 3.1. Hierarchy instance: A concrete instantiation of a part of the hierarchy, e.g. “Title 12”, denoted as 12. Each hierarchy instance is the union of all its sub-parts. For example, 12 is made up of chapters 1, ...18, which transitively are made up of subchapters , etc.
Definition 3.2. Hierarchy level: The union of all hierarchy instances for a certain level. For instance, the “title” hierarchy level T of USC encompasses all 53 title instances : (1, 2, ..., 53) ∈ T . The lowest level we consider for this work are Sections , which contain all the textual information.
Definition 3.3. Hierarchy branch: A unique identifier for a specific part in the corpus hierarchy. The hierarchy branch is a tuple of hierarchy instances that need to be “followed” from the hierarchy root. For example, the tuple (‘USC’, ‘12’, ‘2’, ‘221’), identifies Section 221 in Chapter 2 (“Federal Reserve System”), within Title 12 (“Banks and Banking”) in the United States Code.
Definition 3.4. Hierarchy reference: The hierarchy branch of a reference made to the hierarchy in legal text. For example, if the text in (‘USC’, ‘12’, ‘2’, ‘12’) references its chapter, then the resulting hierarchy reference is: (‘USC’, ‘12’, ‘2’). We define custom regular expressions to identify hierarchy references in the text. Definition 3.5. Hierarchy-reference graph: A weighted, directed graph where the nodes are hierarchy branches and the edges are hierarchy references between them. The edge weight is the number of references between a pair of nodes.
Definition 3.6. Root context: A specific hierarchy instance, which makes up a contextually consistent information unit. Definition 3.7. Root context identification: Given a document collection (i.e., a law corpus), extract all hierarchy references from the text and combine them with the existing document hierarchy to form a hierarchy-reference graph = ( , , ). More specifically, is the set of all hierarchy branches, ⊆ {(, ) | (, ) ∈ 2} are all directed hierarchy references and : → R the edge weights. The goal is to find a set ⊆ with all hierarchy branches (i.e., nodes in the graph) that are root contexts. 4
Algorithm 1: Root Context Identification
Data: , a document collection (i.e., a law corpus), a set of all hierarchy branches Result: , a set of all root contexts with ⊆ begin = ( , , ) ← extract hierarchy references and build graph; ← identify hierarchy references in definition and purposes sections; ← ∅; _ ← 10; for ∈ do ℎ ← 0; _ ← ; while ℎ < _ do ℎ ← ℎ + 1; if _ ∈ then ← ∪ _; break; ← ∅; for (, ) ∀ = _ do
← ∪ (, ( (, ))); if | | > 0 then _ ← max ((,)) ; if _ = _ then ← ∪ _; break; _ ← _; else
_ ← ( _);
Our approach consists of three phases: 1) extract hierarchy references and build the hierarchy-reference graph 2) find root context indicators within definitions and purposes sections 3) perform multi-hops following the graph’s edges with the highest weight until a root context is identified. Algorithm 1 describes our methodology for identifying root contexts. The input to the algorithm is a document collection and a set of all hierarchy branches . The output is a set of root contexts .
Build hierarchy-reference graph ( = ( , , )): We start with the nodes of the graph , which are all hierarchy branches in a legal corpus. Because only the section hierarchy level S includes textual context, we start investigating the text of all ∈ S. For each , we extract all hierarchy references by looking for textual references of the hierarchy. We use a set of regular expressions to find these references in text, i.e., “this (<hierarchy level>)”. For instance, if the legal text in (‘USC’, ‘12’, ‘2’, ‘12’) says “this chapter” we extract (‘USC’, ‘12’, ‘2’). We then add an edge between all distinct hierarchy references and the current , where the edge weight is the sum of the hierarchy references for the edge’s target node. Once this step is completed for S, we begin a count aggregation step at each hierarchy level, moving “upwards” in the hierarchy. In our running example, we move from S to the chapter hierarchy level C. For each ∈ C, we aggregate all hierarchy references that go from to other parts in the hierarchy. For instance, we aggregate all hierarchy references to 12 within the hierarchy instance of 2, as a single edge from (‘USC’, ‘12’, ‘2’) to (‘USC’, ‘12’), with the combined edge weight of all its children that are pointing to 12.
definitions definitions purposes purposes
this (\w+) (?:describes|sets forth|governs|contains) the following definitions apply to this ( \w+) the purposes? of (?:the )?.*this (\w+) (?:are|include|is) this (\w+) sets forth Identify root context indicators within definitions and purposes sections (): References in certain sections that contain definitions or purposes carry more weight than regular sections, as these were authored with the intend of helping the reader with contextualization. Our extraction algorithm goes through all sections that contain either “definition” or “purpose” in their title. We define a set of regular expressions that extract the hierarchy references in these sections. Table 1 shows some examples, where “(\w+)” matches a reference to the hierarchy (e.g., “chapter”). If one of the regular expressions matches the text within a section, the hierarchy branch of the extracted reference is added to a set .
Traverse hierarchy-reference graph to generate : Starting at every node in the graph, we perform multiple hops (in our case 10) along the graph, following always the outgoing edge with the highest weight. If a node is found that is in , it will automatically be added as a root context1. If a node’s highest weighted outgoing edge points to itself, then we also consider the node a root context. If a node has no outgoing edges, we move up one level in the hierarchy, to the node’s parent, and continue the procedure. 5 5.1
Our experimental evaluation aims at measuring the eficacy of our
root context identification approach outlined in Section 4. For our
experiments, we run our algorithm and compare its predictions to
the expert annotations. We experiment with three full law corpora:
United States Code (USC) [
For each hierarchy branch in , we have a domain expert annotate whether it is a root context (“1” label) or not (“0” label). Table 2 shows the statistics for each dataset, with its total number of nodes (“Total”), the number of annotated root contexts (“Number Root Contexts”) and the percentage of root contexts out of all nodes in the graph (“%”).
Since our experimental setup is equivalent to a classification problem, we report 1-score, precision, recall of the class under consideration (i.e., “1” label) and classification accuracy. In this case, false positives are instances that our method identifies as root contexts, however, the expert annotated these instances as not a root context. False negatives are root contexts that were identified by the expert but not found by our method.
USC CACL TXST ILCS NYCL
USC CACL TXST ILCS NYCL
We presented the problem of finding contextually consistent information units to assist professionals when reading legal text and developed novel methodology to find these units. We evaluate our method and find that it achieves high precision and 1 score on multiple datasets. The high accuracy of our method indicates that the task is not hard and the proposed method has worked well. Since the method is unsupervised, it does not require any manual work and can thus be applied broadly to all such application problems. Naturally, we may further improve performance by applying supervised learning with our method used as one feature and combine it with other features, especially text-based features, which would be future work. While we find our method to be efective, there are instances where root contexts cannot be identified due to the lack of hierarchy references. To solve this in the future, we envision to ifnd root context indicators by looking at the evolution of legal text over time and study the diferences of diferent versions of the law.
This work further aids Regology’s2 machine learning framework. For instance, the root context concept has been successfully utilized to build a keyword extraction algorithm; increase the performance of existing information retrieval components (e.g., grouping search results, assessing law changes, diferentiating between new and amending laws), and extract topics. 1We experimented with diferent weighting schemes for nodes that are in , however, we found that “overriding” the edge weights in works best.