This research project develops a methodology to utilize machinelearning analysis of live disputes to assist legal professionals in narrowing issues and comparing relevant precedents. An important part of this research project is an ontology of target court decisions, which is first constructed from relevant legal statutes, probably assisted by human legal experts. Utilizing the ontology, we extract and classify sentences in court decisions according to type. Since court decisions are typically written in natural languages, such as English and Chinese, chosen court decisions are pre-processed with the Natural Language Processing library spaCy, and then fed into the machine learning component of the system to extract and classify the sentences of the input court decisions.
This research project develops a methodology to utilize machinelearning analysis of live disputes to assist legal professionals in narrowing issues and comparing relevant precedents. At this early stage of the research, we limit the scope to a specific methodology for legal research of breach of contract issues that also creates a general template for all other legal issues with common elements and/or factors. Our first step is to extract and classify sentences in breach of contract court decisions according to type. Such court decisions have five basic sentence types: sentences on contract law, sentences on contract holding, sentences on contract issues, sentences on contract reasoning, and sentences on contract facts. Classifying sentences is an important first step of our long-term research goal: developing a machine learning system that analyzes hundreds or thousands of past breach of contract court decisions similar to the case at hand, thus providing a powerful tool to legal professionals when faced with new cases and issues. Further downstream processing can be made possible by this project, such as constructing decision trees that predict the likely outcome for the case at hand, displaying the rationales on which court decisions are based, and calculating the similarity of previous legal precedents.
An understanding of the document structure of breach of contract court decisions is crucial to this research. Ontologies, or formal representations, have been created for many applications of diverse academic disciplines, including Artificial Intelligence and Philosophy. An ontology of the target court decisions, which formally defines the components and their relationships thereof, is therefore ifrst constructed from relevant legal statutes, likely assisted by human legal experts. The constructed ontology will later be utilized in the machine-learning phase of the system.
Court decisions are typically written in natural languages, such as English and Chinese; hence, the proposed system must be able to process natural language. At this early stage of the research we focus on English. In recent years, Natural Language Processing (NLP) has made significant progress. Years of research in linguistic and NLP have produced many industrial-strength NLP Python libraries. Due to its ease of use and generality, spaCy [10] is chosen for this research project. Chosen court decisions are first preprocessed with spaCy. After that, the court decisions, and the information added by spaCy, are fed into the machine learning component of the system.
The rest of the paper is organized as follows. Relevant past
research activities are presented in Section 2, which also puts our
research into a boarder perspective. The ontology of breach of
contract court decisions is given in Section 3. Classification of the
sentences with respect to the chosen sample court decisions, based
on our implementation of the logistic regression algorithm and
that of scikit-learn [
The organization and formalization of legal information for
computer processing to support decision making and enhance search,
retrieval, and knowledge management is not recent; the concept
dates back to the late 1940s and early 1950s, with the first legal
information systems being developed in the 1950s [
To address these challenges, expert system developers have
turned to legal ontology engineering, which converts natural
language texts into machine extractable and minable data through
categorization [
NLP can be employed to assist in legal ontology engineering; NLP
is an area of research and application that explores how computers
can be used to understand and manipulate natural language text or
speech to do useful things [
In terms of Computer Science, an ontology is a conceptual model for a certain system, which defines its components, and the attributes of, and the relationships among, those components. Following the notation of [11, 14], the ontology created for this paper is shown in Figure 1.
Each rectangle in the model denotes a set of objects. The rectangle “Breach of Contract Court Decision” represents the set of all the breach of contract court decisions in the state of Alabama. A filled triangle denotes a whole-part relationship, which means that an object of the rectangle connected to the apex of the triangle is a whole while an object of the rectangle connected to the base of the triangle is a part. Hence, a breach of contract court decision is composed of many sentences. An open triangle denotes a superset-subset relationship where the set of objects connected to its apex is a superset of the set of objects connected to its base. Thus, the set “Sentence” has six subsets: “Title”, “Law”, “Holding”, “Fact”, “Issue”, and “Reasoning”. The plus sign + inside of the open triangle means that in addition to the superset-subset relationship, no two subsets can have any common element. This means that no two of the six subsets intersect. A line in the model denotes a relationship set between the rectangles (object sets) connected by the line. A relationship set contains all of the relationships of interest between the set of objects located at one end and the set of objects located at the other end of the line. The connection point of a relationship set and a rectangle is associated with a participation constraint. A participation constraint 1:* means that the minimum is 1 and the maximum is *, where * can mean any positive integer. A participation constraint 1 is a shorthand symbol for 1:1. The pentagon in the figure means a 5-way relationship set, rather than the usual binary relationship sets.
Breach of Contract
Court Decision 1:* Plaintiff 1:*
1:* 1:* Defendant 1:* 1
Sentence Fact 1:*
Law 1:*
Holding 1:* 1:* Reasoning 1:*
Title
1 Issue
Title sentences are not as useful as the other five categories because a title sentence is a part of a court decision’s title or format, although it might contain useful information, like the names of the plaintifs and defendants.
To justify the other categories of sentences, note that legal data is composed of written text that is broken down into sentences. Sentences in legal texts can be further categorized into five sentence types relevant to legal analysis: (1) fact sentences, (2) law sentences, (3) issue sentences, (4) holding sentences, and (5) reasoning sentences. In essence, legal texts arise from disputes between two or more parties that are ultimately decided by a third party arbitrator, which in most cases is a judge. In this study, machine learning will be strictly applied to case law legal texts, which are written decisions by a judge that detail a dispute between two or more parties and the application of law to the dispute to reach a conclusion. In each case decision, there are facts that form the background of the dispute. Fact sentences contain the relevant facts as determined by the judge that underlie any written decision. This is discretionary in nature and relies on the judge to correctly evaluate the facts underlying the dispute and explicate them. Law sentences are statements of law that the judge deems applicable to the dispute. Law sentences are always followed by legal citations. Issue sentences are statements made by either party in the dispute, or by the judge, that can be (a) assertions made by either party of what are correct applications of law or true determinations of facts, or (b) statements by the judge of what he or she determines are the relevant issue(s) underlying the dispute, and how he or she frames such issue(s). Issue statements include the arguments that either party makes to support their respective interests; such sentences are not considered fact sentences or law sentences because only the judge is able to make dispositive statements of fact and law as the final arbitrator of the dispute. Holding sentences are determinations by the judge that apply the law to the dispute and reach a conclusion, also known as a case holding. Holding sentences are the judge’s determination of how the law is applied to the facts of the dispute and the corollary conclusion that sides with one party or the other. Holding sentences are actually considered new statements of law as application of law towards a new set of facts—as no dispute is identical—made by a judge that sets precedent for future disputes; holding sentences are arguably the most important sentences of a case decision as such sentences are the only operative sentences that have bearing on future disputes. Only direct application of law to a present set of facts is considered a holding of the case and binding precedent. Reasoning sentences are statements by the judges that explain how he or she reached the conclusion that he or she did in the case. Reasoning sentences will typically detail the steps the judge made to reach his or her conclusion, interpretation of law and/or fact, comparison and diferentiation of prior case law, discussion of societal norms, and other analytical methods judges utilize to reach conclusions. Holding sentences and reasoning sentences are similar, but can be distinguished in the regard that holding sentences are direct applications of law to the present facts of the present dispute, whereas reasoning sentences do not provide any direct applications of law to present facts of present disputes and only explain the mental gymnastics the judge has performed to reach his or her conclusion. At times it is even dificult for legal professionals to distinguish holding sentences from reasoning sentences, but any hypothetical application of law to fact is considered reasoning. As such, the five types of sentences are related in a nontrivial way. Hence, the five-way relationship set in Figure 1 denotes such a complicated relationship among the five categories of sentences. 4 4.1
In this early stage of our research project, three sample breach of contract court decisions are considered. As the research progresses, more court decisions will be added.
(1) 439 So.2d 36, Supreme Court of Alabama, GULF COAST FABRICATORS, INC. v. G.M. MOSLEY, etc., 81-1042, Sept. 23, 1983. (2) 256 So.3d 119, Court of Civil Appeals of Alabama, Phillip JONES and Elizabeth Jones v. THE VILLAGE AT LAKE MARTIN, LLC, 2160650, January 12, 2018. (3) 873 F.Supp. 1519, United States District Court, M.D. Alabama, Northern Division, Kimberly KYSER-SMITH, Plaintif, v. UPSCALE COMMUNICATIONS, INC., Bovanti Communication, Inc., Sally Beauty Company, Inc., Defendants, Civ. A. No. 94D-58-N, Jan. 17, 1995.
The industrial-strength NLP library spaCy is chosen to extract the sentences from the selected court decisions because of its ease of use and cleanliness of design [10]. 4.2
We have not only implemented the logistic regression algorithm,
but have also compared our implementation with the one provided
by scikit-learn [
self.probs[k] = self.activationProb(X, self.ws[k]) self.results = [""] * numSamples for i in range(numSamples): maxk = 0 for k in range(len(self.ansList)): if self.probs[k][i] >= self.probs[maxk][i]:
maxk = k self.results[i] = self.ansList[maxk] return np.array(self.results)
Regarding the three sample court decisions, the parameter w of the function activationProb is a one-dimensional Numpy array of 1862 integers. w[1:], of size 1861, stores the current weights for the 1861 key words and w[0] is the constant term of the net input z. The function activationProb first calculates the net input z by computing the dot product of the rows of X and w. In the next step the function calculates the probabilities ϕ(z) for the sample sentences (rows) of X based on the net input z. (Technically, the function calculates the conditional probability that an input sentence has a certain sentence type given the sample sentences in X. However, they are simply called probabilities in this paper to avoid being verbose.) It then returns the one-dimensional Numpy array phiOfz, which has the activation probability for each sample sentence in X.
The function fit_helper receives three parameters: the same twodimensional Numpy array X, the one-dimensional Numpy array y that stores the correct outputs for the sample sentences of X, and the one-dimensional Numpy array w that has the weights and the constant term calculated for the 1861 key words. The function first calls activationProb to calculate the probabilities for the sample sentences of X. It then calculates the output for each sample sentence: 1 if the probability ≥ 0.5; 0 otherwise. The errors for the sample sentences are then calculated accordingly. The most important part of the function is the calculation of the slopes (partial derivatives) of the current weights, which will be updated in the next line. The constant term w[0] is updated after that.
Since there are 1861 weights, there are 1861 slopes. The ideal
scenario is that each slope will become zero in the calculation,
which is practically impossible. We thus allow an allowance for such
a small discrepancy, which is stored in self.slopeErrorAllowance.
Thus, the total allowance for 1861 slopes is self.slopeErrorAllowance
* 1861 (= X.shape[
The function fit accepts two parameters: X the two-dimensional Numpy array, and aL the one-dimensional array that stores the correct sentence types of the sample sentences of X. It first finds the unique sentence types in aL. Then, it applies the one versus the rest approach for each unique sentence type to calculate the weights and the constant term for the 1861 key words. To do so, each appearance of the sentence type in y, which is a copy of aL, is replaced with a 1 and 0 elsewhere. The function fit then calls self.fit_helper to calculate the 1861 weights and the constant term for that particular sentence type, which will be used to make predictions.
The calculated weights and constant term for each sentence type are stored in self.ws, which is a 6×1862 Numpy array. To make a prediction for a sentence, we use the predict function, which calculates the probability of each sentence type on the input sentence and assigns the sentence type that has the greatest probability to the sentence. 4.2.3 scikit-learn’s implementation of the logistic regression algorithm. scikit-learn’s implementation of the logistic regression algorithm can be straightforwardly applied. We first randomly shufle the 529 sentences and their corresponding sentence types. We then apply the function train_test_split to split the 529 sentences and their sentence types into two sets: 370 training sentences and 159 testing sentences. Afterwards, a logistic regression object is created and fitted with the training data. 4.2.4 Discussions. To compare our implementation and that of scikit-learn for the logistic regression algorithm, we randomly select 370 training sentences and 159 testing sentences from the 529 sentences of the three sample court decisions and feed them to our implementation. The results are shown in Table 1, which indicates that our implementation has more correct predictions than the highly optimized implementation of scikit-learn. However, since we are familiar with our code, our implementation can serve as a platform for future improvements.
Our result is remarkable if we compare our implementation with randomly guessing the sentence types for the 159 sentences with six possibilities for each sentence. Note that the probability of randomly making a correct guess is p = 1/6 = 0.166667. Assuming each guess is an independent trial, making exactly 87 correct predictions and making 87 or more correct predictions should follow the binomial distribution. Thus, applying Microsoft Excel’s binomial distribution statistical functions BINOM.DIST and BINOM.DIST.RANGE to the problems, we obtain the probabilities in Table 2.
The probabilities for both events are close to zero. Hence, our keyword approach to capturing the characteristics of the sentences of breach of contract court decisions is on the right track, although much improvement remains to be made. or 87 or more correct predictions for the 159 testing sentences 9.08471E−28 1.08519E−27 5
In our experiments, we discover that if the sum of the absolute values of the slopes does not reduce to zero, it usually will go through cycles with respect to the number of iterations determined by the number self.n_iter, which is current set to 500. Our next goal is to find the right number for self.n_iter so that the sum of the absolute values of the slopes is as small as possible.
The ontology in Figure 1 requires more details. More details will result in more accurate characterization of the five types of sentences, which will lead to more accurate classification algorithms.
The word-vector approach of calculating the similarity of two sentences might yield good results. Thus, we will utilize Word2vec, which is one such algorithm for learning a word embedding from a text corpus [16]. 6
In this early stage of the research, we focus on breach of contract court decisions. Five critical contract sentence types have been identified: contract fact sentences, contract law sentences, contract holding sentences, contract issue sentences, and contract reasoning sentences. spaCy, an NLP Python library, is used to parse the court decisions
Three sample breach of contract court decisions are chosen to test our own implementation and the highly optimized scikit-learn implementation of the logistic regression algorithm. Our implementation consistently has more correct predictions than the one provided by scikit-learn and it can serve as a platform for future improvements. Our keyword approach is a good starting-point because the number of correct predictions is far greater than the number produced by randomly guessing the sentence types for the 159 testing sentences. Lastly, many possible future improvements have also been identified.