Imagine a lawyer invoking in a bankruptcy case as an argument in favour of her client the outcome of a natural language processing (NLP) and machine learning (ML) algorithm that analyses past judicial decisions in bankruptcy cases, and ‘predicts’ outcomes, taking into account many different dimensions of bankruptcy cases – such as assets, liabilities, debtors, creditors, economic sector, the judge and the lawyers involved etc. These technologies give insights in patterns and probabilities, whereas adjudication is about arguments, reasoning and giving reasons for a decision.
This paper addresses the question how courts, if at all, can integrate such outcome in their practice. How should a court approach a case where a plaintiff or defendant uses the outcome of NLP and ML technologies as an argument? Is the outcome of such an NLP and ML algorithm admissible in a court procedure? If so, what is its value? How can a court engage with it? This essay argues that such outcome is not to be considered as a fact in a court case, nor as law, nor as a secondary source of law such as comments or treatise. Furthermore, it claims that it is a sui generis concept, a concept of its own kind, with its own set of questions to be addressed before it can be integrated in a reasoned court decision, deciding on its (in)admissibility and its value.
The paper aims to contribute to the integration of the quantitative methods of computer science, data science and statistics with the qualitative method of the law, taking the rule of law into account. * I thank Prof. Dr. Henry Prakken and Prof. Dr. Kevin Ashley, warmly for their valuable comments on the earlier draft of this paper. I express an equally deep gratitude to my colleague Dr. David Harvey for his detailed and valuable comments on my M.Phil thesis (see n4 below).
Supervised machine learning (SML) and NLP are the technologies on which I mainly focus in this article. Hereafter I replace the term ‘supervised machine learning with case law as an input’ by ‘Legal Analytics’ or ‘LA’. The term ‘outcome of Legal Analytics’ and its abbreviation ‘ocLA’ substitute ‘outcome of supervised machine learning with case law as an input’. This allows for a more fluent reading. In addition, SML with the aim to ‘predict’ court decisions is a part of the broader subject of Legal Analytics, to which intelligent search, legal information extraction and automated casebased and rul-based reasoning also belong. 3
In current practice, courts analyse the facts and the law. In coming to a decision, there is a subsumption of facts and law in a continuous process; facts are selected and analysed from the perspective of the relevant law, and at the same time the relevant law is selected on the basis of the facts of the case. This process can be iterated if necessary, depending on the arguments submitted by the parties. Furthermore, courts in common law countries are bound by stare decisis, the principle ‘that decisions of earlier cases sufficiently like a new case should be repeated in the new case’.1 On the other hand, to create a supervised machine learning algorithm, an input, system and output are necessary. With supervised machine learning, defining the problem implies defining the outcome variable.2 In addition, facts, law, metadata of cases and societal values of a number of cases are selected for the input.3 Information is combined in a different way compared to traditional dispute resolution. In addition, machine learning algorithms sort, search, count and classify according to their requirements and specifications, rendering patterns and probabilities. The results are interpreted, and the outcome constitutes new information that is abstracted from the original cases that were used as an input. Each step in the algorithm-making process uses a methodology that is at least partially different from the methodology used in adjudication, distinguishing existing legal dispute resolution practices from supervised machine learning.4 I claim that it is precisely because of this partial difference in methodology that the ocLA cannot qualify as any of the concepts used in current adjudication.
1. I first argue that the ocLA cannot be considered and treated as a fact.
An issue is one of fact where it concerns the reliability or credibility of direct evidence, or inferences from circumstantial evidence.5 Facts relate to the case that is being heard. They are often concrete and must be proven by evidence specific to the case and given by the parties. In principle, the facts of a case cannot be proven by reference to facts in other cases.6 By contrast, the ocLA gives a correlation between parts of a number of court decisions. As the input for the algorithm consists of an interpretation of both facts and law and metadata of an amount of earlier cases, and as the input was turned into a vector and a statistical analysis was performed, the ocLA is about more than a concrete fact or set of facts in a concrete case. For instance, the concrete trademark and its use by the trademark holder are specific to a case against an alleged infringer. The ocLA of a number of trademark cases rendering a prediction on infringements in similar cases constitutes information that, for instance, allows to compare the given case to a larger number of cases, and thus might provide a new context for a concrete case, but does not meet the legal standard of proof in a given dispute. Also, the input for and the methodology of the algorithm discern it from a mere statistical analysis of a phenomenon. The latter can be analysed as a fact, whereas the former has, as discussed, aspects to it which do not allow to reduce it to a fact.
To what extent could the ocLA be equated to expert evidence, if at all? Evidence involving a technological or scientific aspect and the ocLA both have their technicality in common. The skilled witness must demonstrate to the court that they have relevant knowledge and experience to give either factual evidence, which is not based exclusively on personal observation or sensation, or opinion evidence.7 Where the skilled witness establishes such knowledge and experience, they can draw on the general body of knowledge and understanding of the relevant expertise.8 Hence, the question must be examined whether the ocLA could receive the status of a ‘non-human skilled witness about the general body of knowledge and understanding of the relevant expertise’. I argue that this is not the case. First, new technologies represent information in a novel way, involving interpreted facts and the applied law from earlier cases, which is epistemologically as well as methodologically different from a general body of knowledge and understanding of the relevant expertise.9 Therefore, the ocLA cannot receive the same treatment as expert evidence currently receives in court procedures. For the same reason, the ocLA is different from expert opinions on foreign law. By analogy, the use of technology assisted review (TAR) or e-discovery is considered by the U.S. District 5 S.H. Bailey and M.J. Gunn, Smith & Bailey on the modern English legal system (Sweet &
Maxwell, 3d ed., 1996) 415. 6 There are exceptions in case of established facts of criminal cases, a discussion of which is beyond the scope of the paper. 7 Myers v The Queen [2015] UKPC 40 [2015] 3 WLR 1145, para 63. 8 Ibid. 9 See G. Vanderstichele, (n3), Chapter 1. Court for the Southern district of New York as a tool to assist the parties in the discovery process,10 and not as a fact in itself.11 The ML algorithm is a tool with the possibility to assist in dispute resolution. Second, in the UK experts can and often do give evidence of fact as well as opinion.12 However, in the UK, as well as in civil law countries, it is unlawful to shift the responsibility for deciding the issues brought before the court to an expert witness,13 be it a scientist of any kind, or a computer engineer. By law, such decisions belong to the competence of the court.14 The input for the ML algorithm is in part the law, and a ML algorithm requires selective decisions about the facts and the law, by selecting input data, requirements, specifications, a specific ML algorithm, etc. Therefore, equating the ocLA to expert testimony would equate to the court shifting their decision-making responsibility, which goes against the fundamental principle ‘iura novit curia’ or ‘da mihi factum, dabo tibi ius’,15 applicable in both civil and common law systems. Finally, it can be questioned whether evaluating new technologies exclusively from the perspective of scientific peer review sufficiently respects the principle of due process. Parties ought to have the possibility to adequately contest the algorithms,16 by questioning them without intermediation.
Moreover, for the first and second reasons it would not be legal to have a human expert witness explain the methodology and the significance of the ocLA to the court, as this would amount to explaining the court what the law and its application is, which is the task of the courts. At least, lawyers and judges ought to have a sufficient understanding of the involved quantitative methodologies to be able to question thoroughly an expert witness with regard to the invoked ocLA in a court procedure. This poses a question with regard to legal education: what exactly do lawyers need to study, in addition to the substance of the law? It also raises the question about organising the time of sitting judges in order to allow them to acquire new knowledge and skills.
The technical nature of algorithms and their outcome, might incite to consider the outcome as a fact with special authority, as has been done to establish social science in law.17 In that concept, findings of social science research that are relevant to create a rule of law are treated as a source of authority, rather than a source of facts, and treated in the same way as legal precedent.18 The outcome of social science research would have similarities with facts, as it is empirical research, and with the law, as both research findings and the law are general; findings of social science research extend beyond the 10 Rio Tinto PLC v. Vale S.A. [2015] (S.D.N.Y.) WL 8722294. 11 See infra for further discussion. 12 Kennedy v Cordia (Services) LLP [2016] UKSC 6, [2016] 1 W.L.R. 597, para 40. 13 See e.g. Kennedy v Cordia (Services) LLP [2016] UKSC 6, [2016] 1 W.L.R. 597, para 49; Advisory Opinion on the Western Sahara, [1975] I.C.J. Rep. 12, 138; Article 112.1 Dutch constitution; Article 40 and 144.1 Belgian constitution. 14 Ibid. 15 Resp. ‘the court knows the law’ and ‘give me the facts and I shall give you the law’. 16 Rebecca Wexler, Life, Liberty, and Trade Secrets (2018) 70 Stan. L. Rev. 1343, 1375. 17 John Monahan and Laurens Walker, Social Science in Law. Cases and Materials (Foundation
Press, 1st ed. 1985, 7th ed. 2010) 387. 18 Ibid. 388. concrete facts involved in the study.19 However, this special status for social science findings is not necessary to allow them in court. They are related to empirical facts and can be analysed as such. Even when scientists speak about ‘scientific laws’, they remain empirical findings. The arguments discussed above with regard to expert evidence in general are valid here as well.
2. Second, the ocLA is not law as we know it today, for the following reasons.
First, the ocLA is conceptually too different from current precedent to be equated. The ocLA renders a probabilistic outcome after a statistical treatment of data, which is very often not transparent. In general, statistical outcomes of machine learning algorithms rendering an estimation of x%, compared to the chosen baseline, of what a court might decide, call for the conclusion that predicting how a court might decide in a future case may be sufficient for a lawyer to tentatively advice and orient their client who would be willing to estimate what their chances are of winning or losing a case, but is not a precedent in common law, nor a statute or an authoritative court decision in civil law countries. Importantly, LA do not give reasons for their outcome, while reasons are precisely what doctrine of precedent and jurisprudence require to qualify a court decision as law. A proposition of law in precedent is only binding when it forms part of the ratio decidendi, i.e. the reason(s) for the decision of the court: ‘As with judicial or other opinions, what carries weight is the reasoning, not the conclusion’.20 Furthermore, Article 6 §1 of the European Convention for the Protection of Human Rights and Fundamental Freedoms (ECHR) prescribes that in the determination of their civil rights and obligations or of any criminal charge against them, everyone is entitled to a fair and public hearing within a reasonable time by an independent and impartial tribunal established by law. The guarantees enshrined in Article 6 §1 include the obligation for courts to give sufficient reasons for their decisions. A reasoned decision shows the parties that their case has truly been heard.21 Additionally, the Convention and the case law holding the proposition that reasons are essential in adjudication are supported by the jurisprudence of Joseph Raz. According to his theory, all normative phenomena are normative in as much as, and because, they provide reasons or are partly constituted by reasons.22 For Raz, courts ought to act on exclusionary reasons, which are legal standards23 and court decisions are issued for reasons which are deemed to justify them.24
Such reasons are absent in Legal Analytics. At best, they can be inferred from the computation, which requires an additional reasoning. Without the justifying reasons, court decisions lose their reason for action and thus their authority. At least, a theory explaining as to why the ML is valid is needed in addition.
While it is beyond the scope of this paper to discuss the ongoing research on explainable AI,25 it is relevant for the research question to elicit that explaining an algorithm by referencing the optimization process is not an actual explanation as it does not tell more than ‘the algorithm says so’, which does not meet the legal standards in an individual court case.26 Also, there is no supervised machine learning that can assess for each individual subject to which a running model is applied exactly to what about that particular individual or subject caused its prediction. It is not possible to say exactly how changes in a certain input variable’s values for that individual would have yielded a different prediction, holding all of the individual’s other input variable values constant.27 If, for instance, in a trademark case one would change the location of the trademark holder, then it is likely that other relevant input variables, such as the type of business or its name, would be different as well. Even when one does not consider the model of the algorithm as static, but as an ongoing process, and uses an importance measuring technique such as explaining what the most important variables were for a given individual’s predictions,28 under current law these explanations should be incorporated into an argument, as they are reasons for estimations and not per se reasons for an individual decision. The same is valid for the method that describes how increases or decreases in the various input variables translate to changes in the outcome variable.29 Regardless, these methods are not always available,30 reason why informed model selection is an important explaining element as well.31
Second, context is essential to precedent.32 In civil law countries the context of the dispute is essential as well to come to a correct decision. By contrast, supervised machine learning algorithms remove context from the decisions used as an input, taking away an essential feature of precedent and stare decisis. Alternatively, it can be argued that they create a new and different context. However, that is on a more abstract level, and it requires additional discussion and reasoning, as argued supra.
Third, there is a legislative limitation on the equation of the ocLA with precedent by itself. Article 22.1 of the GDPR states: ‘The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.’ I claim that on the basis of this provision courts can engage with the ocLA, without however resorting to merely referencing it, as this could amount to basing the decision solely on automated processing. Courts rubberstamping a decision in an individual case, decided solely on the basis of an ocLA violate the provision of Article 22.1 GDPR.
There are conditional exceptions in the following sections of Article 22 GDPR. First, explicit consent is such an exception. This legal ground is for instance used in current Dutch court cases about health care insurers who obtained explicit consent from their contracting patients to handle disputes by private, partly automated courts. Upon discussions whether such consent can be qualified as really voluntary, and whether the due process requirement of article 6.1 ECHR was respected, the health insurers almost entirely discontinued their project on private dispute resolution.33 In addition, automated decision-making could be authorised by Union or Member State law, on the condition that suitable measures are taken to safeguard the data subject’s rights and freedoms and legitimate interests. Thus, a legal intervention is arguably required before the ocLA would be admissible, let alone before it could obtain normative value. Also, it will have to be defined what such suitable measures would be.34
Although it is not a law, I point to principle 5 ‘Under user control’ of the European ethical Charter on the Use of Artificial Intelligence in judicial systems and their environments of the European Commission for the Efficiency of Justice (Cepej), as it illustrates a line of thinking about the research question and is intended for AI developers as well as ‘public decision-makers’.35 The 5th principle states among others36 that ‘professionals in the justice system should, at any moment, be able to review judicial decisions and the data used to produce a result and continue not to necessarily be bound by it in the light of the specific features of that particular case’.37
For the above mentioned three groups of reasons, the ocLA cannot qualify as precedent by itself, but needs further discussion in and consideration by the court to be useful in dispute resolution.
3. Third, the outcome of machine learning algorithms is not to be equated with a secondary source of law for the following reasons.
The ocLA and secondary sources of law, such as comments on the law, have in common that they both analyse the facts and law. The ocLA might in addition have metadata as an input. Some information on lawyers or judges, not related to the law or the facts of the cases that are commented, can be incorporated in a classic comment as well, be it not as a precise parameter. However, the difference in methodology comes again to the forefront. A comment discusses the case in its context. Part of the discussion of a comment may provide the reader with a different view on the context. By contrast, supervised ML algorithms initially remove context. The ocLA possibly constitutes a whole new result that can be used as a new element of context, but this comes after the numerical treatment and is not part of it. The ocLA is a different set of observables, compared to the observables of law, facts and metadata from the input. The algorithm creates a proxy for reality.38 By contrast, even when the classical comment sheds a new light on the content, the set of observables remains unaltered. In addition, a comment offers an interpretation after the analysis, whereas the Legal Analytics render patterns, with at least two instances of interpretation. There is a preliminary interpretation that was built into the system through the selection of the facts, or the requirements of the system by humans. Afterwards, the outcome is open for a second analysis and interpretation, again by humans. Finally, a comment or treatise is most often focused on a qualitative analysis, whereas a supervised machine learning algorithm renders a quantitative analysis. Hence, I conclude that the ocLA cannot qualify as a secondary source of law.
4. Fourth, if the ocLA cannot be qualified as any known category or concept in current legal practice, be it a fact, or law, or a comment, I claim that by elimination the only remaining logical conclusion is that the ocLA ought to be considered as a sui generis concept in dispute resolution, a concept of its own kind.
With regard to the terminology, I propose to keep using the term ‘outcome of Legal Analytics’ or ‘ocLA’, at least for the time being. It is preferable to the expression ‘synthetic facts’. Although, the word ‘synthetic’ might be useful to refer to the aggregation of metadata and data, encompassing facts and law, limiting the term to ‘facts’ renders it inaccurate, since Legal Analytics take as an input data that not only come from the facts of court cases.39 The same argument is valid for the term ‘derived facts’. ‘Synthetic precedent’ and ‘synthetic case law’ imply normative value, which it hasn’t as such. Moreover, ‘synthetic legal information’ is unnecessarily vague. Finally, ‘quantitative legal argument’ probably comes closest to what the outcome could be in dispute resolution. However, it is too early to fix the concept on a narrower term than ocLA. 4
Introducing the ocLA as a sui generis concept in court proceedings has several
advantages. The first important advantage that it allows for conceptual clarity; the parties
and the court can consider the ocLA at its own level of abstraction, distinct from the
other parts of a dispute and its resolution. It furthermore allows to reason and to argue
with this new kind of information. The sui generis concept encourages the parties and
38 Patrick Allo ‘Mathematical values and the epistemology of data practices’, Emre Bayamlioglu
et al. (eds.) Being profiled: cogitas ergo sum (AUP, 2018) 20, 23.
39 See the discussion in G. Vanderstichele (n3) chapter 1, section 2.
the court to explicitly ask specific questions concerning the different aspects of the
input, the system with its different aspects of the software cycle, and the output. The
parties can develop separate arguments e.g. with regard to the number of cases that
were used, the kind of data and metadata, the cleaning of the data, the requirements
submitted to the software developers, the chosen model with its design decisions40, the
proxies that were used, the outcome, the inferences made, etc. A good documentation
of all elements of the NLP and ML algorithm and its outcome will facilitate and
enhance the discussion in court. Parties and judges probably will also discuss related
issues such as problems of privacy, reasonable inferences, datafication, opacity,
intellectual property rights v. transparency, ‘gaming’ the system, etc.41 Third, treating the ocLA
as a concept of its own kind in the court procedure helps preserving the rights of due
process and fair trial in an information society.42 One of the dangers of the argument
that ‘courts decide x in 79% of this kind of case/legal issue’ is indeed that parties and
judges fall in the trap of automation bias – the propensity to uncritically favour
automated decision-making systems, only because it is technology – as well as the
McNamara fallacy – decision-making based solely on quantitative observations, while
ignoring important observations of another kind43. By considering the ocLA at its own
level of abstraction, these traps, which would violate the rights to due process and fair
trial, can be averted. The entire algorithm can receive the required adversarial
discussion in a court case, before the judge rules either to admit the ocLA in the proceedings
or to declare it inadmissible, and in the former case to decide what weight to accord it
in the proceedings. In addition, it permits to give specific reasons for a judicial decision,
which is essential in common law,44 as well as civil law45 systems. Moreover, the sui
generis concept offers a method to safeguard human oriented values in adjudication,
and in society in general, as opposed to a reductionist approach of conflict, as it is the
human judge who has the proverbial last word. The judge adds context by studying the
case and analysing it for facts or legal questions a programmer could not have foreseen,
compares their analysis with the ocLA, and corrects where necessary. In general, it is
an element in preserving the rule of law when Legal Analytics are used in
adjudication.46
40 See e.g. Remi Wieten, Floris Bex, Henry Prakken, Silja Renooij, ‘Supporting discussions
about forensic Bayesian networks using argumentation’,
<https://dl.acm.org/citation.cfm?id=3326710> accessed
BMJ 339 <https://doi.org/10.1136/bmj.b3141> accessed
From the above, it becomes clear that the ocLA is an additional factor for judges to consider in the decision-making process.47 Courts presented with an ocLA will interpret facts and law as they do today. They will furthermore hear the parties’ thorough analysis of the algorithm they invoke. Once the question about a sufficient correctness of the algorithm48 is established, the court will confront the outcome of the concrete interpreted facts and law with the ocLA and it might reinterpret its preliminary finding on the basis of the new information. A subsumption of principles of computer programming, statistics, law and facts will take place. At the same time, law is put into the ocLA.
Common law courts might establish a precedent that involves an algorithm. In such a way the algorithm does not have normative value in itself but acquires at least indirectly normative value as it is interpreted in a given set of facts and law. The ocLA will have to be used in a specific context in order to become a precedent, giving the ocLA limited precedential value as, and only as, tied to the facts of the case. This would add context to the data. It would interpret the data and reconnect them to society. Additionally, this modus operandi would preserve the fairness, consistency and predictability precedent provides.49
In civil law systems, as the law is created by Parliament, a statute is required for an algorithm to acquire the force of law. Alternatively, the legislator could establish a regulatory body with the power to grant licenses for tested and approved algorithms.50 However, I claim that courts should not necessarily declare the ocLA inadmissible as long as there is no statutory intervention of a kind. The applicable code of civil procedure must be applied. Where there is sufficient possibility for interpretation, the ocLA as an argument of its own kind could be admitted in the procedure and subjected to discussion by the parties.
It ought to be clear that I have proposed here a method for judges to engage critically with the ocLA when submitted in court, as a portal to a further discussion about the content and the methodology of the algorithm. However, a comprehensive discussion of the relevant questions and criteria for assessment of the ocLA in court is outside the scope of this contribution and will be a research project in itself.
Therefore, I offer in this article only the following general remarks with regard to the content of a further argumentation and reasoning in court. It is to be foreseen that using only ocLA with case law as an input will have a rather limited use. First, the quantity and the quality of the data used as an input for the natural processing and machine learning algorithm is crucial. This poses a problem to the extent that access to court data is limited in many countries, and access to the submitted evidence is even more restricted. More specifically, whenever the algorithm is made with court cases as an input, and not with the entire case file, its input is the facts as interpreted by the courts and not the full facts as presented to the courts. It is to be taken into consideration that there might be a difference between the ‘raw’ facts of a case and the withheld facts in the court’s decision.51 Moreover, if ‘wrong’ court decisions, i.e decisions that do not uphold a rule of law are used as an input, the outcome will be legally wrong, even when the prediction is correct. Hence, it is valid to point to the limited value of the accuracy measure of an algorithm. In a legal context this measure can be misleading. On its own, it does not allow to conclude that the outcome will predict in a legally correct way the outcome of a new individual case. Additionally, it has to be repeated that NLP and ML algorithms render patterns and probabilities. They do not give reasons, whereas reasons are essential in adjudication in the common law52 as well as in the civil law53 traditions. Also, NLP and ML algorithms with case law as an input do not seem to be the appropriate tool to address legal cases with complex and multiple legal issues. This approach to Legal Analytics could be used for a system to assist judges in processing cases where the legal question is relatively straight forward, and the variables are restricted to a limited number of rules and facts involved, and where the intervention of a judge is required as a safeguard for fundamental rights such as due process. For instance, it could be used to select ‘outliers’ for (faster) treatment on appeal.54 Finally, the quality of the program, the training and the testing of the data also determine the quality of the NLP and ML algorithms. 5
I discussed above a method for parties and judges to examine in a court case the outcome of an NLP and ML learning algorithm with case law as an input. I proposed to discuss the outcome of a machine learning algorithm with case law as an input at its own level of abstraction in court cases. In adjudication, it is a sui generis concept, preliminary called ocLA, to be distinguished from facts, law and secondary sources of law. By considering it as a concept of its own kind, parties and judges can reason with this new kind of information.
This method is taking into account that courts operate today in a mixture of the socalled printed press paradigm, where the affordances of the printed press determine the rule of law and its application in adjudication, and the digital paradigm, whilst legal analytics are evolving towards a fully digital paradigm. For instance, several scholars are researching systems that imitate the human back and forth argumentation in court.55 Also, researchers create algorithms that account for legal values in dispute resolution.56 The latter allows for a so-called empirical law making, as an algorithm can be ablated to simulate the effect of values on the outcome. As courts will further develop their role into that of an interface between legal analytics and humans in a fully digital paradigm, I claim that the proposed method probably will be less used to the extent legal analytics are accepted in practice but will remain valid for judicial review of algorithms. Future research will examine what the benchmarks are for assessing the outcome of Legal Analytics, in light of the rule of law.