In this paper we aim to discuss the current challenges of legal science caused by the diffused use of AI software in various legal operations (verification, drafting, risk analysis and prediction). Working on the threshold between legal philosophy and computer engineering, we are going to address especially one problem, i.e. how the meaning of terms used in legal documents might be fixed differently because of the progressive and increased use of AI software by legal professionals (lawyers, judges, notaries). To do so, we are going to refer to Herbert Hart's idea that all legal concepts - via the terms referring to them - always have a core of settled meaning, but are also characterized, as well, by a penumbra of debatable cases in which words are neither obviously applicable nor obviously ruled out. These are so-called “hard cases”. Here are some questions that we aim to address: is it possible to anticipate the potential emergence of hard cases and hence prepare legal software to deal with the core-penumbra problem in legal meaning? How can machines perceive the relevance of the context and of societal change (fundamental aspects in Hart's legal theory of meaning)? 0000-0001-9709-6392 (G.M. Di Nunzio); 0009-0009-6299-5991 (A. Condello)
In recent years, the "algorithmic turn" in legal science has gained traction, significantly altering
the landscape of legal operations with the adoption of AI across a multitude of functions, from
document automation and contract drafting to complex judgment predictions and risk analyses [
In examining the integration of AI within legal operations, we confront both “direct” and “indirect” consequences that affect legal reasoning. Direct consequences, such as errors, biases, and so-called hallucinations in AI outputs, highlight immediate risks of embedding prejudices within predictive algorithms. However, a concern lies in the indirect consequences, notably the risk of semantic un-flexibility which could suppress the interpretative adaptability that legal concepts require for responding to complex concepts.
Given these challenges, this paper seeks to add to the growing discussion about AI’s role in legal science by asking whether AI systems can be developed to accommodate the different degrees in the interpretive needs of legal contexts. Can these technologies effectively distinguish between clear-cut cases and more ambiguous ones, and adapt to evolving contexts and interpretive shifts? We explore how an approach combining legal theory, terminology, and computer engineering may help address these issues, allowing AI to support the interpretive complexity of legal work while preventing the risks of overdetermined or overly narrow legal meanings. 3rd Workshop on Augmented Intelligence for Technology-Assisted Review Systems (ALTARS 2024): Evaluation Metrics and Protocols for eDiscovery and Systematic Review Systems. March 28, 2024, Glasgow, UK.
giorgiomaria.dinunzio@unipd.it (G.M. Di Nunzio); angela.condello@unime.it (A. Condello)
In this paper, we explore the multifaceted challenges facing legal science in light of the
widespread adoption of AI software across legal operations such as document automation and
verification, contract drafting, risk analysis, judgment, and prediction.2 In this interdisciplinary
context, between legal philosophy and computer engineering, the focus of our proposal will narrow
down onto a pivotal issue: the evolving dynamics of legal terminology and the consequences for
legal interpretation and argumentation, due to the pervasive and ever-increasing use of AI software
by legal professionals, including lawyers, judges, notaries, police forces, and labor consultants (see,
among others, [
Central to our investigation is Herbert Hart's theoretical framework [
The integration of AI software in legal practice raises critical questions that we want to explore; in particular, whether it is conceivable to anticipate the potential emergence of hard cases and deep interpretive disagreements (see [6]) and, subsequently, prepare legal software to navigate the intricate core-penumbra problem inherent in legal meaning, considering both the vagueness and the often necessary indeterminacy of legal terms, which are necessary for the adjustment of general prescriptions to singular cases. [7] Working at the intersection between legal theory, terminology and computer engineering is timely in particular given the supranational dimension in which legal actors and decision-making bodies operate. In addition, the increasing sophistication of digital technologies and their availability have generated two divergent narratives about their potential implications, as described by [8]. These narratives alternately express excitement about legal technology’s potential to make the law more efficient and improve access to justice, or concern about the ways in which it may actually exacerbate existing biases or otherwise systematically harm justice.
From our transdisciplinary point of view, two types of issues are generated by the widespread adoption of AI software across legal operations: some of them can be considered as “direct” consequences, while others are more “indirect”. Hallucinations, errors, discrimination fall into the first category, i.e., they are direct consequences of the use of AI software: since the software can confuse situations or judge the potential events of the future in relation to past events, it can also reproduce prejudice and more technically generate biases towards certain categories of subjects. Although these are the consequence more often discussed by legal scholars and practitioners, we believe that working on the terminology (via Hart’s theoretical framework) can demonstrate the focal importance also of what can be described as “indirect” consequences. These are the effects of the probabilistic logic underlying AI software that might produce, across time, a narrowing of meaning and rigidity in legal terms, something that might generate a scientific revolution in law in terms of the determinacy/indeterminacy of legal meaning [9]. Such semantic is, though, essential to the evolution of legal concepts through time and is often the consequence of legal interpretation in cases of deep disagreement and political conflict (see [10]). 2 https://joinup.ec.europa.eu/collection/justice-law-and-security/solution/leos-open-source-software-editinglegislation/discussion/smart-leos-which-new-functionalities-should-be-implemented-next-and-what-can-be-learntcorrigenda
Our research extends beyond the theoretical questions and addresses practical considerations tied to the intersection of AI and legal semantics. In this context, one main issue arises: can automatic systems be trained to foresee the contours of hard cases and adapt to the subtle distinctions of legal meaning? Can we measure the uncertainty of legal concepts and argumentation to handle the conflicts between different interpretations of norms [11]?
This challenge involves understanding how AI systems can effectively discern the relevance of contextual complications as well as societal changes, a task that is very important in the context of Hart's legal theory of meaning and well as in further developments of legal interpretivism until today.
As we examine the implications of AI software on legal terminology, our analysis recognizes the transformative impact on traditional legal practices. The diffusion of AI technologies introduces a paradigm shift, necessitating a reevaluation of established legal methodologies. We explore the potential repercussions of this shift on the interpretation of legal documents and the inherent stability (or instability) of legal concepts. The balance between settled meanings and the penumbra of hard cases becomes increasingly important in a field where AI contributes to legal decisionmaking processes not only via software of predictive justice (simulating the act of judging) but also via semantic searches in jurisprudential materials and operations aimed at clarifying or specifying the content of legal concepts. By examining the core-penumbra problem through the lens of Hart's legal theory of meaning, we shed light on the challenges and opportunities posed by the integration of AI in legal science. Through this interdisciplinary analysis, we contribute to the ongoing discourse on the evolving nature of legal semantics in an era marked by the influence of augmented intelligence. We aim to show that cases such as Loomis (2017, US Supreme Court) or Deliveroo (2020, Bologna Court of First Instance) or the UK Post Office scandal (also known as the Horizon IT scandal) might be avoided, in the near future, thanks to a continuous cooperation between legal theory, terminology and computer engineering aimed at working through direct and indirect risks and consequences of the use of AI software in legal operations. In particular, we aim at (i) foreseeing upcoming potential legal conflicts (via a screening of the typical aspects of hard cases) and (ii) preserving the “penumbra” and the essential indeterminacy of legal concepts.
In this section, we briefly list some possibilities to represent the concept of penumbra from a machine learning point of view. Particularly, in the context of natural language processing (NLP) and decision-making algorithms, the penumbra can be linked to areas of uncertainty or ambiguity where the model's predictions may not be unequivocal.
In exploring the concept of "penumbra" within legal interpretation, we draw parallels to machine learning models that face similar challenges of ambiguity and uncertainty. Just as legal penumbra deals with cases that do not fit neatly into established categories, machine learning models often encounter "gray areas" in their predictions and decisions. The following sections examine this relationship more closely, highlighting how machine learning approaches—whether dealing with prediction uncertainty, boundary cases, context sensitivity, or interpretability—can mirror the interpretative demands of legal reasoning. By analyzing these facets, we would like to uncover how probabilistic and context-dependent models reflect the indeterminate nature of certain legal terms and cases, emphasizing the shared need for flexibility in interpreting complex scenarios.
In machine learning models, especially those based on probabilistic frameworks like Bayesian models, predictions are often associated with a degree of uncertainty [12]. The model may provide a probability distribution over possible outcomes rather than a definitive answer. This uncertainty may reflect the penumbral aspect, where certain instances may fall into a gray area, making it challenging for the model to make a clear-cut decision.
Much like legal penumbra involving hard cases, machine learning models may struggle with boundary cases [13]. These are instances that lie on the edge of the decision boundary, where small changes in input features can lead to different predictions. These boundary cases represent situations where the model's confidence is lower, and decisions may be less straightforward.
In legal terms, the interpretation of a term may vary based on the context in which it is used. Similarly, machine learning models, specifically NLP models, often rely on context to make accurate predictions [14]. The model's understanding of certain terms or features may exhibit variability based on the surrounding context, introducing a level of interpretation flexibility analogous to the legal penumbra.
Interpretable machine learning models aim to provide transparency into how decisions are made [15]. Despite efforts to achieve interpretability, there may still be instances where the model's reasoning is not entirely clear. This lack of clarity aligns with the penumbral nature, where certain cases may defy straightforward interpretation.
In essence, the concept of penumbra in legal science, with its shades of interpretation ambiguity, can find in machine learning models dealing with uncertainty, boundary cases, context sensitivity, and adaptability.
By incorporating the penumbra concept, an eDiscovery system may predict and prioritize hard cases within debatable scenarios and deep interpretive disagreements especially in the case of contested concepts (such as, for instance, life, dignity, self-determination, autonomy). The same idea (“difficult” concepts that are designated by multiple quasi-synonym terms) may be applied to the case of medical review systems. This approach promises to enhance the efficiency of such kinds of systems but, at the same time, may ensure that users are supported, not replaced, in the decisionmaking process, maintaining accountability for actions taken.
This work is part of the initiatives carried out by the Center for Studies in Computational Terminology (CENTRICO) of the University of Padua and in the research directions of the Italian Common Language Resources and Technology Infrastructure CLARIN-IT. [6] V. Villa, Deep Interpretive Disagreements and Theory of Legal Interpretation, in Pragmatics and Law. Philosophical Perspectives, A. Capone and F. Poggi (eds.), Springer 2016, pp. 89-119. https://doi.org/10.1007/978-3-319-30385-7_5 [7] A. Marmor, Defeasibility and Pragmatic Indeterminacy in Law, in Pragmatics and Law.
Philosophical Perspectives, A. Capone and F. Poggi (eds.), Springer 2016, pp. 15-32. https://doi.org/10.1007/978-3-319-30385-7_2 [8] R. Whalen, “Defining legal technology and its implications,” Int. J. Law Inf. Technol., vol. 30, no.
1, pp. 47–67, Mar. 2022, doi: 10.1093/ijlit/eaac005. [9] B. Bix, Law, Language, and Legal Determinacy, Oxford 1995. [10] C. Sunstein, Legal Reasoning and Political Conflict, Oxford University Press 2018. [11] C. da Costa Pereira, A. G. B. Tettamanzi, B. Liao, A. Malerba, A. Rotolo, and L. van der Torre, “Combining fuzzy logic and formal argumentation for legal interpretation,” in Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law, in ICAIL ’17. New York, NY, USA: Association for Computing Machinery, Jun. 2017, pp. 49–58. doi: 10.1145/3086512.3086532. [12] M. Neil, N. Fenton, D. Lagnado, and R. D. Gill, “Modelling competing legal arguments using Bayesian model comparison and averaging,” Artif. Intell. Law, vol. 27, no. 4, pp. 403–430, Dec. 2019, doi: 10.1007/s10506-019-09250-3. [13] K. Atkinson and T. Bench-Capon, “Reasoning with Legal Cases: Analogy or Rule Application?,” in Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law, in ICAIL ’19. New York, NY, USA: Association for Computing Machinery, Jun. 2019, pp. 12–21. doi: 10.1145/3322640.3326695. [14] M. Sosa Andrés, “Legal uncertainty and its consequences : A natural language processing approach.” 2023. Accessed: Feb. 01, 2024. [Online]. Available: https://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-495521 [15] M. M. Farayola, I. Tal, R. Connolly, T. Saber, and M. Bendechache, “Ethics and Trustworthiness of AI for Predicting the Risk of Recidivism: A Systematic Literature Review,” Information, vol. 14, no. 8, Art. no. 8, Aug. 2023, doi: 10.3390/info14080426.