This paper describes an AI-based legal assistant that would support case assessment based on the extraction of arguments from court decisions. Open norms are often used in law as a way to set a legislative frame that allows for individual justice under the specific circumstances. By extracting the underlying arguments from existing corpus of annotated court decisions, a reliable legal framework can be formed in order to give more insight and clarity to both judges and parties in the case. This contributes to legal certainty and equality, without losing justice in individual cases. The system provides legal specialists a practical tool to help their clients in a legal procedure, without the necessity of going through all relevant case law themselves. The general public will also benefit from the available knowledge.
It seems Artificial Intelligence and Intelligent Assistants already
have their place in law firms, although much of this place is devoted
to Document Management Systems and various Information
Retrieval systems. We suggest that under a stricter scrutiny, it would
be clear most of these systems were pushed by the business need to
reduce the cost of support systems, a vital need under the paradigm
of the billable hour [
We observe that most of the Information Retrieval or Natural
Language Processing problems are by large unsolved in the legal
domain. The specifics of the language, or the manipulation of
concepts and abstractions contribute to create this distance between
legal texts and general literature, as observed in [
We make the hypothesis that lawyers internalise a summarised
knowledge of case law, that allow them to assess individual
situations with regards to norms, and that this process can be formulated
as a knowledge extraction and summarising task[
Open norms are general, flexible terms, used to provide justice in
individual cases and are therefore not restricted to specific
situations that need to be written in advance. In this way, yet unknown
cases can always fit in the norm.[
By applying open norms, judges can deliver tailor-made solutions that suit the material status quo of the specific case. This is a rather dificult task. A judge is supposed to carefully collect the evidence, retrieve all arguments that can be of any value, without benefiting one of the parties over the other. At the same time, he needs to take into account the inequality of parties and therefore protect the weaker one. As a result, the judge may ask questions to get a better idea of the underlying case, but too much translating facts to legal grounds is not allowed. The broader scope of an open norm gives judges a desired flexibility. In the ideal situation, the judge will base his conclusion on all the relevant factors and still in the framework of the law, of the purpose of the legislator and reflect the social view of that time.
Although justice might be reached, one of the downsides of open
norms is the high level of legal uncertainty, which leads to
unpredictability and a lack of transparency.[
In the common law system very few laws are written. The legal scope is set by a constitution, filled with open norms and some prohibiting laws. What is not written is allowed, unless there is case law on the certain topic or situation. The value of jurisprudence is significant, since it sets the scope of rules and (new) legal outcomes. Although the civil law system is more about written laws and gives the assumption to be covering all possible situations beforehand, jurisprudence is still of great value. It may be impractical or even impossible to foresee, and at the same time describe, every set of characteristics of a case in all possible combinations. This is why also in the civil law systems open norms are used. The result in both systems is a lack of certainty how judges will rule. It is also quite demanding for judges themselves to deal with a broad and unclear range of possibilities. Would it not be better to provide all (legal) actors with a clear scope of arguments that influence the ifnal outcome? If not defined in advance, it seems better to establish the ruling aspects, apparently found to be of value by other judges and parties. If possible, giving such insight will lead to more legal equality and certainty to all involved. 3 3.1
We formulate the Argument Extraction problem as an Information Extraction task that has to be repeated once per jurisdiction and/or language. We consider that an information system capable of linking an argument of a case to an article of the common law, can only be valid within the boundaries of the jurisdiction where the article applies. Further usage of the system is bounded by the accessible underlying data.
To this end, we design an annotation process, pictured in Fig. 1, driven by data needs for the downstream tasks, as assessed by a collaborative team of AI and Law practitioners, providing the necessary expertise to shape both content and form, and align the outputs of the system with the needs of the practitioners, in a perspective that makes sense for the business at hand. This business perspective will as well guide the choice of relevant metrics to optimise, as there is no one-size-fits-all answer, but only tailored answers to specific needs.
Given the designed task and tools, annotators proceed with identifying which fragments of the text qualify as Arguments, either as facts or as statute law references, and which pairs are tied together. The pairing associates one fact to one statute law, while each fact can be paired to multiple statute law references, and each law article can be paired to multiple fact.
The training and evaluation process, pictured in Fig. 2, follows the classical work flow, under the supervision of an AI expert that provides the modelling setup and the evaluation tools. The selected metrics will provide the drive for system improvement.
We formulate the Precision-Recall balancing problem with regards to the usage of the system and the type of downstream errors we aim at minimising. It seems natural to prioritise Precision over Recall, for a knowledge that is correct but maybe not exhaustive.
We observe recent similar researches in the field of Legal
Information Retrieval, using either traditional techniques: Ontologies
and Combinatory Categorial Grammar in [
We consider legal qualification as the association of 3 information: a fact or a collection of facts, a statute law article, and a judgement whether the facts constitute a breach of that article or not. Having annotated those links in the existing corpus of decisions, we formulate a system that models the legal qualification of facts. It is pictured in Fig. 3
As an AI system is also an attempt to reverse-engineer a decision scheme, it enables both prediction based on known inputs, and description of which inputs are the most likely to make a given decision.
The review and the evaluation of that model will be driven by legal expertise. The descriptive power of the model will be evaluated with regards to its capacity to summarise which clusters of facts explain a legal qualification, to show which factors can explain variation in the certainty of a qualification. 4
In this paper we want to focus on open norms in labour law, more specifically, on dismissal law. One of the open norms in several legal systems is about culpable acts. If for instance, an employee violates the law or breaks the rules within the company, it is up to the judge to decide whether, in his perspective, the act is a reason for dismissal or if there are reasons to rule in favour of the employee. Acts under the scope of culpable behaviour are diverse. It covers for example theft, being late for work, sexual harassment and breaching a non-compete clause. This wide range of specific situations, makes the outcome even more uncertain and unclear for both parties and even for the judge. First of all, it is not decided beforehand if a situation will fall under the scope of possible culpable behaviour and secondly, if it does, it is not definite the situation will be qualified as a culpable act. The outcome relies on a variety of details, from the view of both the employer and employee. What leads to fair dismissal within one company may not be severe enough within another, whilst the act can be exactly the same.
If machine learning makes it possible to come up with an insight into arguments that are of value for legal decision making, it will be a great gain. This way of extracting relevant factors from previous judge-made law, can contribute to these main principles in law, achieving greater legal equality, certainty and even justice. 5
AI is influencing the work of legal practitioners in several ways. In addition, AI is used nowadays to give insights on how a specific judge is most likely to rule in a case. By developing these kinds of AI aids, legal research can become less time-consuming and more efective. They can even contribute to more equal outcomes, thus legal certainty and equality. This however, is disputable, since the predictions are still mainly black boxes. Therefore, parties cannot define how their case might difer from the data the prediction is based on. Judges furthermore, miss out on the substantiation and therefore, will not be able to have the complete legal or practical framework in order for them to implement the prediction.
In our practical research, we aim for the public community to benefit from the result and the usage of legal professionals to difer from existing use seen in the reduction of time and costs, as well as in the creation of clarity in the legal practice, specifically, in the use of open norms. Parties can either solve their own conflicts once they know what is relevant to a case and what ought to be required behaviour, and thus prevent a conflict about the expected behaviour. Consequently and in the most optimal result, this would lead to fewer justice seekers, which can be a relaxation of the judicial system. If parties still want a judge deciding on their dispute, our research will be an aid to come up with the valuable circumstances, since some details brought up by parties can be irrelevant and may lead to piles of legal documents for a judge to read through. On the other hand, parties may be incomplete and consequentially harm their case. In the Netherlands for instance, a judge can only base his ruling on facts presented by parties. He is allowed to ask questions and to order a hearing, but should always keep equality of parties in mind and is, thus, limited. Once less necessary efort is needed, this will as well lead to a relief of burden of the judicial system. 6
The goals of our research and development are multiple and depend on the user type. We aim to reach the following: • Lawyers and legal support: Through an easily accessible desktop tool they can assess a case that is based on available facts extracted from previous case law. The goal is not to simply provide them the chance of a possible breach of the open norm, but to give them an overview of relevant practical arguments. These arguments can be used to build up their case and that will colour in the open norm with actual and practical circumstances. These users would only need to define and ultimately substantiate and/or prove which aspects are applicable. It saves them time and they do not have to compile extensive pleadings and other procedural documents that might contain irrelevant arguments that only distract from the valuable ones. • Judiciary: For judges the system will also be available through a desktop. Similar to lawyers, they have the ability to assess jurisprudence. They can quickly see what aspect(s) is or was relevant in other comparable cases where there was a breach of the invoked open norm. Higher courts will prevail over lower judiciaries. As a result, judges have a practical tool that gives them an idea of what the common relevant and justified aspects are that colour in the open norm, since the outcome is based on a large number of previous cases, without losing their own power to decide on the case. The higher goal is to create more legal equality and certainty. • Legal sciences: Most research studies demand case studies.
Questions that arise, such as what the trend in jurisprudence is or whether there has been a change in ruling opinions, are mostly about open norms and can only be answered by going through a reasonable number of cases. It takes time and requires skills as constructing databases to get a suficient insight. With the targeted practical tool research can be done in less time and might be of higher quality since the amount of investigated cases should increase. • Public community/Parties: Law should be accessible to everyone. This includes not only access to court, but also having knowledge about what rules are applicable and what the practical results of these rights or prohibitions are. By setting a clear scope of practical behaviour that colours in an open norm, we can increase the desired accessibility. Parties would have access to an online tool and could perform a self-assessment. This could form the basis of negotiations between parties to either prevent a conflict or to solve one.
Summarised, the research goal and practical result are aimed to allow a new value proposition for diferent users. Instead of predicting outcomes, we are focusing on how to get to a justified outcome that is in line with common ideas. Furthermore, we will try to relief the judicial system by preventing conflicts, since potential parties should get a clearer insight of how they should behave towards each other, in order for them not to breach the applicable open norm.
We will be applying this research on an existing open norm, culpable act, in order for us to have enough relevant case law. The legal research on this topic has been mostly completed humanly, so we tend to know what relevant practical arguments judges use to support their ruling in case of the underlying question whether certain behaviour is culpable. In this way we can train the system, and see whether the generated outcomes are correct.
We consider using Court Records as a primary source of data. A properly formed court ruling will motivate the final decision by elaborating on how the given facts were legally qualified. The proliferation of "Open Data" policies increase the amount of information made public each year. Most of this data comes unstructured, and most of the structured data provide mainly a structure for document meta-data, such as bibliographical details, time stamps and document origin. We have to consider that our main source of data will consist of raw text files.
The annotation project receives a lot of attention with regards to aligning its design with the needs of the downstream tasks. We form an open collaboration team able to tackle both the content (legal experts, legal operations experts) and the form (AI experts) in a holistic manner. Cross-education of team members on key topics is essential to lower the walls and break free from a top-down waterfall approach.
RQ1. How to arrange annotations of Court Decisions? How to use unstructured legacy annotations? RQ2. How to extract arguments from court decisions? How to optimise on a business relevant metric? How to assemble a knowledge base? How to evaluate accuracy and completeness? RQ3. How to model a decision boundary based on facts associations? How to model facts importance? How to model facts composition?
In this paper, we introduced a novelty Intelligent Assistant based on Artificial Intelligence techniques, that would leverage the knowledge contained in court decisions in order to support stakeholders of the judicial system at large in assessing the efectiveness of policies, the actual usage of legislation and norms, the merits of new cases with regards to jurisprudence.
We associate the tool to multiple stakeholders of the judicial system with diferent use cases where the additional information benefits all parties.
We recognise that the challenge lies not only on the technology itself, but on the capacity to produce a fruitful collaboration between the world of legal work and the world of Artificial Intelligence, and the co-engineering of a solution that is driven by business or societal needs.