This paper aims to present the development of an instrumental solution to a necessity raised from an artificial intelligence project, latter called Victor robot project. The Victor robot demands a methodological combination of the reasoning of the areas of software engineering, computer science and Law. For its unprecedented factor, all researchers must develop knowledge in an intense form, while working in different thought process, language and very specific legal texts in a huge volume of data. In a second part, this paper presents some sui generis features of general repercussion as a constitutional filter and a possible field for ontological development for machine learning, and important to understand your potential application at Supreme Court activity. Finally, the article presents some steps of the project that is still in progress, but is already considered the largest artificial intelligence project in the Brazilian judiciary, which has 100 million cases in stock.
1 {Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).} The first goal of this paper is to present the development of an instrumental solution to a necessity raised from the work plan established in a artificial intelligence project between the University of Brasilia and the Brazilian Supreme Court (STF), called "Research and development project on machine learning about judicial data on general repercussion in the Supreme Court", latter called Victor Robot Project. The Victor robot project demands knowledge and researchers from the areas of software engineering, computer science and Law. For its unprecedented factor, all researchers must develop knowledge in a intense form, while working in different thought process, language and very specific methodologies. Therefore appears the need of an integrative methodology, that allows the implementation of advanced artificial intelligence instruments for the legal area. For the development values and guidelines must be really clear, and they will be addressed by deductive method, from the findings on Artificial Intelligence (AI), Law and the agile methods.
This work plan foresees the engagement of three different areas of knowledge: software engineering, computer science and law for the development of a unique innovative solution in an environment with thousands of lawsuits and millions of legal texts, all this in absolutely non-structured data. This scenario represents a series of challenges to be faced for areas that are traditionally structured on diametrically different rationalities but can – in a convergent and synergic way – develop the steps (stages, phases) to achieve a common central goal: the development of a system based in Machine Learning (ML) algorithms and, possibly, Deep Neural Networks to be used in a specific phase of the judicial process called “general repercussion”, a brazilian (typical country of civil law tradition) adaptation to an approach to the solutions of the common law tradition. The use of the system will allow the improvement of the contingency of judicial process in Brazil. To start the planned research work, a methodology was customized: it is more familiar to the software development area and still insipid in the law studies. The first objective of this paper is to present and justify these choices. As two of the main critical factors of the project were a short calendar and a strict budget, as well as the innovation character of the research – and its dynamics needs of correction and adaption, the agile scrum methodology presented itself as extremely interesting. Nevertheless, the legal character of the research demanded a few adjustments on the methodology framework. In the same sense of the necessary adjustments to justify a specialized research methodology, it was needed to fixate some conceptual cuts, bounding properly the research field. In this sense, it will be presented the concept and the impacts of the general repercussion in the process stock and the strategies of application of machine learning for identification and classification of themes in the general repercussion. This line of thinking it is not only valid for lawyers, but to all legal practitioners. Activities in legal administration logistics that could be performed in a fraction of time with a high level of accuracy, may be benefit, and allows human talent to be concentrate in strategic areas. The benefits of a research on AI and Law are yet underestimated, since one cannot limit remotely precisely the combination of human knowledge and AI tools. However, it is possible to previously define some topics that should necessarily guide an AI research in a way to maximize social benefits with accuracy and speed. This research is in this field.
2.1
As two of the main critical factors of the project were a short calendar and a strict budget, as well as the innovation character of the research – and its dynamics needs of correction and adaption, the agile methodology presented itself as extremely interesting. Nevertheless, the legal character of the research demanded a few adjustments on the methodology framework2.
A crucial issue to the accuracy is to identify/decide which is the level of symmetry
between computer activity and what really happens in the human reasoning.
With this perception, the idea of the story of seven blind wise men makes sense
again. The specificities, contingencies, bias - although possibly existing or true (in
the indian story sense) do not match the holistic, global or collective concept. This
perception is crucial to define any work method between AI and law3.
2 Therefore, it is not the purpose of this paper to present: 1) the work plan of the “Machine
Learning of the general repercussions in the Brazilian Supreme Court”; 2) The state of the
art in artificial intelligence (AI) or IA and law. These approaches will be made here only to
in a way to instrumentalize the methodological definition. The subjects listed above are part
of the research in a broader way, and will be released properly when opportune.
3 An in depth historic view of the development process and AI perspectives is described in
the paper "A review of artificial intelligence”, from E.S. Brunette, R.C. Flemmer and
C.L.Flemmer. School of Engineering and Advanced Technology of Massey University,
New Zeland. There, beyond the historic perspective, there is the register of Mikawa (2004)
work on the variables of the human mind: “consciousness, preconsciousness and
unconsciousness. In this model most data processing is done in the non-conscious states. He
therefore proposed a system where the level of information processing changed, based on
visual information being received. In his model, external information processing is
conducted when the robot is awake. However, when the robot is in sleep mode, external
information processing is reduced and more internal information possessing is conducted.”
(BRUNETTE, 2009, p. 386) [All content following the pages of the paper was uploaded by
Claire Flemmer on 25 March 2014.]
The synergy between artificial intelligence and law was detected by Professor
Edwina L. Rissland, from the Department of Computer Science from the
University of Massachusetts, in a very traditional article in 2003. In there, [
In the AI general research outlook can be identified nuances and limitations of the existing techniques for the law functionality, as well as catalyzing elements for the development of the new sustentable approaches. The mentioned Rissland work is strengthened by the observation of the consolidation of standard legal argumentation theories, with the refining of the design of models to analyze and evaluate legal reasoning.
A process characteristic of software engineering, created by Kent Back, Extreme Programming (XP)4 is a set of development practices guided by values. These practices and values seek to face the most common difficulties in software development, they are: missing deadlines and overspending to develop solutions. For dealing with extremely delimited values and practices equally simple, when applied together shows a considerable synergistic potential. Better decisions, quick answers, efficient communication and wise efforts investments are noble goals of a research, especially of researches with the characteristics pointed above5. There is emphasis in the interactions and personal aspects of the teams involved. The process itself - as well as its tools - are of development and innovation. The developed work is important and - probably a defining factor for the methodology choice - the customer collaboration (as the guideline set above of law and its legal language`s characteristics working as a customer). This metaphorical use of law as 4 It is not the intention of the present paper to develop a dissertation on the subject. 5 Tripp, Saltz and Turk recently reordered the cautions for the use of the agile methodology, and guided the customization for the present project: “We believe that agile in software development is an “instance” of agility in projects. Agile can be used in many other project contexts beyond software development, […] Some foundational questions include: • What guidance can we provide to create and sustain better agile and lean behaviors and more successful outcomes? • How can we incorporate other functions, such as architecture and production support, into agile and lean frameworks?• How can organizations and cultures restructure to support these philosophies? • What are the measurable outcomes of using agile techniques? • What additional metrics might a team use to measure team performance? • What are the measurable differences in outcomes when using traditional vs agile techniques? • What are ways that we can create a repository of knowledge, experiences, cases and empirical data that could be used by research and industry to leverage and expand our understanding of and practical skills in agile techniques?” [18, p. 5470] a customer allows the process to be tolerant with the unexpected (typical of innovation research), and give quick answers to the problems - enabling changes, tests and adjustments6 - a typical Victor robot routine! The agile method has the characteristIcs7 of a cycle (SDLC) that where very well grouped by [13, p. 198-199): characteristics Conduct a survey and assess the feasibility of information systems development project.
Study and analyze the information systems that are running Determine the requests of the information system users.
Select the best solution or problem solving Determine the hardware and software.
Design a new information system Build a new information system.
Communicate and implement the new information system.
Maintain and repair / improve the new information system if necessary.
What should guide the agile methodology application is the verification of its
potential benefits in face of a traditional work methodology. This paper agrees with
the statement. There are a series of studies in this sense, and the main reasons for
its use are empirically verified.8
The choice for the agile method imposes severity, risking otherwise that the
benefits do not overcame the difficulties and imprecisions. [
application repercussion elements for
3.1
The brazilian’s institute of the general repercussion was instituted in order to operate as a recursional filter, thus avoiding the knowledge of extraordinary appeals, whose constitutional cause is irrelevant or of sole and exclusive interest of the parties. It is understood that a cause demonstrates relevance when it has real and undoubted importance, standing out against those other causes that involve the same object. Thus, relevant issues are those that have great value or interest, which is why they are clamoring for guidance from the Federal Supreme Court. With regard to the requirement of transcendence, it is understood that the cause can not be limited to matters of individual interests. That is why the experts and the jurisprudence reinforce the need for transcendence, that is, the analysis must exceed the individual interests and set up a true collective interests in the examination of matter. In this way, the general repercussion allows that in cases founded on the same controversy, one or more resources that adequately and fully represent the leading cases be selected, to be analyzed by the Supreme Court, according to the Code of Civil Procedure; the effects of the decision on only one appeal will bind so many others of equal matter, which, of course, will denote greater effectiveness and celerity in the jurisdictional activity.
This scenario goes back to the adoption of the system of precedents by the Brazilian procedural system. The flow of administrative tasks, both for the purpose of directing the judicial process itself and for developing services in support of judicial activity, has taken a great deal of effort, time, resources and other valuable elements of the public service.
Nor is it today that these flows are ever more precise and produce many elements of efficiency and effectiveness metrics. Therefore, there is a good substrate for the application and measurement of gains in approved and implemented Artificial Intelligence (AI). Based on the idea of a flowchart, such as the representation of a sequence of events that map some (some) types of decision, the tasks often go through needs for treatment, reading and understanding of data, classification and comparison with pre-existing parameters or indication of a new possible standard. Terminologies may vary according to the representation in the flow, but the activities usually pass through an (optical or other sensory recognition) of the data, a structuring of this data (a form of storage, organization), an optimization of information relevant to some form of classification and decision of the path to follow in the flow or - possibly - change of flow.
For a first step, it is possible to contribute, for example, to the development of "knowledge", that is to say with technology of optical character recognition or similar, that allows the recognition of text characters in images, transforming them into text capable of edition. Even the recognition of handwritten images is possible. For the structure of data, the AI can contribute, for example, with organization algorithms forming structures oriented by their function, purpose and conditions for storage (classic vectors, lists, stacks, trees, etc).
Both the form of organization and the methods used must be made according to a methodology appropriate to the characteristics of the judicial data. Classification is one of the most frequent features offered by AI. For example, a multi-layered neural network architecture can perform, with very acceptable parameters of accuracy and verification and ethical validation, treatment services and data structuring to function as a real input data sorter.
4.1
The Victor robot was designed to act in the flow of management of court proceedings in the stage of evaluation and framing of selected theme on general repercussion. Due to the quality of the data at the beginning of this flow, the server (without the help of ML) takes considerable time (close to 30 minutes) to locate and organize the relevant data to be read and interpreted to perform the classification. This statistic, combined with another statistic: approximately 400 new processes arrive at the beginning of this flow per business day. This identified a serious management problem that absorbs something close to 200 hours/worker for organization and flow initiation (pre-activity). The quality of initial text and image data is also highly variable due to the large number of process sources addressed to the Supreme Court and the variation in electronic process systems and eventual digitization capabilities.
Briefly, the project aims the development and application of the newest Artificial Intelligence concepts and techniques, especially Machine Learning for relevant needs in terms of processing, classification of procedural parts and classification of temes/classes on general repercussion management at Supreme Court and support the decisions of the technical team. The objectives are to increase the speed of processing, increase the accuracy (accuracy) in the involved steps and optimize the human resources to perform more strategic activities to the Court. The following stages of the Victor Project was planned: 1) Preparation and structuring of the General Repercussions database for training of machine learning models. 2) Evaluation of more efficient training algorithms and strategies for the context of General Repercussions, including deep artificial neural networks. 3) Prototyping and training of the chosen algorithms including their evaluation. 4) Preparation of the communication architecture for real-time process classification along with the interface for recording possible errors in model responses, including integration with the STF solution park. A mapping of human activity was done and an identification of the most recurring themes of general repercussion, and the division of data for testing, modeling and comparison between the activity simulated by human legal experts and the machine were made. Thus, from a preprocessing model, a classifying system of 28 general repercussion classes is being developed, selected by association with a large number of similar court lawsuits. The most several simulations indicate a high accuracy index. 4.2
All goals (Table 2.) are in execution and improvement in agile medium. In texte environment with some encouraging results: stage
lawsuits were collected from the STF database, preprocessed and entered into a structured database hosted on the research laboratory's servers.
preprocessing methods and classifiers were investigated. In application to 28 themes of general repercussion, the best results achieved using the XGBoost technique model, with an average accuracy (F1-Score) above .90
been adapted to work within the STF (STF-Digital) technology park.
Throughout its execution, there were changes in the way to extract the text from pdfs files, a fact that required reprocessing of the entire database. Once this extraction is completed it will begin the machine learning remodeling process. 5
This way it is intended to conduct an intensive research, in a limited time, optimizing the available resources, with instruments that allow problems identification and necessary adjustments. From the human relations point of view, although the team has different theoretical backgrounds, it is intended to keep the team united, stable and productive. Thereby, it is understood that the elected methodology is justified, with its modifications to attend the research work plan.
The general repercussion allows that in cases founded on the same controversy, one or more resources that adequately and fully represent the leading cases be selected, to be analyzed by the Supreme Court, according to the Code of Civil Procedure; the effects of the decision on only one appeal will bind so many others of equal matter, which, of course, will denote greater effectiveness and celerity in the jurisdictional activity. The Victor Project, although very recent, has been presenting very interesting, strategic and relevant results in an attempt to reduce the processing time of court proceedings. As said several hypotheses of uses of preprocessing methods and classifiers were investigated. In application to 28 themes of general repercussion, the best results achieved using the XGBoost technique model, with an average accuracy (F1-Score) above .90.