Automated identification of relevant documents or statutes for the given query is a time eficient process. Artificial Intelligence in Legal Assistance task (AILA) task is about using the artificial intelligence to identify the suitable prior documents or statute for the given query and semantic segmentation of the legal documents. We have identified the suitable statute for the given query using Best Matching (BM25) ranking algorithm. We have calculated the scores using BM25 algorithm and sorted the statutes according to the scores and then identified the suitable statutes with better MAP score of 0.2975 than most of the other approaches on AILA@FIRE-2020 dataset.
Artificial Intelligence in Legal Assistance task is about identifying the relevant cases or statutes for the given query and semantic segmentation. There are two tasks namely Task 1: Precedent
& Statute retrieval and Task 2: Rhetorical Role Labeling for Legal Judgements. Task 1 has subtasks with Task 1a for identifying the precedent case document and Task 1b for identifying the relevant statutes. Task 2 is the classification task of classifying the legal statements in one of the seven semantic segments/rhetorical roles.
The seven classes include: Facts : the events that led to the case.
Ruling by Lower Court : the decision of the lower court Argument : arguments Statute : relevant prior statute Precedent : relevant prior case Ratio of the decision : statement given for judgement Ruling by Present Court : the decision of the Supreme court
The AILA@FIRE-2020 data set has 2914 case documents and 197 case statutes from which the relevant case documents and statutes has to be identified. The training set had 50 queries along with their relevant case documents and statutes. 10 queries were given as a testing set for identifying the relevant case document and statutes. For Task 2 of Rhetorical Role Labeling for Legal Judgements, 50 case documents along with their seven labels of rhetorical roles were given as training set and the test set is of 10 case documents which has to be classified according to the semantic segments which is explained in Table1.
We have presented a ranking algorithm for the task 1b of identifying the relevant statutes
among the tasks provided by the Artificial Intelligence for Legal Assistance. BM25 [
4.1. Task 1b: Identifying relevant statutes - BM25 In this approach, the statutes are pre-processed, vectorized and ranked according to the relevant queries.
All the given statutes are written in a single file by removing the characters that are not letters or numbers, the stop words in the NLTK corpus stop words and converting the words in the sentences to a lower case.
For ranking the statutes, initially a word dictionary is built with the words of the statutes except “for, a, of, the, and, to, in" and its frequency. Then, the document length, average document length , inverse term frequency are also calculated to calculate the BM25 score. The default parameters are taken which is 1.5 for k (saturation parameter) and 0.75 for b(length parameter) to calculate the score.
After calculating the score of individual statute document vs queries, the statutes are sorted according to their scores.
The results of Task 1b of identifying the relevant statutes is shown in Table 2. The performance of our team is denoted by SSN_NLP. Our approach has attained a MAP score of 0.2975, BPREF score of 0.2531, recip_rank of 0.4769 and P@10 score of 0.15. Our approach performed better than other approaches and is comparable to the top ranking approaches.
Thus, artificial intelligence can be used for the identification of precedent documents to identify the documents efectively. We have used Best Matching ranking algorithm (BM25) to rank the relevant statute with respect to the given query which have attained better results than most of the other approaches with a MAP score of 0.2975, BPREF score of 0.2531, recip_rank of 0.4769 and P@10 score of 0.15. The performance can further be improved by altering the values of hyper parameters such as saturation parameter and length parameter.
We would like thank the Department Of Science and Technology (DST)-SERB funding scheme and HPC laboratory for providing the resources and space for our research.
SSN_NLP SSNCSE_NLP IMS_UNIPD Uottawa_NLP
UB
LAWNICS TUW-informatics nlpninjas scnu_1 fs_hit_1
fs_hu fs_hit_2 0.2975 0.3423 0.3383 0.2506 0.3134 0.2962 0.2619 0.00917 0.3851 0.2139 0.235 0.2003 0.2531 0.136 0.279 0.186 0.2633 0.2812 0.2033 0.024 0.3054 0.1587 0.198 0.1587 0.4769 0.3423 0.5349 0.3144 0.5787 0.4607 0.4855 0.1204 0.5615 0.3371 0.3581 0.3452