Text retrieval has taken its role in almost all domains of knowledge understanding. It has applications in the legal eld where there is an extensive collection of structured and unstructured texts. Arti cial Intelligence is now applied in this area to understand and retrieve legal documents. This paper explains a working model developed for the track Arti cial Intelligence for Legal Assistance in Forum for Information Retrieval Evaluation, 2019 (AILA-FIRE2019). We have used an embedding model approach to represent these legal texts in a semantic vector space. The similarity between these document embeddings is found using an existing method of cosine similarity. The corpus used for building embedding models is the dataset provided in AILA-FIRE2019.
Nowadays, the amount of legal content available from online sources is high. The rapid change in handling legal documents and related pieces of information in electronic form has increased the need for using arti cial intelligence in this domain. Interfaces that provide a semantic understanding of texts would be useful for non-specialists as well as legal practitioners in understanding statutes and related prior cases. A layman gets the basic knowledge about legal proceedings by looking through the most relevant precedent cases and statutes retrieved for each query situation.
The rest of this paper is organized as follows: Section 2 presents related works in the area of document retrieval. Task description and dataset details are provided in Section 3. Section 4 explains the methodology used. Section 5 relates to experimental details and evaluation results. Finally, Section 6 concludes the work with some future improvements. 2
Text retrieval also called document retrieval, matches some query against a set of
text records either structured or unstructured. Methods to retrieve texts is more
or less based on some similarity measures. Various clustering methods were also
employed. Graph-based approaches were also used to nd document similarity
using modi ed shortest path graph kernals[
AILA Track consists of two tasks, namely precedent retrieval (Task 1) and statute retrieval (Task 2). Task 1 aims at nding relevant previous case documents/ precedents for a situation mentioned in the query document. Task 2 retrieves the most similar statutes for a scenario given in the query.
The table below shows the statistics of the dataset for both evaluations. It
is a collection of Indian legal documents, i.e., statutes in India and prior cases
decided by Indian courts of Law.[
Paragraph vectors are inspired from architecture for learning word vector representations, in which document or paragraph id is assigned to each paragraph. Each word in a paragraph is represented by a column vector in a matrix, say W. Concatenation or sum of these vectors are features for predicting next word in the sentence. For sequence of words w1,w2,...,wT word vector model maximizes the average log probability 1 T k
X log p(wtjwt k; :::; wt+k) T t=k (1) Prediction is done using a softmax classi er.
p(wtjwt k; :::; wt+k) = eywt P eyi i where, yi can be computed as unnormalized log probability for each output word i using
y = b + U h(wt-k; :::; wt+k; W ) where U, b are the softmax parameters. h is constructed by a concatenation or average of word vectors and paragraph vectors extracted from W and D respectively.
The paragraph vectors and word vectors are averaged or concatenated to predict the next word in a context. Paragraph vectors and word vectors are trained using stochastic gradient descent, and then paragraph vector for new paragraphs can be inferred. The two stages in paragraph vector algorithm are: 1) Training word vectors W, softmax weights U, b and paragraph vectors D on already seen paragraphs. 2) Inference step derives paragraph vectors for new paragraphs by adding more columns in D and gradient descending on D keeping W, U, b xed.
There are two variations of this model: PV-DM (Distributed Memory Model of Paragraph Vectors) which considers word order with the concatenation of word vectors and paragraph vector. The missing context information is in the paragraph vector, and it memorizes the semantic content of the paragraph. PVDBOW (Distributed Bag of Words model of Paragraph Vector) neglects word ordering by taking random samples of words to predict another word. Since word ordering is essential to understand the semantics of a text, a distributed memory model is preferred while training the paragraph vector model. Vector representation for documents are inferred from this model and are then compared using Cosine similarity.
Cosine similarity [8] measures the angle between two vectors in multidimensional space. Cosine of two vectors is : A:B = jAjjBjcos and cosine similarity, given two vectors A and B, is de ned as: similarity = cos =
A:B jAjjBj
Pn = pPn i=1 AiBii=1 Bi2 i=1 Ai2pPn Using this similarity metric, we can understand the orientation of documents as its based on angle measurement. Documents with similar orientations can be considered as similar and if the document vectors are created in such a way that the semantics of element words are taken care of, then these orientations can also be semantically similar. 5
A work ow of the proposed system is presented in Figure 2. The collection of precedent documents, statutes and query cases are provided in form of text les (2) (3) (4) as input. The rst step is tokenization of each document to a collection of tokens or words and then each collection is labelled with a tag and number to uniquely identify documents. This represents paragraph id in the paragraph vector model discussed above.
In the next phase, doc2vec model from gensim library is used for training the labeled precedents, query, and statutes. Doc2vec model is trained using the precedents and query documents collection with ve epochs and embedding dimension set as 100. This model is then used to obtain the vector representation of each document for precedents retrieval task. Statutes collection is also added to the existing doc2vec corpus for retrieving vector representations for task 2. In the nal stage, vector representations obtained for each case document, precedents and statutes are compared using cosine similarity.
For Task 1, cosine similarity between present case situation and precedents are calculated, sorted, and ranked. Similarly, for Task 2, cosine similarity between each present case situation and statutes are found. The similarity scores are then sorted and ranked. The nal result gives a list of ranked documents for each case scenario presented in the query document.
The proposed system uses 50 cases as query documents out of which ten
queries were used for pre-evaluation and 40 in the nal evaluation. TREC
Evaluation [