Shift-of-Perspective Identification within Legal Cases

Abstract

Arguments, counter-arguments, facts, and evidence obtained via documents related to previous court cases are of essential need for legal professionals. Therefore, the process of automatic information extraction from documents containing legal opinions related to court cases can be considered to be of significant importance. This study is focused on the identification of sentences in legal opinion texts which convey diferent perspectives on a certain topic or entity. We combined several approaches based on semantic analysis, open information extraction, and sentiment analysis to achieve our objective. Then, our methodology was evaluated with the help of human judges. The outcomes of the evaluation demonstrate that our system is successful in detecting situations where two sentences deliver diferent opinions on the same topic or entity. The proposed methodology can be used to facilitate other information extraction tasks related to the legal domain. One such task is the automated detection of counter arguments for a given argument. Another is the identification of opponent parties in a court case.

INTRODUCTION

Documents describing legal opinions related to previous court cases carry a significant importance when it comes to the legal literature. The information presented in these legal opinion texts are used in diferent capacities such as evidence, arguments, and facts by legal oficials in the process of constructing new legal cases [ 29 ]. Therefore, information extraction from legal opinion texts can be considered as an area of significant importance, within the topic of automatic information extraction in the legal domain. In order to perform systematic information extraction from a legal opinion text, a system should be able to interpret the meaning of a given text. In the process of interpreting the meaning of a text, understanding the context can be considered as a major requirement, especially in the legal literature.

Identifying how textual units are related to each other within a machine-readable text is an important task when it comes to interpreting the context. Humans are good at comparing two textual units to determine the way in which those two units are connected. Granting this ability to computers is a major discussion topic in the research related to areas of Natural Language Processing and Artificial Intelligence. A sentence can be considered as a textual unit with significant importance in a text. Therefore, analysis of relationships between sentences can be useful to get a clear picture on the information flow within a text which is made up of a considerable number of sentences.

Similarly, identifying the types of relationships existing between sentences in legal opinion texts can be used to identify the information flow within a legal case. Within a document describing legal opinions related to a court case, diferent types of relationships between sentences can be observed such as elaboration and contradiction. Pairs of sentences can be classified into two major groups based on whether the topics which are being discussed by the two sentences in the sentence pair is the same or not. In other words, the two sentences in a sentence pair may discuss the same topic or they may discuss completely diferent topics. Even if the two sentences are discussing the same topic, the opinions or views presented in the two sentences on the topic may be diferent. Consider the following sentence pair taken from Lee v. United States [ 2 ].

Example 1 • Sentence 1.1: Applying the two-part test for inefective assistance claims from Strickland v. Washington, 466 U. S. 668, the Sixth Circuit concluded that, while the Government conceded that Lee’s counsel had performed deficiently, Lee could not show that he was prejudiced by his attorney’s erroneous advice. • Sentence 1.2: Lee has demonstrated that he was prejudiced by his counsel’s erroneous advice.

The above two sentences discuss whether a person named Lee was able to convince that he was prejudiced by his attorney’s advice or not. While the first sentence says that Lee could not show that he was prejudiced by his attorney’s advice, the second sentence contradicts the first sentence by saying that Lee has demonstrated that he was prejudiced by his counsel’s erroneous advice. Thus, the two sentences provide diferent opinions on the same topic. Contradiction is not a necessary condition in order to classify a pair of sentences as providing diferent opinions on the same topic. For example, consider Example 2 which consists of two adjacent sentences which are also taken from Lee v. United States [ 2 ].

Example 2 • Sentence 2.1: Although he has lived in this country for most of his life, Lee is not a United States citizen, and he feared that a criminal conviction might afect his status as a lawful permanent resident. • Sentence 2.2: His attorney assured him there was nothing to worry about–the Government would not deport him if he pleaded guilty.

It can be seen that both sentences in this example discuss the topic – the deportation of a person named Lee. Though the two sentences here do not provide contradictory information, they provide two diferent viewpoints regarding the same topic. It can be seen that the opinions of Lee and his attorney on the possibility of Lee being deported is diferent. Therefore, when discussing sentences with diferent opinions on the same topic, not only the sentences providing contradictory information but also the sentences providing multiple viewpoints on the same discussion topic should also be considered. In each of the above two examples, Sentence 1 comes before Sentence 2. From this point onward, the first sentence in a sentence pair will be referred to as the Target Sentence and the second sentence as the Source Sentence.

An important observation which can be made by considering Example 2 is that the identification of the shift in the viewpoint in that particular occasion is not straightforward. This implicit nature makes the task of identifying sentences which provides diferent opinions on the same discussion topic even more challenging. At the same time, it can be considered a vital task due to its potential to enhance the capabilities of Information Extraction from Legal Text by facilitating automatic detection of counter-arguments, identification of the stance of a particular party in a court case and to discover multiple viewpoints to analyze or evaluate a particular legal situation.

Hence, the objective of this study is to identify sentences which have diferent perspectives on the same discussion topic in a given court case. For this study, legal opinion texts related to United States court cases were used.The next section provides details on the previous work which are related to our study. Section 3 describes the methodology followed in this study while the outcomes of the study are discussed in Section 4. Finally we conclude our discussion in Section 5. 2

RELATED WORK

Computing applications which can be considered to be both eficient and efective are scarce due to the challenges in handling legal jargon[ 11, 12, 25 ]. The nature of legal documents employing a vocabulary of mixed origin ranging from Latin to English has been put forward as a reasoning for dificulties of building computing applications for the legal domain [ 29 ].

Regardless, there have been some recent attempts to circumvent these problems in the legal domain including information organization [ 10–12 ], information extraction [ 28 ] and information retrieval [ 29 ]. Going forward, owing to the popularity of knowledge embedding in the literature, several studies have taken up the task of embedding legal jargon in vector spaces [ 19, 28 ]. Further, in the information extraction domain, the study by Gamage et al [ 8 ] attempted to build a sentiment annotator for the legal domain and the study by Ratnayaka et al [ 24 ] attempted to identify relationships among sentences in legal opinion texts.

Discovering situations where two sentences are providing diferent opinions on the same topic or entity is an important part when it comes to identifying relationships among sentences [ 24 ]. Contradiction is a suficient but not a necessary condition in this regard. The study [ 17 ] is focused on finding contradictions in text related to the real world context. In an attempt to define contradiction, the same study[ 17 ] claims that “contradiction occurs when two sentences are extremely unlikely to be true simultaneously” and the study [ 20 ] also agrees on that definition. However, the study [ 17 ] also demonstrates that two sentences can be contradictory while being true simultaneously. These characteristics of contradiction make the process of detecting contradiction relationships more complex.

In order to become contradictory, two textual units can elaborate not only on the same event but also on the same entity. For example, if one sentence in a sentence pair is saying that a person is a United States citizen while the other sentence is saying that very same person is not a United States citizen, it is obvious that the two sentences are providing contradictory information. Here, the contradictory information is upon a person which can be considered as an entity. Therefore, it is more reasonable to consider that in order to be contradictory, texts must elaborate on the same topic.

In order to detect contradiction, diferent features based on the properties of text have been considered in the previous studies [ 17, 20 ]. Polarity features and Numeric Mismatches are such commonly used features. The study [ 17 ] empirically claims that the precision of detecting contradiction falls when numeric mismatches are considered.

The structures of the texts also play a vital role when it comes to contradiction detection [ 17, 20 ]. Analysis of text structure is helpful in identifying the common entity or event on which the contradiction is occurring. When the structure of a given sentence is considered, the subject-object relationship plays an important role [ 3, 18 ]. Analysis of Typed Dependency Graphs [ 4 ] is another useful approach to understand the structure of a particular text and to obtain necessary information.

Polarities of the sentences in relation to the sentiments can also play a vital role when it comes to identification of sentence pairs which provide diferent opinions on the same topic. It can be observed the seminal RNTN (Recursive Neural Tensor Network) model [ 27 ] which is trained on movie reviews is used in many recent studies [ 16, 26 ] which perform sentiment analysis. The trained RNTN model [ 27 ] has a bias towards the movie review text[ 8 ]. In order to overcome this problem, the study by Gamage et al [ 8 ] has proposed a methodology to develop a sentiment annotator for the legal domain using transfer learning and has obtained 6% increase in accuracy over the original model [ 27 ] within the legal domain.

The study by de Silva et al [ 7 ] introduces a new algorithm to calculate the oppositeness of triples that can be extracted from microRNA research paper abstracts using open information extraction. As the study proposes a mechanism to detect inconsistencies within paragraphs, we see it as one potential methodology which can be adapted to detect Shift-in-View relationship between sentences. However, as the above-mentioned study [ 7 ] specifically focuses on discovering inconsistencies in the medical domain, it is needed to adopt the proposed methodology to the legal domain in order to detect shift-in-perspectives in legal opinion texts. From this point onward in this paper, we will refer to the study [ 7 ] as the PubMed Study.

In the study [ 32 ], discourse relations between sentences have been used to generate clusters of similar sentences within texts. A Support Vector Machine model is used in this study[ 32 ] to determine the relationships existing between sentences. In the process of Multi-Class classification performed using the SVM Model, the study [ 32 ] has defined a class named Change of Topic which combines the Contradiction and Change of Perspective relations as defined in Cross Document Structure Theory (CST) [ 23 ]. The study[ 32 ] has obtained lower results for Change of Topic than other relationship types and it claims that average results are due to lack of significant features which could properly detect Contradiction and Change of Perspective. CST relations and data from CST bank have also been used to train an SVM model in another study [ 24 ] in order to predict relationships between sentences in the legal domain. Though the study has done improvements to the features in [ 32 ] and introduced new features which suit the legal domain, the results obtained in relation to the Contradiction and Change of Perspective relationships as defined in CST [ 23 ] is very low. One possible reason is that the CST Bank[ 22 ] data set is made up of sentences from newspaper articles, where the structural and linguistic features may difer from that in the court case transcripts, especially when it comes to relationships such as Contradiction and Change of Perspective.

In identifying whether two sentences are providing diferent perspectives or opinions regarding the same topic, it is important to identify whether the two sentences are discussing the same topic. The study [ 24 ] has proposed a successful methodology to identify whether a given two sentences are discussing the same topic or not. In the same study [ 24 ], five relationships that can be observed between two sentences are defined as shown below. From this point onward we refer to the system proposed in the study [ 24 ] as Sentence Relationship Identifier (SRI) .

• Elaboration - One sentence adds more details to the information provided in the preceding sentence or one sentence develops further on the topic discussed in the previous sentence. • Redundancy - Two sentences provide the same information without any diference or additional information. • Citation - A sentence provides references relevant to the details provided in the previous sentence. • Shift-in-View - Two sentences are providing conflicting information or diferent opinions on the same topic or entity. • No Relation - No relationship can be observed between the two sentences. One sentence discusses a topic which is diferent from the topic discussed in another sentence.

It can be seen that the relationship type Shift-in-View defined in SRI study [ 24 ] aligns with the relationship type that is being discussed in this study. It can also be further confirmed by looking at how CST[ 23 ] relationships are adopted in the study[ 24 ].

As shown in Table 1, the Shift-in-View relationship includes both Contradiction and Change of Perspective relationships as defined in CST [ 23 ]. Elaboration, Redundancy, Shift-in-View or Citation relationships defined in the study[ 24 ] suggest that a sentence pair is discussing the same topic while No Relation suggests that the two sentences are discussing completely diferent topics. It has been stated that SRI is able to detect situations where the discussion topic is changed with a considerable accuracy [ 24 ]. However, it is also stated that the proposed methodology is not able to detect situations where two sentences provide diferent opinions on the same topic.The results obtained in this study [ 24 ] are shown in Table 2.

It is clear that the machine learning model inside the SRI is not able to detect Shift-in-View relationship. However, Table 2 shows that the sentences pairs having Shift-in-View relationships are detected as Elaboration. It can be considered as a positive aspect, as Elaboration suggest that both sentences are elaborating on the same topic, which is a necessary condition when detecting sentences providing diferent perspectives on the same topic or entity as described in other studies [ 17, 20 ] too.

METHODOLOGY

Identifying Sentence Pairs where Both

Sentences Discuss the Same Topic

It is needed to identify whether the two sentences are discussing the same topic in detecting sentence pairs which provide diferent opinions on the same topic. Therefore, as the first step, we implemented the Sentence Relationship Identifier(SRI) as it is successful in identifying whether two sentences are discussing on the same topic or not [ 24 ].

According to the study [ 24 ], Elaboration, Redundancy,Citation and Shift-in-View relationships occur when both sentences discuss the same topic. Shift-in-View occurs over Elaboration when the two sentences provide diferent opinions on the same topic.

We only consider sentence pairs which are detected as having Elaboration relationship type in order to identify whether Shiftin-View relationship is present. Though Redundancy and Citation relationship types also suggest that two sentences are discussing the same topic, the sentence pairs detected with those relationship types are not considered. As the Redundancy relationship suggests that two sentences provide similar information, there is no possibility of having diferent perspectives. In Citation relationship, one sentence provides evidence or references to confirm the details presented in the other sentence. Thus, it is not probable to have a situation where two sentences provide diferent perspectives on the same topic.

However, if the machine learning model described in the study [ 24 ] detect a pair of sentences as having Shift-in-View relationship, such a pair will be detected as a sentence pair which provides diferent opinions on the same topic. Confirming the observations of the study [ 24 ], SRI did not identify any pair of sentences as having Shift-in-View relationship. 3.2

Filtering Sentences using Transition Words and Phrases

There are Transition Words or Transition Phrases which suggest that the Source Sentence of a sentence pair is elaborating or building up on the Target Sentence. In the Source Sentence of Example 3 (which was taken from Lee v. United States [ 2 ]), the transition word "Accordingly" implies that the Source Sentence is being developed while agreeing with the Target Sentence.

Therefore, when such a Transition Word or Transition Phrase is present in the Source Sentence, such a sentence pair will be considered as having the Elaboration relationship. As a result, such sentence pairs are not processed further for detecting the Shiftin-View relationship type. We have implemented this mechanism

Example 3 • Sentence 3.1: Lee’s claim that he would not have accepted a plea had he known it would lead to deportation is backed by substantial and uncontroverted evidence. • Sentence 3.2: Accordingly we conclude Lee has demonstrated a “reasonable probability that, but for [his] counsel’s errors, he would not have pleaded guilty and would have insisted on going to trial” as a way to increase the precision of the Shift-in-View detection approaches. Given below are some Transition Words and Transition Phrases we used.

Transition Words: thus, accordingly, therefore

Transition Phrases: as a result, in such cases, because of that, in conclusion, according to that 3.3

Use of Coreferencing

Prior to checking for linguistic features which imply that the sentence pair is showing Shift-in-View relationship, co-referencing is performed on the sentence pair. For coreferencing, Stanford CoreNLP CorefAnnotator (“coref") [ 5 ] was used. The co-referencing provides a better picture when the same entities are being mentioned in the two sentences using diferent names[ 24 ]. 3.4

Analyzing Relationships between Verbs

The first linguistic approach to detect deviations in opinions expressed in sentences regarding a particular topic is based on verb comparison. Under this approach, verbs are compared using the negation relationship and using adverbial modifiers.

In this approach, subject-object pairs in the Target Sentence is compared with that of the Source Sentence. If the subject or object in one sentence is present in the other sentence, the verbs in the sentences are considered. Here, we do not consider verbs which are lemmatized into "be", "do" in order to focus only on efective verbs. The Stanford CoreNLP POS Tagger (“pos") [ 30 ] was used in identifying verbs in sentences. After extracting the verbs in two sentences, each verb in Target sentence is compared with each verb in Source sentence to detect verb pairs with similar meaning. 3.4.1 Determining Verbs which Convey Similar Meanings. In order to convey a similar meaning, it is not necessary that both verbs are the same. Also, when semantic similarity measures between two verbs are considered, it can be observed that there are verb pairs which have very similar meanings but diferent semantic similarity scores. For example, if the lemmatized forms of verbs in Example 1 are considered, it can be observed that the verb demonstrate in the Target sentence and verb show in the source sentence have similar meanings. Confirming that observation further, a Wu-Palmer similarity score of 1.0 can be obtained for that verb pair. When the lemmatized forms of verbs in two sentences in Example 2 are considered, it can be observed that the word "fear" in the Target sentence and "worry" in Source sentence are two verbs with similar meanings. However, the Wu-Palmer semantic similarity score between verbs fear and worry is 0.889. Therefore, it is needed to determine an acceptable threshold based on semantic similarity scores in order to identify verbs with similar meanings.

In order to determine this threshold, we first took 1000 verb pairs from legal opinion texts, whose Wu-Palmer similarity scores are greater than 0.75. As our objective is to identify pairs of verbs with similar meanings, it could be observed that a Wu-Palmer score of 0.75 was a reasonable lower bound as per the precision values. We annotated those 1000 pairs of verbs based on whether a given verb pair actually has two verbs with similar meanings or not. Then we gradually incremented the threshold by 0.1 from 0.75 to 0.95 and observed the precision and recall values as shown in Table 4.

In addition to Wu-Palmer scores, we performed the same experiment on the verb pairs using all the eight semantic similarity measures available in Wordnet[ 21 ]. It was observed that Jiang-Conrath [ 13 ] and Lin [ 15 ] are the two measures which provides reasonable accuracy in addition to Wu-Palmer semantic similarity[ 31 ]. The results from these experiments are shown in Table 4 and in Fig.1. It could be observed that Lin outperforms other two measures when F-Measures are considered. It can be seen that 0.75 is the Lin score which has the highest F-Measure. But, it is due to considerably high recall and undesirably low precision values. As our intention is to maintain a proper balance between precision and recall, Lin Score of 0.86 is selected as the threshold to detect verb pairs with similar meaning. 0.86 is the Lin Score with the second highest F-Measure.

Detecting Shift-in-View Relationships by

Comparing Properties Related to Identified Verbs

3.5.1 Negation on Verbs. Usage of negation relationship is a popular approach when it comes to detecting inconsistencies and contradictions in text [ 7, 9, 17 ]. In this study, we checked for the negation relationship within verbs in verb pairs identified using the method proposed in the section 3.4. If one verb is detected as being negated while the other verb is not being negated, the sentence pair is considered as having Shift-in-View relationship. Stanford CoreNLP dependency parser was used to detect the negation by identifying occurrences of the "neg" tag as described in "Stanford typed dependencies manual" [ 6 ]. 3.5.2 Using Adverbial Modifiers to Detect Shifts-In-View. Another approach to detecting diferent viewpoints on the same subjects or entities can be formulated by considering adverbial modifiers. If the adverbial modifiers related to two verbs with similar meanings give opposite or contradictory meanings, that means the viewpoints on how that task was performed is diferent. Therefore, the adverbial modifiers related to the verbs in verb pairs identified using the methodology described in section 3.4 were considered. We classiifed adverbial modifiers in to three main classes shown in Table 5. Within each class, there exists a positive subclass and a negative subclass. In the table, we have shown the positive sub classes with unshaded rows while the negative sub classes are shown with shaded rows. After defining major classes into which adverbial modifiers can be classified, lists containing adverbs related to each class were created. Table 5 further contains examples of adverbs related to each type. This table does not include all the adverbs we are maintaining in the lists.

If adverbial modifiers connected to both verbs in a verb pair with similar meaning belong to same Adverbial modifier type, but with opposite polarities (one positive and one negative), it can be identified that the two sentences provide diferent views in relation to the entities that are connected by those verbs. 3.6

Discovering Inconsistencies between

Triples

Following the methodology presented in the PubMed study [ 7 ], a legal term dictionary was constructed to be served as a Semantic Lexicon for the system. 200+ legal opinion texts were used to extract words for the process. Then a word list consisting 17,000+ unique words were developed by removing stop words. A TF-IDF algorithm [ 14 ] based method is used to calculate a value for each term in the dictionary.

Ícasecount ft,d T ermV alue = i=1 t ermcount (1)

D.F

Raw count (ft,d ) for each term is taken, considering each legal opinion text as a seperate document. Term frequency value for a term is calculated by dividing the raw term count by the total number of terms in the case. Term frequency value for each case is added together and the result value is divided from the document frequency (D.F ), to calculate the value for a term in the dictionary. Then all the term values are normalized according to the equation 2.

N ormalizedTV = (TV − TVmin ) ∗ (1 − TVmin ) + TVmin (2)

TVmax − TVmin

Here TVmin and TVmax represent the minimum and maximum values of the term values respectively. This normalized value is used to be served as the semantic weight for the system.

First, coreference resolving is done on the sentence pairs using the Stanford CoreNLP CorefAnnotator [ 5 ] and the pairs with Transition Words and Phrases are filtered out. Then OLLIE [ 18 ], open information extraction system, is used to extract triples, in (Subject; Relationship; Object) format, from sentences. When comparing two sentences, for the Shift In View relationship, only triple pairs with same subject or object are considered, as the Shift In View relationship talks about diferent perspectives on the same topic or entity. The stop words removed relationship strings of a triple pair are then compared with each other word by word. The comparison is performed in three ways.

(1) Words which are exactly the same (2) Exactly same words with one word negated with “not" (3) Diferent words

In our study we consider the negation of words with similar meanings (Lin score above 0.86) instead of considering only the words which are exactly the same. Then, an oppositeness value is obtained for each sentence pair by comparing the triples following the algorithmic approach proposed in the PubMed Study [ 7 ]. A threshold based on the oppositeness values is introduced empirically to select sentence pairs which have the Shift In View relationship. 3.7

Sentiment-based Approach

Though valuable information can be obtained by analyzing the sentiment of a sentence, the sentiment of a sentence alone hardly gives any details on the topics which are being discussed within a sentence and on the viewpoint in which the sentence is describing the topic. It is known that the two sentences which are being compared to detect shifts in view discuss on the same topic as we consider only the sentence pairs with "Elaboration" relationship. But, when the sentences in legal opinion texts are considered, even if the sentiments of two sentences which elaborate on the same discussion topic is diferent, it can not be concluded that the two sentences are providing diferent opinions on the topic.

The reason is that the person entities which are described in a sentence and connected with the sentiment of the sentence have a significant impact on the topic which is being discussed. For example, consider two sentences which elaborate on the same discussion topic and having opposite sentiments. If the sentiment of the sentence with negative sentiment is connected with the proposition party while the sentiment of sentence with positive sentiment is connected with the opposition party, it might be the case where both sentences are conveying opinions which are beneficial for the opposition party in relation to the topic which is being discussed.

The problem becomes even more complex when the sentence is made up of several sub-sentences because each sub-sentence may have a "Subject" of its own. Therefore, when using the sentiment based approach to detect "Shift-in-View" relationship, we consider only the sentence pairs in which each sentence has only one explicit subject. If the subjects in both sentences are the same in such a sentence pair, it can be concluded that two sentences are elaborating on the same topic in relation to the same subject. Then, it is checked whether the two sentences are providing sentiments with opposite polarities. If one sentence provides negative sentiment and other provides positive sentiment while discussing the same topic in relation to the same subject, it can be concluded that the probability of two sentences giving diferent perspectives on the same topic is very significant.

In this approach, the sentences which are composed with subordinate clauses are first split using those clauses. When the sentence is split using a subordinating conjunction, that subordinate clause can be identified as another sentence entity. Throughout this section, we will refer the subordinate clause as inner sentence and the main clause will be referred to as outer sentence. After the sentence is annotated using Stanford CoreNLP Constituency Parser [ 4 ] , the splitting happens by identifying associated terms with SBAR tag.

The proposed approach is based on analyzing the sentiment of this inner sentence to identify if there is a shift in view relation between a sentence pair. If we consider the Example 4 (which was taken from Lee v. United States [ 2 ]), The phrases “Lee cannot convince the court that a decision to reject the plea bargain”, and “he can establish prejudice under Hill” are the inner sentences. The outer sentences are “The government argues”, and “Lee, on the other hand, argues”.

Example 4 • Sentence 4.1: The Government argues that Lee cannot "convince the court that a decision to reject the plea bargain. • Sentence 4.2: Lee, on the other hand, argues that he can establish prejudice under Hill.

If we consider the sentence pair mentioned in Example 4, both the inner sentences’ subject is Lee. The phrase “Lee cannot convince the court that a decision to reject the plea bargain” is having a negative sentiment while the other inner sentence “Lee can establish prejudice under Hill” denotes a positive sentiment. Both the outer sentences are having neutral sentiment. Therefore, it can be observed that there is a shift in view regarding the subject Lee. 4

EXPERIMENTS AND RESULTS

As the first step, the 3 major approaches used to detect Shift-inView relationship type were evaluated. In order to perform this evaluation, 2150 sentence pairs from legal opinion texts related to criminal court cases were extracted from Findlaw [ 1 ]. Each of these sentence pairs contains two sentences which are consecutive to each other within a legal opinion text document. Next, the extracted sentence pairs were input into the Sentence Relationship Identifier (SRI). Input sentence pairs are first processed inside SRI. The sentence pairs which are identified as having Elaboration by the SRI were further processed in order to detect whether there is Shift-in-View relationship using the three Shift-in-View detection approaches mentioned under Section III.

As the next step, the sentence pairs detected as having the Shiftin-View relationship under each approach were taken into consideration. The number of detected sentence pairs from each approach is shown in Table 6. Then, the precision of each approach was calculated. All 46 sentence pairs detected from Verb-Relationship approach were used when calculating the precision of that approach. 100 sentence pairs randomly selected from the detected 246 sentence pairs, which were identified using the Sentiment-Polarity approach was used to determine the precision of the approach. 95 sentence pairs were detected from the approach which uses inconsistencies between triples to determine Shift-in-View. All of those 95 sentence pairs were used to calculate the precision of that approach. The precision values obtained for each of these approach are also shown in Table 6. When performing this evaluation, each sentence pair was first annotated by two human judges. If the two judges did not agree on a relationship type for a particular sentence pair, that sentence pair was annotated by an additional human judge. When the results were calculated, the consideration was given only to the sentence pairs which were agreed by at least two human judges to have the same relationship type.

Due to the scarcity of resources, it was not possible to annotate all 2150 sentence pairs based on the relationship type. As a result, calculating recall of each approach was not possible. If Table 2 related to SRI study [ 24 ] is considered, it can be observed that only 3 out of 165 sentence pairs are determined as having the Shift-in-View relationship type by the human judges. It suggests that the Shift-in-View relationship type does not occur frequently when we consider sequential sentence pairs in a legal opinion text. Furthermore, Table 2 suggests that the SRI tends to misattribute sentence pairs having Shift-in-View as having the Elaboration relationship type. That means, the SRI is successful in determining if the two sentences in the sentence pair is discussing the same topic or not. In such circumstances, it is important to be precise when determining a sentence pair as having the Shift-in-View relationship type. Considering these facts, we can conclude that it is important to prioritize the Shift-in-View detection approaches based on the precision.

According to the Table 6, it can be seen that the precision which could be obtained from analyzing relationships between verbs is around 0.6. As mentioned earlier we have selected the Lin semantic similarity score of 0.86 as the threshold to identify verbs with similar meaning after analyzing diferent semantic similarity measures. The precision of identifying verbs with 0.86 Lin score is 0.67. Thus, it can be seen that there is a potential to improve the precision of detecting Shift-in-View relationships using relationships between verbs by developing a semantic similarity measure which is more accurate in identifying verbs with similar meanings for the legal domain.

Using the sentiment based model, the achieved precision is 0.38. There are few possible reasons behind this observation. The study on the sentiment annotator model [ 8 ] used in this case, states that the accuracy of the model is 76%. The study says that the errors present in its parent model [ 27 ] can be propagated to the target model [ 8 ]. The paper on the source model[ 27 ] which is based on recursive neural tensor network, shows that the accuracy is reduced down to 0.5 when the n-gram length of a phrase increases (n>10). As most of the sentences in court case transcripts are reasonably lengthier, there is a potential that the proposed sentiment based approach used for the identification of Shift-in-View is afected by the above mentioned error.

Only a precision of 0.27 could be observed in the approach which considers inconsistencies using triples as proposed in PubMed Study. The following reasons may have contributed to the poor performances of that approach. From the 2150 sentence pairs which were considered, oppositeness values were not calculated for 1570 pairs. Containing at least one sentence within a sentence pair in which the triples could not be extracted by OLLIE[ 18 ] is a major reason for not having an oppositeness value. Even if the triples are extracted from both sentences, if there is no matching between either subjects or objects of the two sentences, an oppositeness value will not be calculated for a sentence pair.

Evaluation results demonstrates that analysis of relationships between verbs in two sentences as the only approach which performs the task of detecting Shift-in-View relationships with a precision more than 0.5. Many studies convince the dificulty of detecting contradiction and change of perspective relationships over other relationship types that can be observed between sentences[ 17, 24, 32 ]. The study [ 17 ] also claims the dificulty of generalizing contradiction detection approaches. When considering these facts, it can be considered that the results obtained via analyzing verb relationships are satisfactory. Therefore, we combined only that approach with the Sentence Relationship Identifier (SRI) and evaluated the overall system made up by combining Shift-in-View detection with SRI as shown in Table 7.

The results shown in Table 7 were obtained using 200 annotated sentence pairs. Each of the considered sentence pair was agreed by at least two human judges to have the same relationship type. Furthermore, 21 randomly selected sentence pairs which were agreed by at least two human judges as having Shift-in-View are contained within the 200 sentence pairs which were used in this evaluation. improvement, especially in relation to the Shift-in-View relationship type when compared with the results in the study[ 24 ] as given in Table 3. 5

CONCLUSION

Developing a methodology to detect situations where multiple viewpoints are provided in regard to the same discussion topic within a legal opinion text is the major research contribution of this study. This study has introduced novel approaches to detect deviations in the opinions provided by two sentences regarding the same topic. At the same time, existing methodologies to detect contradiction and change of perspectives have been evaluated within the study. Additionally, it has been empirically demonstrated the way in which the outcomes of the study can be used to facilitate the process of identifying relationships between sentences in documents containing legal opinions on court cases. Evaluation of the performance of existing semantic similarity measures in relation to identifying verbs with similar meaning can be considered as another key research contribution of the study.

The proposed approach can also be used to facilitate several other information extraction tasks related to the legal domain such as identifying counter arguments to a particular argument, determining representatives related to the proposition party and the opposition party in a court case.

The accuracy of the approaches proposed in this study can be further improved by developing semantic similarity measures and sentiment annotators which can perform in the legal domain with an improved accuracy. Coming up with such mechanisms can be considered as the major future work.

References

[1] [n. d.]. Caselaw: Cases and Codes - FindLaw Caselaw. https://caselaw.findlaw. com/. ([n. d.]) . (Accessed on 05/20/ 2018 ).

[2] 1977 . Lee v. United States . In US , Vol. 432 . Supreme Court, 23 .

[3] Gabor Angeli , Melvin Jose Johnson Premkumar, and Christopher D Manning . 2015 . Leveraging linguistic structure for open domain information extraction . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1 : Long Papers) , Vol. 1 . 344 - 354 .

[4] Danqi Chen and Christopher Manning . 2014 . A fast and accurate dependency parser using neural networks . In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 740 - 750 .

[5] Kevin Clark and Christopher D. Manning . 2015 . Entity-Centric Coreference Resolution with Model Stacking. In Association for Computational Linguistics (ACL).

[6] Marie-Catherine De Marnefe and Christopher D Manning . 2008 . Stanford typed dependencies manual . Technical Report. Technical report , Stanford University.

[7] Nisansa de Silva , Dejing Dou, and Jingshan Huang . 2017 . Discovering Inconsistencies in PubMed Abstracts Through Ontology-Based Information Extraction . In Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics. ACM , 362 - 371 .

[8] Viraj Gamage , Menuka Warushavithana, Nisansa de Silva, Amal Shehan Perera, Gathika Ratnayaka, and Thejan Rupasinghe . 2018 . Fast Approach to Build an Automatic Sentiment Annotator for Legal Domain using Transfer Learning . arXiv preprint arXiv: 1810 . 01912 ( 2018 ).

[9] Jesse A Harris and Christopher Potts . 2009 . Perspective-shifting with appositives and expressives . Linguistics and Philosophy 32 , 6 ( 2009 ), 523 - 552 .

[10] Vindula Jayawardana , Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Keet Sugathadasa, and Buddhi Ayesha . 2017 . Deriving a representative vector for ontology classes with instance word vector embeddings . In Innovative Computing Technology (INTECH) , 2017 Seventh International Conference on. IEEE , 79 - 84 .

[11] Vindula Jayawardana , Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Keet Sugathadasa, Buddhi Ayesha, and Madhavi Perera . 2017 . Semi-supervised instance population of an ontology using word vector embedding . In Advances in ICT for Emerging Regions (ICTer) , 2017 Seventeenth International Conference on. IEEE , 1- 7 .

[12] Vindula Jayawardana , Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Keet Sugathadasa, Buddhi Ayesha, and Madhavi Perera . 2017 . Word Vector Embeddings and Domain Specific Semantic based Semi-Supervised Ontology Instance Population . International Journal on Advances in ICT for Emerging Regions 10 , 1 ( 2017 ), 1 .

[13] Jay J Jiang and David W Conrath. 1997 . Semantic similarity based on corpus statistics and lexical taxonomy . arXiv preprint cmp-lg/9709008 ( 1997 ).

[14] Jure Leskovec , Anand Rajaraman, and Jefrey David Ullman. 2014 . Mining of massive datasets . Cambridge university press.

[15] Dekang Lin et al. 1998 . An information-theoretic definition of similarity. . In Icml , Vol. 98 . Citeseer , 296 - 304 .

[16] Christopher Manning , Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky . 2014 . The Stanford CoreNLP natural language processing toolkit . In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations . 55 - 60 .

[17] Marie-Catherine Marnefe , Anna N Raferty , and Christopher D Manning . 2008 . Finding contradictions in text . Proceedings of ACL-08: HLT ( 2008 ), 1039 - 1047 .

[18] Mausam , Michael Schmitz , Robert Bart, Stephen Soderland, and Oren Etzioni . 2012 . Open Language Learning for Information Extraction . In Proceedings of Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CONLL).

[19] John J Nay . 2016 . Gov2vec: Learning distributed representations of institutions and their legal text . arXiv preprint arXiv:1609.06616 ( 2016 ).

[20] Michael J Paul, ChengXiang Zhai, and Roxana Girju . 2010 . Summarizing contrastive viewpoints in opinionated text . In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics , 66 - 76 .

[21] Ted Pedersen , Siddharth Patwardhan, and Jason Michelizzi . 2004 . WordNet:: Similarity: measuring the relatedness of concepts . In Demonstration papers at HLT-NAACL 2004. Association for Computational Linguistics , 38 - 41 .

[22] Dragomir Radev , Jahna Otterbacher, and Zhu Zhang . 2003 . CSTBank: Crossdocument Structure Theory Bank . http://tangra.si.umich.edu/clair/CSTBank. ( 2003 ).

[23] Dragomir R Radev . 2000 . A common theory of information fusion from multiple text sources step one: cross-document structure . In Proceedings of the 1st SIGdial workshop on Discourse and dialogue-Volume 10. Association for Computational Linguistics , 74 - 83 .

[24] Gathika Ratnayaka , Thejan Rupasinghe, Nisansa de Silva, Menuka Warushavithana, Viraj Gamage, and Amal Shehan Perera. 2018 . Identifying Relationships Among Sentences in Court Case Transcripts Using Discourse Relations . In 2018 18th International Conference on Advances in ICT for Emerging Regions (ICTer) . IEEE, 13 - 20 .

[25] Erich Schweighofer and Werner Winiwarter . 1993 . Legal expert system KONTERM - automatic representation of document structure and contents . In International Conference on Database and Expert Systems Applications . Springer, 486 - 497 .

[26] Richard Socher , Danqi Chen, Christopher D Manning , and Andrew Ng . 2013 . Reasoning with neural tensor networks for knowledge base completion . In Advances in neural information processing systems . 926 - 934 .

[27] Richard Socher , Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning , Andrew Ng , and Christopher Potts . 2013 . Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 conference on empirical methods in natural language processing . 1631 - 1642 .

[28] Keet Sugathadasa , Buddhi Ayesha, Nisansa de Silva, Amal Shehan Perera, Vindula Jayawardana, Dimuthu Lakmal, and Madhavi Perera . 2017 . Synergistic union of word2vec and lexicon for domain specific semantic similarity . In Industrial and Information Systems (ICIIS) , 2017 IEEE International Conference on. IEEE , 1- 6 .

[29] Keet Sugathadasa , Buddhi Ayesha, Nisansa de Silva, Amal Shehan Perera, Vindula Jayawardana, Dimuthu Lakmal, and Madhavi Perera . 2018 . Legal Document Retrieval using Document Vector Embeddings and Deep Learning . arXiv preprint arXiv: 1805 . 10685 ( 2018 ).

[30] Kristina Toutanova , Dan Klein, Christopher D Manning , and Yoram Singer . 2003 . Feature-rich part-of-speech tagging with a cyclic dependency network . In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1. Association for Computational Linguistics , 173 - 180 .

[31] Zhibiao Wu and Martha Palmer . 1994 . Verbs semantics and lexical selection . In Proceedings of the 32nd annual meeting on Association for Computational Linguistics. Association for Computational Linguistics , 133 - 138 .

[32] Nik Adilah Hanin Zahri , Fumiyo Fukumoto, and Suguru Matsuyoshi . 2012 . Exploiting Discourse Relations between Sentences for Text Clustering . In 24th International Conference on Computational Linguistics . 17 .