Shift-of-Perspective Identification within Legal Cases Gathika Ratnayaka Thejan Rupasinghe Nisansa de Silva gathika.14@cse.mrt.ac.lk thejanrupasinghe.14@cse.mrt.ac.lk nisansaDdS@cse.mrt.ac.lk Department of Computer Science & Department of Computer Science & Department of Computer Science & Engineering Engineering Engineering University of Moratuwa University of Moratuwa University of Moratuwa Moratuwa, Sri Lanka Moratuwa, Sri lanka Moratuwa, Sri Lanka Viraj Salaka Gamage Menuka Warushavithana Amal Shehan Perera viraj.14@cse.mrt.ac.lk menuka.14@cse.mrt.ac.lk shehan@cse.mrt.ac.lk Department of Computer Science & Department of Computer Science & Department of Computer Science & Engineering Engineering Engineering University of Moratuwa University of Moratuwa University of Moratuwa Moratuwa, Sri Lanka Moratuwa, Sri Lanka Moratuwa, Sri Lanka ABSTRACT In the process of interpreting the meaning of a text, understanding Arguments, counter-arguments, facts, and evidence obtained via the context can be considered as a major requirement, especially in documents related to previous court cases are of essential need for the legal literature. legal professionals. Therefore, the process of automatic information Identifying how textual units are related to each other within extraction from documents containing legal opinions related to a machine-readable text is an important task when it comes to in- court cases can be considered to be of significant importance. This terpreting the context. Humans are good at comparing two textual study is focused on the identification of sentences in legal opinion units to determine the way in which those two units are connected. texts which convey different perspectives on a certain topic or Granting this ability to computers is a major discussion topic in entity. We combined several approaches based on semantic analysis, the research related to areas of Natural Language Processing and open information extraction, and sentiment analysis to achieve our Artificial Intelligence. A sentence can be considered as a textual objective. Then, our methodology was evaluated with the help of unit with significant importance in a text. Therefore, analysis of human judges. The outcomes of the evaluation demonstrate that relationships between sentences can be useful to get a clear pic- our system is successful in detecting situations where two sentences ture on the information flow within a text which is made up of a deliver different opinions on the same topic or entity. The proposed considerable number of sentences. methodology can be used to facilitate other information extraction Similarly, identifying the types of relationships existing between tasks related to the legal domain. One such task is the automated sentences in legal opinion texts can be used to identify the informa- detection of counter arguments for a given argument. Another is tion flow within a legal case. Within a document describing legal the identification of opponent parties in a court case. opinions related to a court case, different types of relationships between sentences can be observed such as elaboration and contra- KEYWORDS diction. Pairs of sentences can be classified into two major groups based on whether the topics which are being discussed by the two semantic analysis, sentiment analysis, natural language processing, sentences in the sentence pair is the same or not. In other words, information extraction, law the two sentences in a sentence pair may discuss the same topic 1 INTRODUCTION or they may discuss completely different topics. Even if the two sentences are discussing the same topic, the opinions or views pre- Documents describing legal opinions related to previous court cases sented in the two sentences on the topic may be different. Consider carry a significant importance when it comes to the legal literature. the following sentence pair taken from Lee v. United States [2]. The information presented in these legal opinion texts are used in different capacities such as evidence, arguments, and facts by legal officials in the process of constructing new legal cases [29]. Example 1 Therefore, information extraction from legal opinion texts can be • Sentence 1.1: Applying the two-part test for ineffective assistance claims from Strick- considered as an area of significant importance, within the topic of land v. Washington, 466 U. S. 668, the Sixth Circuit concluded that, while the Govern- ment conceded that Lee’s counsel had performed deficiently, Lee could not show that automatic information extraction in the legal domain. In order he was prejudiced by his attorney’s erroneous advice. to perform systematic information extraction from a legal opinion • Sentence 1.2: Lee has demonstrated that he was prejudiced by his counsel’s erroneous text, a system should be able to interpret the meaning of a given text. advice. In: Proceedings of the Third Workshop on Automated Semantic Analysis of Information in Legal Text (ASAIL 2019), June 21, 2019, Montreal, QC, Canada. © 2019 Copyright held by the owner/author(s). Copying permitted for private and The above two sentences discuss whether a person named Lee academic purposes. Published at http://ceur-ws.org was able to convince that he was prejudiced by his attorney’s advice or not. While the first sentence says that Lee could not show that he ASAIL 2019, June 21, 2019, Montreal, QC, Canada Ratnayaka and Rupasinghe, et al. was prejudiced by his attorney’s advice, the second sentence contra- Regardless, there have been some recent attempts to circum- dicts the first sentence by saying that Lee has demonstrated that he vent these problems in the legal domain including information was prejudiced by his counsel’s erroneous advice. Thus, the two sen- organization [10–12], information extraction [28] and information tences provide different opinions on the same topic. Contradiction retrieval [29]. Going forward, owing to the popularity of knowledge is not a necessary condition in order to classify a pair of sentences embedding in the literature, several studies have taken up the task as providing different opinions on the same topic. For example, con- of embedding legal jargon in vector spaces [19, 28]. Further, in the sider Example 2 which consists of two adjacent sentences which information extraction domain, the study by Gamage et al [8] at- are also taken from Lee v. United States [2]. tempted to build a sentiment annotator for the legal domain and the study by Ratnayaka et al [24] attempted to identify relationships among sentences in legal opinion texts. Example 2 Discovering situations where two sentences are providing differ- • Sentence 2.1: Although he has lived in this country for most of his life, Lee is not a United States citizen, and he feared that a criminal conviction might affect his status ent opinions on the same topic or entity is an important part when as a lawful permanent resident. it comes to identifying relationships among sentences [24]. Contra- • Sentence 2.2: His attorney assured him there was nothing to worry about–the Gov- diction is a sufficient but not a necessary condition in this regard. ernment would not deport him if he pleaded guilty. The study [17] is focused on finding contradictions in text related to the real world context. In an attempt to define contradiction, the same study[17] claims that “contradiction occurs when two sen- It can be seen that both sentences in this example discuss the tences are extremely unlikely to be true simultaneously” and the topic – the deportation of a person named Lee. Though the two sen- study [20] also agrees on that definition. However, the study [17] tences here do not provide contradictory information, they provide also demonstrates that two sentences can be contradictory while two different viewpoints regarding the same topic. It can be seen being true simultaneously. These characteristics of contradiction that the opinions of Lee and his attorney on the possibility of Lee make the process of detecting contradiction relationships more being deported is different. Therefore, when discussing sentences complex. with different opinions on the same topic, not only the sentences In order to become contradictory, two textual units can elabo- providing contradictory information but also the sentences provid- rate not only on the same event but also on the same entity. For ing multiple viewpoints on the same discussion topic should also be example, if one sentence in a sentence pair is saying that a person considered. In each of the above two examples, Sentence 1 comes is a United States citizen while the other sentence is saying that before Sentence 2. From this point onward, the first sentence in very same person is not a United States citizen, it is obvious that a sentence pair will be referred to as the Target Sentence and the the two sentences are providing contradictory information. Here, second sentence as the Source Sentence. the contradictory information is upon a person which can be con- An important observation which can be made by considering sidered as an entity. Therefore, it is more reasonable to consider Example 2 is that the identification of the shift in the viewpoint in that in order to be contradictory, texts must elaborate on the same that particular occasion is not straightforward. This implicit nature topic. makes the task of identifying sentences which provides different In order to detect contradiction, different features based on the opinions on the same discussion topic even more challenging. At properties of text have been considered in the previous studies the same time, it can be considered a vital task due to its potential [17, 20]. Polarity features and Numeric Mismatches are such com- to enhance the capabilities of Information Extraction from Legal monly used features. The study [17] empirically claims that the Text by facilitating automatic detection of counter-arguments, iden- precision of detecting contradiction falls when numeric mismatches tification of the stance of a particular party in a court case and to are considered. discover multiple viewpoints to analyze or evaluate a particular The structures of the texts also play a vital role when it comes legal situation. to contradiction detection [17, 20]. Analysis of text structure is Hence, the objective of this study is to identify sentences which helpful in identifying the common entity or event on which the have different perspectives on the same discussion topic in a given contradiction is occurring. When the structure of a given sentence court case. For this study, legal opinion texts related to United is considered, the subject-object relationship plays an important States court cases were used.The next section provides details on role [3, 18]. Analysis of Typed Dependency Graphs [4] is another the previous work which are related to our study. Section 3 describes useful approach to understand the structure of a particular text and the methodology followed in this study while the outcomes of the to obtain necessary information. study are discussed in Section 4. Finally we conclude our discussion Polarities of the sentences in relation to the sentiments can in Section 5. also play a vital role when it comes to identification of sentence pairs which provide different opinions on the same topic. It can be 2 RELATED WORK observed the seminal RNTN (Recursive Neural Tensor Network) Computing applications which can be considered to be both efficient model [27] which is trained on movie reviews is used in many recent and effective are scarce due to the challenges in handling legal studies [16, 26] which perform sentiment analysis. The trained jargon[11, 12, 25]. The nature of legal documents employing a RNTN model [27] has a bias towards the movie review text[8]. In vocabulary of mixed origin ranging from Latin to English has been order to overcome this problem, the study by Gamage et al [8] has put forward as a reasoning for difficulties of building computing proposed a methodology to develop a sentiment annotator for the applications for the legal domain [29]. Shift-of-Perspective Identification within Legal Cases ASAIL 2019, June 21, 2019, Montreal, QC, Canada legal domain using transfer learning and has obtained 6% increase • No Relation - No relationship can be observed between in accuracy over the original model [27] within the legal domain. the two sentences. One sentence discusses a topic which is The study by de Silva et al [7] introduces a new algorithm to different from the topic discussed in another sentence. calculate the oppositeness of triples that can be extracted from It can be seen that the relationship type Shift-in-View defined microRNA research paper abstracts using open information extrac- in SRI study [24] aligns with the relationship type that is being tion. As the study proposes a mechanism to detect inconsistencies discussed in this study. It can also be further confirmed by looking within paragraphs, we see it as one potential methodology which at how CST[23] relationships are adopted in the study[24]. can be adapted to detect Shift-in-View relationship between sen- tences. However, as the above-mentioned study [7] specifically Table 1: Adopting CST Relationships [24] focuses on discovering inconsistencies in the medical domain, it is needed to adopt the proposed methodology to the legal domain Definition CST Relationships in order to detect shift-in-perspectives in legal opinion texts. From this point onward in this paper, we will refer to the study [7] as the Paraphrase, Modality, Subsumption, Elaboration, PubMed Study. Indirect Speech, Follow-up, Overlap, Elaboration In the study [32], discourse relations between sentences have Fulfillment, Description, Historical Background, been used to generate clusters of similar sentences within texts. Reader Profile, Attribution A Support Vector Machine model is used in this study[32] to de- Redundancy Identity termine the relationships existing between sentences. In the pro- Citation Citation cess of Multi-Class classification performed using the SVM Model, Shift-in-View Change of Perspective,Contradiction the study [32] has defined a class named Change of Topic which No Relation - combines the Contradiction and Change of Perspective relations as defined in Cross Document Structure Theory (CST) [23]. The As shown in Table 1, the Shift-in-View relationship includes both study[32] has obtained lower results for Change of Topic than other Contradiction and Change of Perspective relationships as defined relationship types and it claims that average results are due to lack in CST [23]. Elaboration, Redundancy, Shift-in-View or Citation of significant features which could properly detect Contradiction relationships defined in the study[24] suggest that a sentence pair and Change of Perspective. CST relations and data from CST bank is discussing the same topic while No Relation suggests that the two have also been used to train an SVM model in another study [24] sentences are discussing completely different topics. It has been in order to predict relationships between sentences in the legal stated that SRI is able to detect situations where the discussion domain. Though the study has done improvements to the features topic is changed with a considerable accuracy [24]. However, it in [32] and introduced new features which suit the legal domain, is also stated that the proposed methodology is not able to detect the results obtained in relation to the Contradiction and Change of situations where two sentences provide different opinions on the Perspective relationships as defined in CST [23] is very low. One same topic.The results obtained in this study [24] are shown in possible reason is that the CST Bank[22] data set is made up of Table 2. sentences from newspaper articles, where the structural and lin- guistic features may differ from that in the court case transcripts, Table 2: Confusion Matrix from the Sentence Relationship especially when it comes to relationships such as Contradiction and Identifier study [24] Change of Perspective. Shift-in-View In identifying whether two sentences are providing different No Relation Elaboration perspectives or opinions regarding the same topic, it is important Citation to identify whether the two sentences are discussing the same topic. The study [24] has proposed a successful methodology to Predicted identify whether a given two sentences are discussing the same Σ Actual topic or not. In the same study [24], five relationships that can be Elaboration 93.9% 6.1% 0.0% 0.0% 99 observed between two sentences are defined as shown below. From No Relation 11.9% 88.1% 0.0% 0.0% 42 this point onward we refer to the system proposed in the study [24] Citation 0.0% 4.8% 95.2% 0.0% 21 as Sentence Relationship Identifier (SRI). Shift-in-View 100.0% 0.0% 0.0% 0.0% 3 Σ 101 44 20 0 165 • Elaboration - One sentence adds more details to the infor- mation provided in the preceding sentence or one sentence develops further on the topic discussed in the previous sen- It is clear that the machine learning model inside the SRI is not tence. able to detect Shift-in-View relationship. However, Table 2 shows • Redundancy - Two sentences provide the same information that the sentences pairs having Shift-in-View relationships are de- without any difference or additional information. tected as Elaboration. It can be considered as a positive aspect, • Citation - A sentence provides references relevant to the as Elaboration suggest that both sentences are elaborating on the details provided in the previous sentence. same topic, which is a necessary condition when detecting sen- • Shift-in-View - Two sentences are providing conflicting tences providing different perspectives on the same topic or entity information or different opinions on the same topic or entity. as described in other studies [17, 20] too. ASAIL 2019, June 21, 2019, Montreal, QC, Canada Ratnayaka and Rupasinghe, et al. Table 3: Results from Sentence Relationship Identifier study Example 3 considering Sentence Pairs where Both Judges Agree [24] • Sentence 3.1: Lee’s claim that he would not have accepted a plea had he known it would lead to deportation is backed by substantial and uncontroverted evidence. • Sentence 3.2: Accordingly we conclude Lee has demonstrated a “reasonable probability Discourse Class Precision Recall F-Measure that, but for [his] counsel’s errors, he would not have pleaded guilty and would have Elaboration 0.921 0.939 0.930 insisted on going to trial” No Relation 0.841 0.881 0.861 Citation 1.000 0.952 0.975 Shift-in-View - 0 - as a way to increase the precision of the Shift-in-View detection approaches. Given below are some Transition Words and Transition Phrases we used. 3 METHODOLOGY Transition Words: thus, accordingly, therefore 3.1 Identifying Sentence Pairs where Both Transition Phrases: as a result, in such cases, because of that, in Sentences Discuss the Same Topic conclusion, according to that It is needed to identify whether the two sentences are discussing the same topic in detecting sentence pairs which provide different 3.3 Use of Coreferencing opinions on the same topic. Therefore, as the first step, we imple- Prior to checking for linguistic features which imply that the sen- mented the Sentence Relationship Identifier(SRI) as it is successful tence pair is showing Shift-in-View relationship, co-referencing in identifying whether two sentences are discussing on the same is performed on the sentence pair. For coreferencing, Stanford topic or not [24]. CoreNLP CorefAnnotator (“coref") [5] was used. The co-referencing According to the study [24], Elaboration, Redundancy,Citation provides a better picture when the same entities are being men- and Shift-in-View relationships occur when both sentences discuss tioned in the two sentences using different names[24]. the same topic. Shift-in-View occurs over Elaboration when the two sentences provide different opinions on the same topic. 3.4 Analyzing Relationships between Verbs We only consider sentence pairs which are detected as having The first linguistic approach to detect deviations in opinions ex- Elaboration relationship type in order to identify whether Shift- pressed in sentences regarding a particular topic is based on verb in-View relationship is present. Though Redundancy and Citation comparison. Under this approach, verbs are compared using the relationship types also suggest that two sentences are discussing the negation relationship and using adverbial modifiers. same topic, the sentence pairs detected with those relationship types In this approach, subject-object pairs in the Target Sentence is are not considered. As the Redundancy relationship suggests that compared with that of the Source Sentence. If the subject or object two sentences provide similar information, there is no possibility of in one sentence is present in the other sentence, the verbs in the having different perspectives. In Citation relationship, one sentence sentences are considered. Here, we do not consider verbs which provides evidence or references to confirm the details presented are lemmatized into "be", "do" in order to focus only on effective in the other sentence. Thus, it is not probable to have a situation verbs. The Stanford CoreNLP POS Tagger (“pos") [30] was used in where two sentences provide different perspectives on the same identifying verbs in sentences. After extracting the verbs in two topic. sentences, each verb in Target sentence is compared with each verb However, if the machine learning model described in the study in Source sentence to detect verb pairs with similar meaning. [24] detect a pair of sentences as having Shift-in-View relationship, such a pair will be detected as a sentence pair which provides 3.4.1 Determining Verbs which Convey Similar Meanings. In order different opinions on the same topic. Confirming the observations to convey a similar meaning, it is not necessary that both verbs are of the study [24], SRI did not identify any pair of sentences as the same. Also, when semantic similarity measures between two having Shift-in-View relationship. verbs are considered, it can be observed that there are verb pairs which have very similar meanings but different semantic similarity 3.2 Filtering Sentences using Transition Words scores. For example, if the lemmatized forms of verbs in Example 1 are considered, it can be observed that the verb demonstrate in and Phrases the Target sentence and verb show in the source sentence have simi- There are Transition Words or Transition Phrases which suggest that lar meanings. Confirming that observation further, a Wu-Palmer the Source Sentence of a sentence pair is elaborating or building similarity score of 1.0 can be obtained for that verb pair. When up on the Target Sentence. In the Source Sentence of Example 3 the lemmatized forms of verbs in two sentences in Example 2 are (which was taken from Lee v. United States [2]), the transition word considered, it can be observed that the word "fear" in the Target "Accordingly" implies that the Source Sentence is being developed sentence and "worry" in Source sentence are two verbs with simi- while agreeing with the Target Sentence. lar meanings. However, the Wu-Palmer semantic similarity score Therefore, when such a Transition Word or Transition Phrase is between verbs fear and worry is 0.889. Therefore, it is needed to present in the Source Sentence, such a sentence pair will be con- determine an acceptable threshold based on semantic similarity sidered as having the Elaboration relationship. As a result, such scores in order to identify verbs with similar meanings. sentence pairs are not processed further for detecting the Shift- In order to determine this threshold, we first took 1000 verb pairs in-View relationship type. We have implemented this mechanism from legal opinion texts, whose Wu-Palmer similarity scores are Shift-of-Perspective Identification within Legal Cases ASAIL 2019, June 21, 2019, Montreal, QC, Canada greater than 0.75. As our objective is to identify pairs of verbs with how that task was performed is different. Therefore, the adverbial similar meanings, it could be observed that a Wu-Palmer score of modifiers related to the verbs in verb pairs identified using the 0.75 was a reasonable lower bound as per the precision values. We methodology described in section 3.4 were considered. We classi- annotated those 1000 pairs of verbs based on whether a given verb fied adverbial modifiers in to three main classes shown in Table pair actually has two verbs with similar meanings or not. Then we 5. Within each class, there exists a positive subclass and a nega- gradually incremented the threshold by 0.1 from 0.75 to 0.95 and tive subclass. In the table, we have shown the positive sub classes observed the precision and recall values as shown in Table 4. with unshaded rows while the negative sub classes are shown with In addition to Wu-Palmer scores, we performed the same experi- shaded rows. After defining major classes into which adverbial ment on the verb pairs using all the eight semantic similarity mea- modifiers can be classified, lists containing adverbs related to each sures available in Wordnet[21]. It was observed that Jiang-Conrath class were created. Table 5 further contains examples of adverbs [13] and Lin [15] are the two measures which provides reasonable related to each type. This table does not include all the adverbs we accuracy in addition to Wu-Palmer semantic similarity[31]. The are maintaining in the lists. results from these experiments are shown in Table 4 and in Fig.1. It If adverbial modifiers connected to both verbs in a verb pair could be observed that Lin outperforms other two measures when with similar meaning belong to same Adverbial modifier type, but F-Measures are considered. It can be seen that 0.75 is the Lin score with opposite polarities (one positive and one negative), it can be which has the highest F-Measure. But, it is due to considerably high identified that the two sentences provide different views in relation recall and undesirably low precision values. As our intention is to to the entities that are connected by those verbs. maintain a proper balance between precision and recall, Lin Score of 0.86 is selected as the threshold to detect verb pairs with similar 3.6 Discovering Inconsistencies between meaning. 0.86 is the Lin Score with the second highest F-Measure. Triples Following the methodology presented in the PubMed study [7], a legal term dictionary was constructed to be served as a Semantic Lexicon for the system. 200+ legal opinion texts were used to extract words for the process. Then a word list consisting 17,000+ unique words were developed by removing stop words. A TF-IDF algorithm [14] based method is used to calculate a value for each term in the dictionary. Ícasecount f t,d i=1 t ermcount T ermV alue = (1) D.F Raw count (ft,d ) for each term is taken, considering each legal opinion text as a seperate document. Term frequency value for a term is calculated by dividing the raw term count by the total Figure 1: Variation of F-Measures with regard to Different number of terms in the case. Term frequency value for each case is Similarity Measures added together and the result value is divided from the document frequency (D.F ), to calculate the value for a term in the dictionary. Then all the term values are normalized according to the equation 3.5 Detecting Shift-in-View Relationships by 2. Comparing Properties Related to Identified (TV − TVmin ) ∗ (1 − TVmin ) Verbs NormalizedTV = + TVmin (2) TVmax − TVmin 3.5.1 Negation on Verbs. Usage of negation relationship is a popu- Here TVmin and TVmax represent the minimum and maximum lar approach when it comes to detecting inconsistencies and contra- values of the term values respectively. This normalized value is dictions in text [7, 9, 17]. In this study, we checked for the negation used to be served as the semantic weight for the system. relationship within verbs in verb pairs identified using the method First, coreference resolving is done on the sentence pairs using proposed in the section 3.4. If one verb is detected as being negated the Stanford CoreNLP CorefAnnotator [5] and the pairs with Tran- while the other verb is not being negated, the sentence pair is sition Words and Phrases are filtered out. Then OLLIE [18], open considered as having Shift-in-View relationship. Stanford CoreNLP information extraction system, is used to extract triples, in (Subject; dependency parser was used to detect the negation by identify- Relationship; Object) format, from sentences. When comparing two ing occurrences of the "neg" tag as described in "Stanford typed sentences, for the Shift In View relationship, only triple pairs with dependencies manual" [6]. same subject or object are considered, as the Shift In View relation- 3.5.2 Using Adverbial Modifiers to Detect Shifts-In-View. Another ship talks about different perspectives on the same topic or entity. approach to detecting different viewpoints on the same subjects or The stop words removed relationship strings of a triple pair are entities can be formulated by considering adverbial modifiers. If the then compared with each other word by word. The comparison is adverbial modifiers related to two verbs with similar meanings give performed in three ways. opposite or contradictory meanings, that means the viewpoints on (1) Words which are exactly the same ASAIL 2019, June 21, 2019, Montreal, QC, Canada Ratnayaka and Rupasinghe, et al. Table 4: Results Comparison for Different Wu-Palmer, Jiang-Conrath, and Lin Score Thresholds Wu-Palmer Jiang-Conrath Lin Score Precision Recall F-Measure Precision Recall F-Measure Precision Recall F-Measure 0.75 45.65% 100.00% 62.68% 70.78% 51.54% 59.64% 57.29% 72.37% 63.95% 0.80 51.39% 77.19% 61.70% 70.31% 49.34% 57.99% 60.39% 67.54% 63.77% 0.85 54.59% 69.08% 60.99% 71.02% 48.90% 57.92% 64.76% 62.06% 63.38% 0.86 59.34% 59.21% 59.28% 71.02% 48.90% 57.92% 67.15% 60.96% 63.91% 0.90 64.49% 49.78% 56.19% 71.25% 48.90% 58.00% 70.40% 53.73% 60.95% 0.95 72.69% 41.45% 52.79% 71.43% 48.25% 57.59% 72.60% 46.49% 56.68% Table 5: Adverbial Modifiers connected with the opposition party, it might be the case where both sentences are conveying opinions which are beneficial for the Type Class Type Name Modifiers opposition party in relation to the topic which is being discussed. more frequent always, often, regularly The problem becomes even more complex when the sentence is Frequency made up of several sub-sentences because each sub-sentence may accidentally, never, not, less, less frequent have a "Subject" of its own. Therefore, when using the sentiment loosely, rarely, sometimes so, well, really, literally , simply, based approach to detect "Shift-in-View" relationship, we consider amplifiers only the sentence pairs in which each sentence has only one explicit Tone for sure, completely, absolutely kind of, sort of, mildly, subject. If the subjects in both sentences are the same in such a down toners sentence pair, it can be concluded that two sentences are elaborating to some extent, almost, all but positive manner elegantly,beautifully,confidently on the same topic in relation to the same subject. Then, it is checked Manner whether the two sentences are providing sentiments with opposite negative manner lazily, ugly,faint heartedly polarities. If one sentence provides negative sentiment and other provides positive sentiment while discussing the same topic in (2) Exactly same words with one word negated with “not" relation to the same subject, it can be concluded that the probability (3) Different words of two sentences giving different perspectives on the same topic is very significant. In our study we consider the negation of words with similar In this approach, the sentences which are composed with subor- meanings (Lin score above 0.86) instead of considering only the dinate clauses are first split using those clauses. When the sentence words which are exactly the same. Then, an oppositeness value is split using a subordinating conjunction, that subordinate clause is obtained for each sentence pair by comparing the triples fol- can be identified as another sentence entity. Throughout this sec- lowing the algorithmic approach proposed in the PubMed Study tion, we will refer the subordinate clause as inner sentence and the [7]. A threshold based on the oppositeness values is introduced main clause will be referred to as outer sentence. After the sentence empirically to select sentence pairs which have the Shift In View is annotated using Stanford CoreNLP Constituency Parser [4] , the relationship. splitting happens by identifying associated terms with SBAR tag. 3.7 Sentiment-based Approach The proposed approach is based on analyzing the sentiment of this inner sentence to identify if there is a shift in view relation Though valuable information can be obtained by analyzing the between a sentence pair. If we consider the Example 4 (which was sentiment of a sentence, the sentiment of a sentence alone hardly taken from Lee v. United States [2]), The phrases “Lee cannot convince gives any details on the topics which are being discussed within the court that a decision to reject the plea bargain”, and “he can a sentence and on the viewpoint in which the sentence is describ- establish prejudice under Hill” are the inner sentences. The outer ing the topic. It is known that the two sentences which are being sentences are “The government argues”, and “Lee, on the other hand, compared to detect shifts in view discuss on the same topic as we argues”. consider only the sentence pairs with "Elaboration" relationship. But, when the sentences in legal opinion texts are considered, even Example 4 if the sentiments of two sentences which elaborate on the same discussion topic is different, it can not be concluded that the two • Sentence 4.1: The Government argues that Lee cannot "convince the court that a decision to reject the plea bargain. sentences are providing different opinions on the topic. • Sentence 4.2: Lee, on the other hand, argues that he can establish prejudice under The reason is that the person entities which are described in a Hill. sentence and connected with the sentiment of the sentence have a significant impact on the topic which is being discussed. For exam- ple, consider two sentences which elaborate on the same discussion If we consider the sentence pair mentioned in Example 4, both the topic and having opposite sentiments. If the sentiment of the sen- inner sentences’ subject is Lee. The phrase “Lee cannot convince the tence with negative sentiment is connected with the proposition court that a decision to reject the plea bargain” is having a negative party while the sentiment of sentence with positive sentiment is sentiment while the other inner sentence “Lee can establish prejudice Shift-of-Perspective Identification within Legal Cases ASAIL 2019, June 21, 2019, Montreal, QC, Canada under Hill” denotes a positive sentiment. Both the outer sentences Table 6: Results Comparison of Approaches used to detect are having neutral sentiment. Therefore, it can be observed that Shift-in-View there is a shift in view regarding the subject Lee. Approach No. of Sentence pairs Precision 4 EXPERIMENTS AND RESULTS Verb Relationships 46 0.609 Sentiment Polarity 230 0.382 As the first step, the 3 major approaches used to detect Shift-in- Inconsistencies between triples 95 0.273 View relationship type were evaluated. In order to perform this evaluation, 2150 sentence pairs from legal opinion texts related to criminal court cases were extracted from Findlaw [1]. Each of these sentence pairs contains two sentences which are consecutive around 0.6. As mentioned earlier we have selected the Lin semantic to each other within a legal opinion text document. Next, the ex- similarity score of 0.86 as the threshold to identify verbs with similar tracted sentence pairs were input into the Sentence Relationship meaning after analyzing different semantic similarity measures. The Identifier (SRI). Input sentence pairs are first processed inside SRI. precision of identifying verbs with 0.86 Lin score is 0.67. Thus, it The sentence pairs which are identified as having Elaboration by can be seen that there is a potential to improve the precision of the SRI were further processed in order to detect whether there is detecting Shift-in-View relationships using relationships between Shift-in-View relationship using the three Shift-in-View detection verbs by developing a semantic similarity measure which is more approaches mentioned under Section III. accurate in identifying verbs with similar meanings for the legal As the next step, the sentence pairs detected as having the Shift- domain. in-View relationship under each approach were taken into consider- Using the sentiment based model, the achieved precision is 0.38. ation. The number of detected sentence pairs from each approach There are few possible reasons behind this observation. The study is shown in Table 6. Then, the precision of each approach was on the sentiment annotator model [8] used in this case, states that calculated. All 46 sentence pairs detected from Verb-Relationship the accuracy of the model is 76%. The study says that the errors approach were used when calculating the precision of that approach. present in its parent model [27] can be propagated to the target 100 sentence pairs randomly selected from the detected 246 sen- model [8]. The paper on the source model[27] which is based on tence pairs, which were identified using the Sentiment-Polarity recursive neural tensor network, shows that the accuracy is reduced approach was used to determine the precision of the approach. 95 down to 0.5 when the n-gram length of a phrase increases (n>10). sentence pairs were detected from the approach which uses incon- As most of the sentences in court case transcripts are reasonably sistencies between triples to determine Shift-in-View. All of those 95 lengthier, there is a potential that the proposed sentiment based sentence pairs were used to calculate the precision of that approach. approach used for the identification of Shift-in-View is affected by The precision values obtained for each of these approach are also the above mentioned error. shown in Table 6. When performing this evaluation, each sentence Only a precision of 0.27 could be observed in the approach which pair was first annotated by two human judges. If the two judges considers inconsistencies using triples as proposed in PubMed did not agree on a relationship type for a particular sentence pair, Study. The following reasons may have contributed to the poor that sentence pair was annotated by an additional human judge. performances of that approach. From the 2150 sentence pairs which When the results were calculated, the consideration was given only were considered, oppositeness values were not calculated for 1570 to the sentence pairs which were agreed by at least two human pairs. Containing at least one sentence within a sentence pair in judges to have the same relationship type. which the triples could not be extracted by OLLIE[18] is a major Due to the scarcity of resources, it was not possible to anno- reason for not having an oppositeness value. Even if the triples tate all 2150 sentence pairs based on the relationship type. As a are extracted from both sentences, if there is no matching between result, calculating recall of each approach was not possible. If Ta- either subjects or objects of the two sentences, an oppositeness ble 2 related to SRI study [24] is considered, it can be observed value will not be calculated for a sentence pair. that only 3 out of 165 sentence pairs are determined as having the Evaluation results demonstrates that analysis of relationships be- Shift-in-View relationship type by the human judges. It suggests tween verbs in two sentences as the only approach which performs that the Shift-in-View relationship type does not occur frequently the task of detecting Shift-in-View relationships with a precision when we consider sequential sentence pairs in a legal opinion text. more than 0.5. Many studies convince the difficulty of detecting Furthermore, Table 2 suggests that the SRI tends to misattribute contradiction and change of perspective relationships over other re- sentence pairs having Shift-in-View as having the Elaboration rela- lationship types that can be observed between sentences[17, 24, 32]. tionship type. That means, the SRI is successful in determining if The study [17] also claims the difficulty of generalizing contradic- the two sentences in the sentence pair is discussing the same topic tion detection approaches. When considering these facts, it can be or not. In such circumstances, it is important to be precise when considered that the results obtained via analyzing verb relation- determining a sentence pair as having the Shift-in-View relationship ships are satisfactory. Therefore, we combined only that approach type. Considering these facts, we can conclude that it is important with the Sentence Relationship Identifier (SRI) and evaluated the to prioritize the Shift-in-View detection approaches based on the overall system made up by combining Shift-in-View detection with precision. SRI as shown in Table 7. According to the Table 6, it can be seen that the precision which The results shown in Table 7 were obtained using 200 annotated could be obtained from analyzing relationships between verbs is sentence pairs. Each of the considered sentence pair was agreed by ASAIL 2019, June 21, 2019, Montreal, QC, Canada Ratnayaka and Rupasinghe, et al. at least two human judges to have the same relationship type. Fur- [6] Marie-Catherine De Marneffe and Christopher D Manning. 2008. Stanford typed thermore, 21 randomly selected sentence pairs which were agreed dependencies manual. Technical Report. Technical report, Stanford University. [7] Nisansa de Silva, Dejing Dou, and Jingshan Huang. 2017. Discovering Inconsis- by at least two human judges as having Shift-in-View are contained tencies in PubMed Abstracts Through Ontology-Based Information Extraction. within the 200 sentence pairs which were used in this evaluation. In Proceedings of the 8th ACM International Conference on Bioinformatics, Compu- tational Biology, and Health Informatics. ACM, 362–371. [8] Viraj Gamage, Menuka Warushavithana, Nisansa de Silva, Amal Shehan Perera, Table 7: Results Obtained from Sentence Pairs in which At Gathika Ratnayaka, and Thejan Rupasinghe. 2018. Fast Approach to Build an least Two Judges Agree Automatic Sentiment Annotator for Legal Domain using Transfer Learning. arXiv preprint arXiv:1810.01912 (2018). [9] Jesse A Harris and Christopher Potts. 2009. Perspective-shifting with appositives Discourse Class Precision Recall F-Measure and expressives. Linguistics and Philosophy 32, 6 (2009), 523–552. [10] Vindula Jayawardana, Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Elaboration 0.938 0.930 0.933 Keet Sugathadasa, and Buddhi Ayesha. 2017. Deriving a representative vector for No Relation 0.843 0.895 0.868 ontology classes with instance word vector embeddings. In Innovative Computing Technology (INTECH), 2017 Seventh International Conference on. IEEE, 79–84. Citation 1.000 0.971 0.985 [11] Vindula Jayawardana, Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Shift-in-View 0.688 0.423 0.524 Keet Sugathadasa, Buddhi Ayesha, and Madhavi Perera. 2017. Semi-supervised instance population of an ontology using word vector embedding. In Advances in ICT for Emerging Regions (ICTer), 2017 Seventeenth International Conference on. According to the Table 7, it can be seen that there is a significance IEEE, 1–7. improvement, especially in relation to the Shift-in-View relationship [12] Vindula Jayawardana, Dimuthu Lakmal, Nisansa de Silva, Amal Shehan Perera, Keet Sugathadasa, Buddhi Ayesha, and Madhavi Perera. 2017. Word Vector type when compared with the results in the study[24] as given in Embeddings and Domain Specific Semantic based Semi-Supervised Ontology Table 3. Instance Population. International Journal on Advances in ICT for Emerging Regions 10, 1 (2017), 1. [13] Jay J Jiang and David W Conrath. 1997. Semantic similarity based on corpus 5 CONCLUSION statistics and lexical taxonomy. arXiv preprint cmp-lg/9709008 (1997). Developing a methodology to detect situations where multiple [14] Jure Leskovec, Anand Rajaraman, and Jeffrey David Ullman. 2014. Mining of massive datasets. Cambridge university press. viewpoints are provided in regard to the same discussion topic [15] Dekang Lin et al. 1998. An information-theoretic definition of similarity.. In Icml, within a legal opinion text is the major research contribution of Vol. 98. Citeseer, 296–304. [16] Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, this study. This study has introduced novel approaches to detect and David McClosky. 2014. The Stanford CoreNLP natural language processing deviations in the opinions provided by two sentences regarding the toolkit. In Proceedings of 52nd annual meeting of the association for computational same topic. At the same time, existing methodologies to detect con- linguistics: system demonstrations. 55–60. [17] Marie-Catherine Marneffe, Anna N Rafferty, and Christopher D Manning. 2008. tradiction and change of perspectives have been evaluated within Finding contradictions in text. Proceedings of ACL-08: HLT (2008), 1039–1047. the study. Additionally, it has been empirically demonstrated the [18] Mausam, Michael Schmitz, Robert Bart, Stephen Soderland, and Oren Etzioni. way in which the outcomes of the study can be used to facilitate 2012. Open Language Learning for Information Extraction. In Proceedings of Con- ference on Empirical Methods in Natural Language Processing and Computational the process of identifying relationships between sentences in docu- Natural Language Learning (EMNLP-CONLL). ments containing legal opinions on court cases. Evaluation of the [19] John J Nay. 2016. Gov2vec: Learning distributed representations of institutions and their legal text. arXiv preprint arXiv:1609.06616 (2016). performance of existing semantic similarity measures in relation [20] Michael J Paul, ChengXiang Zhai, and Roxana Girju. 2010. Summarizing con- to identifying verbs with similar meaning can be considered as trastive viewpoints in opinionated text. In Proceedings of the 2010 Conference on another key research contribution of the study. Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 66–76. The proposed approach can also be used to facilitate several [21] Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet:: other information extraction tasks related to the legal domain such Similarity: measuring the relatedness of concepts. In Demonstration papers at as identifying counter arguments to a particular argument, deter- HLT-NAACL 2004. Association for Computational Linguistics, 38–41. [22] Dragomir Radev, Jahna Otterbacher, and Zhu Zhang. 2003. CSTBank: Cross- mining representatives related to the proposition party and the document Structure Theory Bank. http://tangra.si.umich.edu/clair/CSTBank. opposition party in a court case. (2003). [23] Dragomir R Radev. 2000. A common theory of information fusion from multiple The accuracy of the approaches proposed in this study can be text sources step one: cross-document structure. In Proceedings of the 1st SIGdial further improved by developing semantic similarity measures and workshop on Discourse and dialogue-Volume 10. Association for Computational sentiment annotators which can perform in the legal domain with Linguistics, 74–83. [24] Gathika Ratnayaka, Thejan Rupasinghe, Nisansa de Silva, Menuka an improved accuracy. Coming up with such mechanisms can be Warushavithana, Viraj Gamage, and Amal Shehan Perera. 2018. Identi- considered as the major future work. fying Relationships Among Sentences in Court Case Transcripts Using Discourse Relations. In 2018 18th International Conference on Advances in ICT for Emerging Regions (ICTer). IEEE, 13–20. REFERENCES [25] Erich Schweighofer and Werner Winiwarter. 1993. Legal expert system KON- [1] [n. d.]. Caselaw: Cases and Codes - FindLaw Caselaw. https://caselaw.findlaw. TERM - automatic representation of document structure and contents. In In- com/. ([n. d.]). (Accessed on 05/20/2018). ternational Conference on Database and Expert Systems Applications. Springer, [2] 1977. Lee v. United States. In US, Vol. 432. Supreme Court, 23. 486–497. [3] Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. [26] Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Rea- 2015. Leveraging linguistic structure for open domain information extraction. In soning with neural tensor networks for knowledge base completion. In Advances Proceedings of the 53rd Annual Meeting of the Association for Computational Lin- in neural information processing systems. 926–934. guistics and the 7th International Joint Conference on Natural Language Processing [27] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, (Volume 1: Long Papers), Vol. 1. 344–354. Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic [4] Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency compositionality over a sentiment treebank. In Proceedings of the 2013 conference parser using neural networks. In Proceedings of the 2014 conference on empirical on empirical methods in natural language processing. 1631–1642. methods in natural language processing (EMNLP). 740–750. [28] Keet Sugathadasa, Buddhi Ayesha, Nisansa de Silva, Amal Shehan Perera, Vindula [5] Kevin Clark and Christopher D. Manning. 2015. Entity-Centric Coreference Jayawardana, Dimuthu Lakmal, and Madhavi Perera. 2017. Synergistic union of Resolution with Model Stacking. In Association for Computational Linguistics word2vec and lexicon for domain specific semantic similarity. In Industrial and (ACL). Information Systems (ICIIS), 2017 IEEE International Conference on. IEEE, 1–6. Shift-of-Perspective Identification within Legal Cases ASAIL 2019, June 21, 2019, Montreal, QC, Canada [29] Keet Sugathadasa, Buddhi Ayesha, Nisansa de Silva, Amal Shehan Perera, Vindula Jayawardana, Dimuthu Lakmal, and Madhavi Perera. 2018. Legal Document Retrieval using Document Vector Embeddings and Deep Learning. arXiv preprint arXiv:1805.10685 (2018). [30] Kristina Toutanova, Dan Klein, Christopher D Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Pro- ceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1. Association for Computational Linguistics, 173–180. [31] Zhibiao Wu and Martha Palmer. 1994. Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Association for Computational Linguis- tics. Association for Computational Linguistics, 133–138. [32] Nik Adilah Hanin Zahri, Fumiyo Fukumoto, and Suguru Matsuyoshi. 2012. Ex- ploiting Discourse Relations between Sentences for Text Clustering. In 24th International Conference on Computational Linguistics. 17.