Evidentiality and Disagreement in Earnings Conference Calls: Preliminary Empirical Findings Andrea Rocci1 , Carlo Raimondo1 , and Daniele Puccinelli2 1 Institute of Argumentation, Linguistics and Semiotics Università della Svizzera italiana (USI) Via Buffi 13, 6900 Lugano, Switzerland {andrea.rocci,carlo.raimondo}@usi.ch 2 Institute for Information Systems and Networking (ISIN) Department of Innovative Technologies (DTI) University of Applied Sciences and Arts of Southern Switzerland (SUPSI) Via Cantonale 2c, 6928 Manno, Switzerland daniele.puccinelli@supsi.ch Abstract. In financial communication, earnings conference calls repre- sent a remarkable resource to understand the impact of argumentation on the decision-making process of the investing community. In this pa- per, we present some preliminary findings from a corpus-based study of earnings conference calls held by listed companies; specifically, we look at the distribution of evidentials in question and answer turns and their correlation with disagreement expressions. Our empirical results suggest that evidentials are argumentative indicators and characterize the argu- mentative roles of executives and analysts. Keywords: Evidentiality · Disagreement · Financial Communication. 1 Introduction Financial communication is an ideal setting for the medium-to-large-scale anal- ysis and evaluation of arguments and the observation of their impact on the decision-making process of investors. Our focus is on earnings conference calls (ECCs); held shortly after the release of a listed company’s quarterly results, an ECC generally begins with a corporate presentation by a company’s chief executives followed by a round of questions posed to the executives by the fi- nancial analysts (the Q&A session). As shown in [1], the most e↵ective part of an ECC is the argumentation in the Q&A session. Earlier argumentation mod- eling e↵orts targeted at ECCs have focused on formalizing an argumentatively relevant dialogue protocol validated on a small manually annotated corpus of ECCs [2]. Here we proceed with a multi-pronged research strategy, which com- bines a coarse rule-based automatic annotation dialogue moves in a large corpus of ECCs with the dictionary-based study of potential indicators of argumenta- tion. These shallow methods are directly comparable with the kind of sentiment analysis used in finance research [3]. Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 Data and Methods Our study of evidentials as potential indicators for the characterization of argu- mentative moves in ECCs is based on two corpora: a small, manually annotated one, containing 46 conference call transcripts (508,787 words) and a relatively large one, containing 1,134 call transcripts (with 3,797,907 words in the corpo- rate presentations 1,605,855 words in the questions, and 4,229,270 words in the replies). The large corpus was segmented based on the deterministic structure of the call transcripts. Since the presentation and Q&A sessions are always labelled and the participants are always listed along with their roles, the call dynamics are predictable, with analysts asking questions and corporate players providing answers. Fully unsupervised coarse-grained dialog act labelling was performed with a Finite State Machine (all operator segments were ignored). Taking inspiration from the results of corpus studies of evidential lexemes as argumentative indicators [5], we built a 208 item dictionary of words and n-grams meant to capture evidentiality as the encoding of the source of information and epistemic status of the propositional content of an utterance. The dictionary was built starting from corpus studies of evidentiality in English (especially [4]) and progressively manually refined through the study of corpus concordances. The dictionary items are categorized according to the types of meanings of evidentials represented in Figure 1. Fig. 1. Types of meanings of evidentials . 101 A disagreement dictionary (158 n-grams) was also created. It includes adver- sative and concessive connectives, lexical expressions of disagreement, negations and hedges. The dictionary, which partially draws on previous argument mining approaches to disagreement [6], and contains a wide array of adversative and concessive connectives, negations, expressions that explicitly indicate disagree- ment, and hedges or mitigating devices to introduce disagreement (e.g. to be honest). 3 Preliminary Empirical Findings 3.1 Distribution of Evidentials in the calls Table 1 shows a breakdown of the various evidentiality types, providing the num- ber of occurrences of each type in the corporate presentations (P), the questions posed by the analysts (Q), and the answers provided by the executives (A), along with the frequency of each evidentiality type per thousand words in the presentations (fP ), the questions (fQ ), and the answers (fA ). Evidentiality type P Q A Total fP fQ fA Common knowledge 550 93 1819 2462 0.14 0.06 0.43 Direct 3946 2872 10237 17055 1.04 1.79 2.42 Epistemic Possibility 1400 8541 5062 15003 0.37 5.32 1.20 Generic Indirect 153 48 76 277 0.04 0.03 0.02 Inference 23847 21102 29144 74093 6.28 13.15 6.89 Report 330 4947 2304 7581 0.09 3.08 0.54 Subjective 4188 5098 30402 39688 1.10 3.18 7.19 Grand Total 34414 42701 79044 156159 9.06 26.61 18.69 Table 1. The table reports the distribution of the evidentials in the corpus, distigu- ishing between the di↵erent types. For each evidential category we report its absolute and relative frequency, also taking in consideration the di↵erent parts of the calls. Table 2 takes a closer look at the distribution of the inferential evidentials, which account for nearly half of all evidentials in our corpus, as shown in Table 1. The distribution of the inferential evidentials is asymmetric between questions and answers, with most indicators showing a strong preference for questions. Only the highest certainty items are equally distributed among turns, or even show a preference for answers. 3.2 Correlation of Evidentiality and Disagreement in the Q&A Section Figure 2 shows the number of tokens expressing disagreement versus the number of evidentials for each ECC in our corpus. The key finding is that evidentials 102 Form Subtype P Q A Total fP fQ fA sign From data 5963 869 4623 11455 1.57 0.54 1.09 prove From data 5944 859 3072 9875 1.57 0.54 0.73 guess Conjecture 164 6630 1982 8776 0.04 4.13 0.47 should Conjecture 1492 2775 2269 6536 0.39 1.73 0.54 obviously Conjecture 240 1653 4139 6032 0.06 1.03 0.98 probably N.A. 168 537 3476 4181 0.04 0.33 0.82 show From data 1621 344 1421 3386 0.43 0.21 0.34 proved From data 1999 150 581 2730 0.53 0.09 0.14 seem From data 101 1390 525 2016 0.03 0.87 0.12 clearly From data 325 275 1182 1782 0.09 0.17 0.28 assume Conjecture 629 582 445 1656 0.17 0.36 0.11 proving From data 891 146 443 1480 0.23 0.09 0.10 looks From data 112 821 506 1439 0.03 0.51 0.12 seems From data 54 987 332 1373 0.01 0.62 0.08 could be From data 159 367 738 1264 0.04 0.23 0.17 Table 2. The table reports the 15 most frequent inferential evidentials as found in our corpus. For each evidential the subtype is report together with its absolute and relative frequency, distinguishing also between the di↵erent parts of the calls. Fig. 2. The figure shows the correlation between evidentials and disagreement . 103 and disagreement are highly correlated (⇢ = 0.69). The correlation between disagreement and evidencial is also substantial if we consider the questions (⇢ = 0.73) and the answers (⇢ = 0.68) separately. 4 Conclusion and Future Work Our ECC corpus shows a marked di↵erence in the distribution of evidentials in Q&A turns, matching the di↵erent epistemic position and argumentative role of executives and financial analysts. We also observe a substantial correlation between evidentiality and disagreement. Contrary to our initial expectation, the correlation is not exclusively driven by the questions posed by analysts, but also by the answers provided by the executives. The results suggest that disagreement and evidentiality dictionaries can function as sub-components of a dictionary- based indicator of argumentativity. Our preliminary findings encourage us to pursue further investigations based on the combination of automatic segmentation of discourse units and dictionary- based methods. References 1. Palmieri, R., Rocci, A., and Kudrautsava, N.: Argumentation in earnings conference calls. Corporate standpoints and analysts’ challenges. Studies in Communication Sciences, 15, (2015). 2. Budzynska, K., Rocci, A., and Yaskorska, O.: Financial Dialogue Games : A Protocol for Earnings Conference Calls. In S. Parsons et al. (Eds.), Computational Models of Argument (pp. 19–30). IOS Press, (2014). 3. Loughran, T. and Mcdonald, B.: Textual Analysis in Accounting and Finance : A Survey. (2016). 4. Bednarek, M.: Epistemological positioning and evidentiality in English news dis- course: A text-driven approach. Text and Talk, 26(6), pp. 635-660, (2006) 5. Musi, E., and Rocci, A.: Obviously Epistentials are Argumentative Indicators : Evidence from an Italian and English Corpus of Newspaper Articles. In Proceedings of the Workshop on the Foundations of the Language of Argument at COMMA (2016) 6. Budzynska, M. Janier, J. Kang, B. Konat, C. Reed, P. Saint-Dizier, and O. Yasko- rska: Automatically identifying transitions between locutions in dialogue, in Pro- ceedings of 1st European Conference on Argumentation: Argumentation and Rea- soned Action (2015) 104