Annotating Argumentation within Questions. Prefaced Questions as a Genre Specific Argumentative Pattern in Earnings Conference Calls.

Abstract

Argumentative patterns - significant constellations of argumentative moves explainable in view of the activity type - can serve as a bridging notion between studies of argumentation in context, argumentation mining and analytics. We report on the discovery of potential pattern in a corpus of earnings conference calls. The internal argumentation structure of Prefaced questions, a “move” already recognized as recurrent in the genre structure of the earnings conference calls activity, is reconstructed via Inference Anchoring Theory to verify their status as argumentative patterns and preliminarily assess their significance for earnings conference calls.

Keywords

• 1 Argumentative patterns
• argumentation in context
• financial communication
• earnings conference calls
• prefaced questions

1. Introduction

We report work in progress within a project2 devoted to a large corpus study of argumentative patterns (APs) [ 1 ], [ 2 ] – i.e. significant constellations of argumentative moves whose occurrence can be explained in view of the goals and rules of the activity type – in the Q&A section of earnings conference calls (ECCs) of listed companies. The final aim of the project is to make it possible investigate the effects of argumentation in ECCs on the financial markets. Dialogue between managers and financial analysts in ECCs has been shown to impact the markets [ 3 ], but the features investigated have been mostly limited to sentiment and more recently semantic-pragmatic features [ 4 ], which do not include argumentation. Empirical investigation of the impact of ECCs argumentation requires Argumentation Mining to overcome the manual annotation bottleneck and Argumentation Analytics that can be correlated with market data.

Being specific to the activity type, APs offer interesting affordances both as the main target of the mining and as the basic units whose distribution is correlated with extra-discursive market data. We see APs as molecular units, with internal inferential and dialogical structure, which become meaningful as a whole as they fit specific goals, incentives and constraints of the activity, and recognizable as members’ categories [ 5 ] by the participants of the activity type. By focusing on APs we hope to leverage on activity specific genre structure [ 6 ], specific lexis and phraseology to drive the mining, while feeding the analytics with argumentation molecules that have recognized contextual significance. In an AP based perspective, context specificity is not seen as a mere lack of generalizability, characteristic of early approaches to argumentation mining [ 7 ], [ 8 ], but as a theoretically motivated feature designed to empirically investigate the extra-discursive social impact (e.g. the market impact in our case) of argument molecules whose variation is hypothesized significant on the basis of qualitative studies.

Similarly, in [ 9 ] and [ 10 ], earlier qualitative studies of adversarial features of journalistic questioning were used in large scale studies to empirically test hypotheses on the historical evolution of journalistic questioning in press-conference and on its relationship with the economy. While these discourse-based studies in empirical social science could benefit enormously from argumentation mining, they do not appear to be among the most prominent use cases featured in the argumentation mining literature (see [ 11 ], Chapter 10). Our project aims to contribute to fill this gap by investigating how patterns of dialogical argumentation in ECCs relate to market data.

The first step of this process is then discovering interesting patterns. The discovery procedure (a) starts from the annotation of dialogue moves in the genre structure of the ECC, (b) selects those that appear to have an argumentative function fitting the activity type, (c) studies their distribution and (d) reconstructs their internal argumentation structure through fine-grained annotation to capture their argumentative potential.

In this work in progress, we examine a candidate AP, the prefaced questions, whose relevance in the activity type has been already noted in previous studies [ 12 ]. Having observed the distribution of prefaced questions in a corpus of ECCs, in order to assess them as candidate APs, we investigate their internal argumentation structure, verifying, in particular, the hypothesis that the preface (P) provides arguments supporting the relevance of the question (Q) speech act. We validate this hypothesis through a fine-grained annotation of the argumentation based on Inference Anchoring Theory (IAT) [ 13 ], [ 14 ]. In reconstructing the argumentative structures, we wanted to explicit the inferential link that can occur within a question turn, and especially the possible relations between prefaces and questions. Doing this, the reconstructions we propose stretch the expressive limits of IAT standard formalization of argument, by allowing these particular inferential connections to select directly illocutions as their conclusion.

2. Prefaced questions as an Argumentative pattern in ECCs

The candidate AP prefaced question, exemplified in (1) below, arises in the question turns performed by financial analysts in the Q&A phase of the ECC. (1) Hasbro Q1 2021. Eric Handler: [And then secondly, you know,]DR [it looks like the hot new consumer product out there is NFTs. And given that you have a lot of collectible business,]P [have you thought much about what might make sense in the NFT business?]Q

Structurally and functionally akin to press conferences, ECCs include the top management’s presentation of the quarterly results of the firm, followed by a Q&A session where analysts ask questions to the corporate leaders. Analysts are also akin to journalists in that the Q&A will contribute to inform their valuation of the firm, which they share in their reports and recommendations addressed to a public of investors. Like journalists, they are, at least ideally, expected to take the posture of dialectical antagonists holding managers accountable of their stewardship of the company on behalf of stockholders and other stakeholders [ 15 ]. Finance researchers, e.g.[ 16 ], consider the Q&A session is the most informative moment of the ECC, even if little or no new “material information” is provided, so that earlier research on ECCs [ 12 ] discourse suggests that the new “information” is actually the presence of arguments that help analysts and investors connecting the dots.

A preface, tagged as P in example (1), is an assertive statement that can either precede, follow or be contained in a question sentence within a journalist’s or analyst’s question turn, conveying information related to the question. In relation to journalistic interviews and press conferences, Clayman and Heritage [ 9 ] observe that “prefatory statements” have a twofold function as they provide “contextual background information that renders the question intelligible to the audience and provides for its appropriateness”. As Heritage rephrases in, they “provide a motivational context”, i.e. they establish a context that “gives meaning and point to the subsequent question”.

Moving to ECCs proper, Palmieri et al. [ 12 ] define different types of information prefaces can contain on the basis of the source (statements made by managers in the presentation or in the past, widespread opinions, fact noticed or premises inferred by the analyst himself) and highlights that prefaces present “some sort of argumentation that legitimizes the question”. As such they could be viewed as contextual specification of a general pattern described by Hitchcock [ 17 ], [ 18 ] where a question can be the conclusion of premises that attempt to show that the question is correct, i.e. that it is correct to ask it and that the question needs to be answered.

3. Annotation and argumentation reconstruction

We look at ECCs held by Hasbro for the announcement of the results of the four quarters of fiscal year 2021. The corpus includes the transcripts of the four Q&A (22,832 words), which were revised, preprocessed and normalized by means of an ad hoc algorithm, providing participants’ extraction and text segmentation. At first, all the questions of Q&A sections were manually annotated by our team of trained annotators on INCEpTION platform [ 19 ] for genre-based discourse moves, according to the annotation schemes represented in Figure 1 (layer 1) and Figure 2 (layer 2), developed on the basis of [ 12 ]. Annotators were asked to identify in analysts’ turns proper question chunks, and if present discourse regulators, prefaces and varia. Each question was then categorized accordingly to the scheme. Annotation standard was set by a two-layer annotation scheme, the detailed description of which is available to the team in the form of an annotation manual [forthcoming]. Layer 1, Dialogue Moves, captures the basic set of moves available to ECCs participants. For analysts’ question turns, it captures the presence of preface, the question type, the use of speech or thought predicates in mediated formulations (e.g. have you thought much about, in 1) and the presence of presumption. In Layer 2, we annotated types and subtypes of requests – e.g. requests of elaboration, clarification, explanation, opinion, etc. – according to a taxonomy grounded both in the a priori understanding of the activity type and in the abundant recurrent lexis and phraseology used by participants to signal these specific question acts3.

Inter-annotator agreement Kappa was tested both during the training period and occasionally over the course of annotation work and kept being no less than substantial over all phases. Table 1 shows the Kappa values for annotation features relevant for this study, Move type and Prefaced, both of which show an almost perfect agreement. The former concerns the choice of the move for each chunk; it allows to choose among Preface, Discourse regulator, Varia, Reply and Question, thus the correct identification of the Preface falls into this value. The latter, Prefaced, is the feature concerning the presence of a preface for a specific question. 3 This taxonomy has been refined in view of the IAT annotation in [ 20 ].

Table , below, shows the prevalence of prefaced questions over non-prefaced (61% vs 39%), while Table and Table display the distribution of prefaces in question turns in relation to request types, showing the predominance of prefaced questions in each type.

We can observe that, leaving aside requests of justification (n=1), the most prefaced request type is the request of confirmation, while requests of data and elaboration, although the most numerous, have a low rate of prefacing. Previous studies –[ 12 ] and [ 16 ] – have characterized the former as highly argumentative moves, and the latter as moves oriented towards incremental information acquisition [ 21 ]. This would be broadly consistent with an argumentative functioning of prefaced questions.

Argumentative annotation in OVA [ 22 ], [ 23 ] based on IAT theory allowed us to investigate the argumentation within the question turns, focusing on the relation between prefaces and questions and in particular on the role that prefaces have4. The annotation revealed that a preface contains an argument for the performance of a specific questioning illocutionary act. It does so by doing one or more of the following: framing an issue that needs to be solved; manifesting an inferred proposition p; providing an argument to infer a proposition p, where p may be part of the propositional content of the question, a presupposition of the question or an implicit standpoint concerning the request.

Figure 3: OVA reconstruction. shows the argumentative reconstruction of (1). Analyst Eric Handler, asking for an opinion about a certain topic, gives in the preface data that can represent an argument for the relevance of the topic and thus of the opinion. The preface presents two linked arguments for (a) the addressee actually having an opinion on the topic (presupposition of existence) and (b) the relevance of this opinion (presupposition of relevance); these two implicit conclusions are themselves arguments for the relevance of the question and therefore for the questioning illocutionary act. This is represented in the annotation by the highlighted default inference node pointing to the illocution R-of-opinion.

In the example in Figure , the question arises rather from a conflict exposed in the preface. The analyst reports two contradictory statements by Hasbro’s managers, asking for more information and details. In this case, the preface has explicitly the role of reporting statements that generate an issue, and, by framing this issue that needs to be solved, it provides an implicit argument for the question. Both examples indeed show that there is an implicit argumentation for the question’s relevance. Prefaces have the role to provide contextual information or display analysts’ knowledge around the topic of the question that serve as arguments for the question itself supporting either the presuppositions or the relevance of it. 4 The annotated corpus is available in the AIFdb corpus: http://corpora.aifdb.org/preface

4. Conclusions and future work

In this work, we outlined the annotation process exploited to study a candidate AP in a small corpus starting from the coarse annotation of dialogue moves in INCEpTION. The observation of the distribution confirmed the relevance of the prefaced question AP, and the subsequent argumentative annotation corroborate the hypothesis of the argumentative function of the preface. The proposed reconstructions are meant to show the inferential relations linking prefaces and questions; in doing so, the annotation pushed the limits of the IAT formalism, allowing illocution nodes as landing sites for inference nodes, which are not envisaged by the theory. The question remains open on how best to formalize illocution support and the implicit intermediate standpoints satisfying the presuppositions of the question act. This raises important issues about the desirable and realistic targets of argumentation mining in our pattern-based approach and in general.

References

[1] F. H. van Eemeren , “ Identifying Argumentative Patterns: A Vital Step in the Development of PragmaDialectics” , Argumentation, vol. 30 , no. 1 , pp. 1 - 23 , 2016 , doi: 10.1007/s10503-015-9377-z.

[2] E. Musi and M. Aakhus , “ Discovering Argumentative Patterns in Energy Polylogues: A Macroscope for Argument Mining” , Argumentation, pp. 1 - 34 , 2018 , doi: 10.1007/s10503-017-9441-y.

[3] J. V. Chen , V. Nagar , and J. Schoenfeld , “ Manager-analyst conversations in earnings conference calls,” Review of Accounting Studies , no. 23 , pp. 1315 - 54 , 2018 .

[4] K. A. Keith and A. Stent , “ Modeling Financial Analysts' Decision Making via the Pragmatics and Semantics of Earnings Calls,” 2019 . [Online]. Available: http://arxiv.org/abs/ 1906 .02868.

[5] E. Schegloff , “A Tutorial on Membership Categorization, ” Journal of Pragmatics , vol. 39 , no. 3 , pp. 462 - 82 , 2007 , [Online]. Available: https://doi.org/10.1016/j.pragma. 2006 . 07 .007

[6] M. Taboada , “ Stages in an Online Review Genre” , Text & Talk - An Interdisciplinary Journal of Language, Discourse & Communication Studies , vol. 31 , no. 2 , pp. 247 - 69 , 2011 , [Online]. Available: https://doi.org/10.1515/text. 2011 . 011 .

[7] S. Teufel , “Argumentative Zoning: Information Extraction from Scientific Text” . Ph.D. thesis , School of Cognitive Science, 1999 , University of Edinburgh, Edinburgh.

[8] R. Mochales and M.-F. Moens , “Argumentation mining”, Artificial Intelligence and Law , vol. 19 , no. 1 , pp. 1 - 22 , 2011 https://doi.org/10.1007/s10506-010-9104-x

[9] S. E. Clayman and J. Heritage , “ Questioning presidents: Journalistic deference and adversarialness in the press conferences of U.S. Presidents Eisenhower and Reagan” , Journal of Communication , vol. 52 , no. 4 , pp. 749 - 775 , 2002 , doi: 10.1093/joc/52.4.749.

[10] J. Heritage , “ Designing Questions and Setting Agendas in the News Interview”, in Studies in Language and Social Interaction: In Honor of Robert Hopper , P. Glenn , C. D. LeBaron , and J. Mandelbaum, Eds. Mahwah, NJ: Lawrence Erlbaum, 2003 , pp. 57 - 90 .

[11] M. Stede and J. Schneider. Argumentation Mining . Morgan and Claypool, 2018 ,https://www.morganclaypool.com/doi/10.2200/S00883ED1V01Y201811HLT040.

[12] R. Palmieri , A. Rocci , and N. Kudrautsava , “ Argumentation in earnings conference calls. Corporate standpoints and analysts' challenges”, Studies in communication sciences , vol. 15 , 2015 , no. 1 , pp. 120 - 132 , 2015 , doi: 10.1016/j.scoms. 2015 . 03 .014.

[13] K. Budzynska and C. Reed , “ Speech acts of argumentation: inference anchors and peripheral cues in dialogue,” in Computational models of natural argument: papers from the 2011 AAAI Workshop , 2011 , vol. WS-11-10 , pp. 3 - 10 .

[14] A. Hautli-Janisz , K. Budzynska , C. McKillop , B. Plüss , V. Gold , and C. Reed , “Questions in argumentative dialogue, ” Journal of Pragmatics , vol. 188 , pp. 56 - 79 , Jan. 2022 , doi: 10.1016/j.pragma. 2021 . 10 .029.

[15] A. Rocci and C. Raimondo , “Dialogical Argumentation in Financial Conference Calls : The Request of Confirmation of Inference ( ROCOI ),” in Argumentation and Inference: Proceedings of the 2nd European Conference on Argumentation , 2018 , pp. 699 - 715 .

[16] D. Matsumoto , M. Pronk , and E. Roelofsen, “What Makes Conference Calls Useful? The Information Content of Managers' Presentations and Analysts' Discussion Sessions” , The Accounting Review , vol. 86 , no. 4 , pp. 1383 - 1414 , Jul. 2011 , doi: 10.2308/accr-10034.

[17] D. Hitchcock , “ We justify questions, so how does that work?” , in Proceedings of the Ninth International Conference on Argumentation, Amsterdam , 2019 .

[18] D. Hitchcock , “ Arguing for Questions,” in From Argument Schemes to Argumentative Relations in the Wild: A Variety of Contributions to Argumentation Theory ., B. Garssen and F. van Eemeren, Eds. Springer, 2020 , pp. 167 - 184 .

[19] J.-C. Klie , M. Bugert , B. Boullosa , R. Eckart de Castilho, and I. Gurevych , “ The INCEpTION Platform: Machine-Assisted and Knowledge-Oriented Interactive Annotation,” in Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations , Santa Fe, New Mexico, Aug. 2018 , pp. 5 - 9 .

[20] O. Yaskorska-Shah , A. Rocci and C. Reed , “ Conversation shaping questions: a taxonomy used for mapping argumentative dialogues in financial discourse”, submitted to CMNA 2022 .

[21] G. D'Agostino , “ (so long, and) thanks for all the color . Requests of Elaboration and Answers they Trigger in Earnings Conference Calls”, this issue , pp. 80 - 85 .

[22] M. Janier , J. Lawrence , and C. Reed , “OVA+ : an Argument Analysis Interface,” Computational Models of Argument , pp. 463 - 464 , 2014 , doi: 10.3233/978-1- 61499 -436-7-463.

[23] J. Lawrence and C. Reed , “AIFdb Corpora,” in Computational Models of Argument: Proceedings of {COMMA} 2014 , Atholl Palace Hotel , Scottish Highlands, UK, September 9- 12 , 2014 , 2014 , pp. 465 - 466 . doi: 10 .3233/978-1- 61499 -436-7-465.