This paper presents a pilot study of attitude analysis in diplomatic discourse. The study was conducted by following the Appraisal Theory while annotating attitudinal expressions in the diplomatic speeches of the United Nations Security Council. It is supposed that the Appraisal scheme would unfold new valuable information about the distribution of attitudes in the diplomatic discourse due to the highly specific ranking of diferent types of opinion representations. The aim of the study is to analyze the application of the theory to real diplomatic data. To be fully comprehensible, these texts, though straightforward on the surface, require additional attention from the reader. The data consist of 20 speeches that were sampled from the UN Security Council Debates dataset and then manually annotated by applying the “attitude” part of the Appraisal theory scheme. The annotation process has shown problematic points regarding the influence of polarity of the actors on expressing attitudes, attitudes implicated in the proper names and phrases, identifying appraisal, as well as the annotator's subjectivity issue. We have concluded that the Appraisal theory annotation scheme is generally well applicable to diplomatic speeches. As for the unclear cases, we have proposed some solutions which could be taken into modification.
attitudes in diplomatic speeches [1]. These attitudes are expected to be found and annotated on the level of single The United Nations Security Council has proven to be expressions, therefore sentence-level as well as the textone of the organs of international diplomacy that not level are not taken into account. In this pilot study we only provides a platform for exchanging opinions and omit analysis of the broader context of the sentence and discussing the event, but also for taking decisions that text and concentrate on the level of single expressions help cease and prevent international conflicts. The diplo- as the Appraisal theory does not ofer the solution for matic discourse of this organization can give us valuable this task, therefore it would be developed at the later information on how people exchange opinions while be- stages of the research. When the scheme is found and ing in a situation where each word can cost not only the data is annotated, it should be possible to draw conmillions of dollars but can either save or kill people. This clusions about the structural, grammatical, lexical, and context of meetings shape the discourse of the sessions poetic features of attitudes in diplomatic discourse. The making the speeches highly formalized, structured, and final step is expected to be the creation of a dictionary for thoroughly prepared. diplomatic attitudinal expressions and training a model
The aim of this study is to look at how the Appraisal to automatically detect attitudes in diplomatic discourse. theory can be applied at diplomatic speeches, to get to know the scheme through annotation and to find out specificities of annotating diplomatic speeches follow- 2. The term of attitude ing this scheme. It is expected that in case the existing scheme would not cover the research needs in full, it Even though the meaning of the word ‘attitude’ could could be edited according to the needs of the study. One be perceived as clear and understandable, it is important of the expected outcomes of this project would be a cre- to remember that ‘attitude’ is a polysemic word, and the ation of a high-quality language data that could be used definition closest to what we refer to in this work is “a in diferent projects in computational linguistics. mental position or a feeling or emotion with regard to
The presented analysis should be considered as a part a fact or state” [2]. Expressing attitudes, in general, is of a larger study of defining facts and attitudes in diplo- perceived as a natural human social behavior crucial for matic speeches. One of the steps is to find an annotation establishing and maintaining cooperation. Attitude in scenario that is the most suitable for the task of defining this context could be viewed as a synonym to the term of opinion, which “is a subjective statement as it describes the thinking or the beliefs of a person about a particular ITAT’22: Information technologies – Applications and Theory, Septem- thing. Opinion can be defined as a judgment or a belief b$era2n3i–si2m7,o2v0a2@2,uZfualb.meref.ccu,nSlio.cvza(kMia. Anisimova); that lacks absolute conviction, certainty, or optimistic zikanova@ufal.mf.cuni.cz (Š. Zikánová) knowledge. It concludes that certain facts, ideas, etc. are © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License likely to be true or are true.”[3] In the given context, an CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) attitude could also be referred to as being “an estimation tion was incorporated. In cases where a speech was held of the quality or worth of someone or something” [4]. in a language diferent from English, the respective addi
In diplomacy, attitudes can also be viewed from a tional information was added at the beginning of each file. higher perspective of being markers of a diplomat’s po- We had a question of whether to focus only on speeches sition, and therefore, a country’s position towards the held in English or to analyze oficial translations as well. discussed events. This presented analysis should not be From our point of view, oficial translations could be conconsidered to be conducted on this higher level of under- sidered a suitable language material for the research due standing of diplomatic attitude. We are not describing to the excellent qualification of the UN translation team. the position of the speaker as a manifestation of their country’s international politics but are interested in lo- 3.2. Creating a corpus and selecting a cal linguistic manifestations that an attitude may take in order to develop linguistic criteria for distinguishing subset of speeches linguistic manifestations of facts and attitudes on a sen- After the source data was obtained, a smaller corpus (retence level. We, therefore, perceive attitudes as single ferred to as “the 100 corpus” later on) of speeches was expressions rather than a general attitude towards the created. This corpus consists of 100 speeches in total. event discussed. The speeches were selected according to certain crite
The formal view of identifying attitudes is especially ria. We have first selected five international military important in our task as it is connected with the natural conflicts discussed and debated at the Security Counlanguage processing of diplomatic texts. There are many cil within the time frame of the UN Security Council forms that an attitude can take in diplomatic speeches, Debates dataset. The topics selected are the following: among them are adjectives (a just war, the most dificult the Palestinian topic (comprising of meeting recordings decision), nouns (ally), verbal constructions (I am afraid regarding the Israeli-Palestinian conflict), the Yugoslathat), as well as clauses and phraseologisms (the rats are vian topic (the discussions and debates regarding the abandoning ship). There are certain patterns of construc- Yugoslav Wars), the Ukrainian topic (referring to meettions that repeat in variable forms in every single speech ings regarding the Russo-Ukrainian War), the Georgian and therefore are identifiable without dificulty (e.g. for- topic (debates regarding the 1992–1993 War in Abkhazia, mal appreciation as in ‘let me welcome you to your new and the 2008 Russo-Georgian War), and the various quesposition...’, or ‘It is a great pleasure to welcome...’). tions discussed regarding the Iraq War (comprising the
The analysis of attitudes in various fields of study de- Iraqi topic). manded a framework which would define the position of The corpus contains 20 speeches from each topic. The the speaker and their reaction towards the events from speeches were chosen according to certain criteria in multiple viewpoints. One of the solutions to this need order for the dataset to be balanced. The first criterion is the Appraisal Theory [5]. This approach to analyzing was the position of the speaker. This means, that for and classifying attitudes views them as being one of the each meeting a cluster of speeches (usually 5 speeches parts of the more global viewpoint of appraisal of the from the same meeting) at least one would be from the “language of evaluation”. The viewpoint of expressed country sufering from the military conflict, one - from attitudes being resources for evaluation of the ‘social in- the invader, the aggressor, or the opponent, two - from tersubjective perspective on evaluation’ [6] is adopted in the permanent members of the UNSC, and one from a the present paper. non-permanent member. In certain cases, we have also selected speeches from invited guests, usually experts 3. The dataset or politicians. We have also tried to include at least one “supporter” of the invaded country, and one “supporter” of the aggressor or a neutral party. These political criteria 3.1. The source of data served as a mean of selecting the speeches, however, The data for the current analysis was excerpted from The they were not reflected in the metadata of the corpus UN Security Council Debates dataset [7], which contains yet. One of our goals is to investigate various speeches 65393 speeches in total. The speeches were all held dur- from the linguistic point of view to be able to distinguish ing the meetings of the United Nations Security Council the position of the speaker according to their choice of between 1995 and 2007. The dataset contains metadata lexicon. The second criterion was the time, when the for a speaker’s name, the country they represent, as well meeting took place: for each topic we have accomplished as their role in the meeting. The language of the texts to include speeches from various stages of the respective in the dataset is English. This means that the speeches conflicts. are either being added in their original language if the After the corpus was created, a smaller subset of data speaker was using English or that an oficial UN transla- from the “100 corpus” was selected for the pilot annotation. The subcorpus consists of 20 texts. This subset of data was not meant to be fully balanced, however, its speeches are generally representable of the topics, speakers, and the time frame of the “100 corpus” as can be seen in Table 1.
The created sub-dataset has a size of 16540 tokens. The volume of the data analyzed was enough to draw preliminary conclusions on the adaptation of the annotation scenario, however, the statistics of the conducted analysis could only be viewed as preliminary.
There are various ways of analyzing attitudes and it is crucial to compare them in application on the chosen type of data to truly visualize the diference between approaches and to be able to develop or adapt the most applicable annotation scheme. For this introductory analysis, it was decided to apply the Appraisal Theory by Martin and White [5] as it is in many cases the initial one. The theory views appraisal as ‘the language of evaluation’, as ‘meanings in context and towards rhetorical efects rather than towards grammatical forms’, and, therefore, focuses on analyzing such meanings as positivity/negativity, meanings by which intensity and directness are strengthened or weakened, as well as meanings of en- Figure 2: Attitude only analysis scheme [5] gagement. The Appraisal Theory scheme may be seen in Figure 1.
The authors [5] propose to analyze appraisal in three interactional domains: engagement, attitude, and graduation. “Attitude is concerned with our feelings, including
Here, the categories of afect, judgment, and appreciation are further subdivided according to the Martin and White theory [5]. Each category is then assigned a positive or negative polarity. The presented scheme is a gories and sub-categories over the other as could be seen reduced version of the full attitude analysis scheme de- in Table 2. veloped by Martin and White [5]. The subset of speeches was annotated in attitude-type and polarity, while the categories of appraiser, appraised as well as the category of explicitness were omitted in the present study.
The selected speeches were annotated using the doccano annotation wool. Doccano is an open source text annotation tool [8]. It provides the features necessary for the current project, such as sequence labeling and collaborative annotation. It also provides some feedback on statistics of the annotations, as well as the auto-labeling feature which is considered for further annotation.
For the purposes of annotating diplomatic attitudes, a set of tags directly following “Attitude” part of the simplified Appraisal scheme 2 was manually created in the annotation tool [8]. For the purposes of this study, each subcategory of the afect, judgement, and appreciation was selected. Each tag was then assigned either positive or negative value, thus making the total amount of tags 28. The list of tags could be found in Figure 3.
After the set of tags was created and tested, the annotation process began. The annotator’s task was to empirically analyze each speech sentence by sentence and assign the suitable attitude tags to the text spans according to the description of each category [5]. The spans of text selected varied from a single token (e.g. in case of a verb conveying the attitude of judgement) up to a whole clause (e.g. in case of metaphors conveying an attitude). In the next phase of marking attitudes in texts, parallel annotation will be performed, with the evaluation of inter-annotator agreement. A GitHub repository was created [9] to store and share the annotated dataset. As of now, the 20 annotated speeches are available. The remaining part of the “100 dataset”, as well as the explicit and detailed annotation instructions should be expected to appear in the repository of the project later on.
The overall overview of the categorical distribution within the annotated speeches has revealed that even though the diplomatic speeches have shown to be rich in attitudes (with an average of 21.5 attitudes per speech), the overall polarity of the dataset could not be defined as the total number of positive and negative attitudes is very close (213 positive attitudes and 216 negative attitudes). We could also say that the three diferent subcategories are represented relatively equally throughout the annotated data.
However, when we take into account each category separately, it becomes clear that the category of Afect was found to be positive almost twice as many times as it was found negative. Whereas the category of Judgement has shown to be represented mostly negatively (114 negative instances annotated compared to the 39 positive instances), the category of Appreciation was found to have a prevalence towards being positive (88 positive instances as opposed to 56 negative instances found). Let us look at each category, as presented in Figure 3.
can observe that the most frequent category is the category of positive inclination (subcategory of Afect). It occurs 51 times throughout the dataset and the reason for NPeogsaittiivvee that is that this category was selected to annotate such protocol expressions as “Let me welcome you, Sir...”, “We Table 4 are happy to reiterate our ...”, “We would like to thank Distribution of the subcategories of Judgement withing the you...”, “I wish to congratulate you...”, etc. These messages annotated speeches of support towards the procedural events of the organization are present in each speech. Even though they are so Appreciation frequently encountered in the speeches, they are, how- impact quality balance complexity valuation ever, not the true attitudes we were aiming to find as they Positive 12 41 6 14 15 do not convey the true emotion of the speaker, and are Negative 11 37 1 5 2 just there to keep the structure of a speech, and to fulfil Table 5 the text genre expectations 3. Distribution of the subcategories of Appreciation withing the
The second most frequent category is the subcategory annotated speeches of Judgement, namely negative propriety. We consider this category to represent the most common content attitude of the annotated data. This tag could be described as positive and negative judgmental entities observes varies the judgement of the incorrectness of someone’s actions from 1.1 times for the category of normality to 7.8 times or behaviour in a direct or an indirect way. Some of the ex- for the category of propriety. Such a prominent tendency amples from the annotated data include: “thoroughly in- to a negative sentiment within this category could be adequate response”, “this approach is counter-productive explained by the nature of the data and the features of and harmful”, “Despite a clear call, the Committee (...) the specific speeches selected for the dataset. The United has resorted to an even stricter implementation of the Nations Security Council is primarily the organization sanctions”. to present, discuss, and - wherever possible - to resolve
As the categories of the positive and negative quality the international conflicts by means of diplomacy and ne(subcategories of Appreciation) were annotated almost gotiations. This also means that the speakers, especially the same number of times (37 and 41 occurrences respec- the ones directly or indirectly involved, would do their tively), they could be considered equally important for best to persuade the listeners and support their internathe diplomatic discourse of the UNSC. The category of tional partners by reflecting on the agenda in particular quality represents the expression of an attitude towards ways. The speeches of the diplomats and representatives the characteristics of the object or phenomena discussed should reflect this purpose. This is why we suggest that and could very frequently be observed as a collocation the negative, judgemental discourse is dominant. of an adjective and a noun as in “a friendly and allied The attitude type of Appreciation show the opposite country”, “just and able peace”, “a friendly neighbouring polarity of attitudes as shown in Table 5. Here, the posistate”, etc. tive sentiment is more dominant, even though often there
Looking at each of the categories’ polarities individu- are many instances of annotated negativity as in impactally in Table 3, Table 4, and Table 5, we can also observe negative (11 instances, e.g. counter-productive consecertain diferences and tendencies of how sentiment po- quences), and quality-negative (37 instances, e.g. ultralarities are distributed throughout the diferent attitude nationalist credentials). subtypes. The third category, the category of Afect, presented in Table 3, has shown two categories to convey mainly happiness incAlifencattion satisfaction security positive polarity (happiness and inclination), and two Positive 26 51 6 3 - to convey the prevailing negative polarity (categories Negative 18 5 11 12 of satisfaction and security). This peculiarity could be connected to the two features observed in the diplomatic Table 3 speeches analyzed: Distribution of the subcategories of Attitude withing the an- 1. The need for suitable rhetorical devices for expressnotated speeches ing the formal attitudes serving protocol needs; this feature is being served by the positive categories of happi
It is evident that polarity diference could be most ness and inclination. prominently visible at the polarity distribution of the 2. The nature of the diplomatic speeches with an urge Judgement category presented in Table 4. Here, in ev- to express the lack of satisfaction in a situation under ery subcategory the negative judgement prevails over discussion, in the opponents actions or intentions etc., the positive judgement. The diference in the amount of and insecurity with regard to the past, present, or future part of their speeches. These attitudes rarely convey a turn of events. true attitude towards the discussed problems and mostly constitute of diplomatic cliches. It is a question whether 6.2. Pilot annotation outcomes: clear they should be viewed as equal to the attitudes towards the content and topic of a meeting, therefore we have cases of annotation decided to create an additional pair of tags (diplomatic/Annotating the data with the Appraisal theory annota- content) to assign to all the attitudes found in the corpus tion scheme has shown good results in various cases. The in order to be able to distinguish diplomatic cliches and majority of attitudinal expressions were easily classifiable. put them aside if needed.
Of course, not all of the data could have been evaluated undoubtedly, however, there are lexical expressions that 6.3. Pilot annotation outcomes: unclear are clear in the annotation. cases and proposal for their solution
Some of the examples of clear attitudinal lexemes and
expressions in the annotated data include evaluative ad- The repeating challenges arisen during the annotation
jectives (as in the following example: “The represen- process have provided a foundation for extracting the
tatives of Brazil, Djibouti, New Zealand, Pakistan and most prominent features of the diplomatic texts which
Spain all, in an excellent [appreciation - quality - positive] manner, should be taken into consideration while continuing with
contributed their talents to the conduct of the business the further steps of the analysis.
in the Council”), some of the action nouns (as in the The first thing to note is that there are empirically
disexample: “which define abuse [judgment - propriety - negative] of tinguishable attitudes among most of the proper names,
Chapter VII of the Charter of the United Nations”), as as well as phrases used in the diplomatic speeches for
well as verbs and adverbs in cases where they express naming entities connected to the UNSC and describing
appraisal, as could be seen in the following examples events. The most obvious, yet prominent example
supfrom the annotated data: porting this statement is the name of the organization
(
Many of such expressions are recurring and some of Another example of a commonly used name could be the attitudinal expressions could be considered as atti- Bosnian Crisis when referring to the events of 1995. tudinal patterns of diplomatic speeches of the UNSC as A prominent example of such naming is the phrase they are found repeatedly in the same structural spots of so-called Bosnian Serbs often used at the UNSC by the the speeches. These expressions are connected with the speakers not accepting the identity of Bosnian Serbs. functions that a particular structural entity of a speech is This example is particularly interesting as it provides us having. Among these we can identify welcoming (as in “I with a clear picture of the positive and negative attitudes should also like warmly to welcome [afect - happiness - positive] distribution among the speakers, namely there are those The Foreign Minister of Italy . . . ”), congratulating (May who call the group of people Bosnian Serbs and then there I at the outset congratulate you [afect - inclination - positive] , Sir, are those who name them so-called Bosnian Serbs only on your assumption of the presidency of the Security thus denying them a right to call themselves a group with Council ...), expressing condolences, concerns (“The Pak- a separate identity. The term Bosnian Serbs is put under istan delegation once again expresses its deep concern question, as in the given context it slightly depreciates [judgement - propriety - negative] . . . ”), etc. At this point of the the existence of the nation by the speaker. Here, the analysis these recurring expressions are mostly found in propriety of distinguishing Bosnian Serbs is questioned the beginning and at the end of the texts, as diplomats by marking them so-called, and, therefore, fake. tend to begin and end their speeches in a more formal It is fair to note, that there are many entities and proper way suitable for the occasion and to express their point names that occur permanently and definitely prove to of view on the discussed events or issues in the middle be attitudinal expressions. A core part of this group of expressions consists of proper names containing a other features of the diplomatic discourse applied for certain attitude within them, while the other part shows implicit expression of attitude (see Table 6). Here, the appraisal in usage of an originally neutral proper name. speaker (representative of Bosnia and Herzegovina) is [cf. Security Council vs. so-called Bosnian Serbs where the comparing the UNSC with a near-sighted cat that does very name Bosnian Serbs is being deprecated]. not seem to notice violations by the opposite side of the
Taking into consideration all of the above, should we conflict, who are referred to as rats giving them a very annotate the proper names and phrases? These entities strong negative appraisal. The speaker then continues deare apparently on the other level of attitudinal analysis veloping the thought by metaphorizing the whole UNSC than we are interested in. Proper names themselves may session as being a ship that all of the states are in and be designed and perceived as phrases carrying an attitudi- which is, therefore, a common good. He encourages evnal meaning, and they can be analyzed out of the context. eryone to cooperate by bailing out water, plugging the Analyzing a speech in this way, however, would lead to leaks and keeping the course steady meaning that everyone marking such countries as the United States of Amer- should be involved, interested and actively participate in ica or the United Arab Emirates as phrases containing solving the conflict. positive attitudes even if they are not.
A similar phenomenon could be observed in other Attitude ifrm phrases which are not proper names. Analyzing phrases such as ‘peace plan’ or ‘so-called Bosnian Serbs’ also does not seem to be straightforward. Annotation of such phrases being attitudinal expressions may hugely depend on the subjective perception of an annotator.
One of the options to resolve this complication could be creating a list of such names and phases and then analyzing and annotating their inner appraisal structure on the level of a lexeme meaning. Items from the list Judgement-propriety-negative could then be avoided while annotating the speeches, however, we would already know more about their inner semantic structure.
Metaphor It is analogous to allowing the rats to guard the cheese Judgement-tenacity-negative
from the mice while a near-sighted cat judgement-capacity-negative
is asked to report as to how much cheese
is being taken and who is stealing it.
We therefore do not look
to drill holes in the hull of this ship.
(...) we will look, as any passenger would, to bale out water, Judgement-normality-positive
plug the leaks and keep the course steady.
Another device for framing an attitude is expressing emotions towards events or entities in particular contexts. An attitudinal expression cannot be evaluated and measured out of the context particularly because contexts can sometimes change the meaning of the expression towards the opposite side. These cases could be described by the following formula: ‘x = -’, ‘x in y = +’ as in exerting pressure as being inappropriate and having an implicit negative attitude and then exerting pressure towards terrorists as being something valuable and good.
Such instances are another example of a case where the annotation of the formal description of the scenario is expected to be problematic. This question is also a part of the sentiment analysis approach. To solve it, we need to find a way of capturing the context, and the syntactic frames of the words in an annotation. For example, the way of capturing the possible intensification/diminution of meaning when an adjective is preceded by an adverb like in a very bright day or a tremendously bad decision.
When such cases are identified they could be given more attention, therefore the precision of the annotation could be improved and the time spent could be reduced.
The same situation is expected to be encountered with negation elements ‘no’ and ‘not’.
the Attitude part of the Appraisal theory on diplomatic speeches. In most cases, attitudes are identifiable by the Appraisal theory annotation scheme. These include most of the nouns, adjectives, adverbs, and verbs that convey attitudinal meanings. Some of the lexemes and phrases have been found in multiple texts and are considered to be patterns of expressing attitudes in diplomatic texts.
Among these are greetings, welcoming, congratulating, condoling, and farewelling. There are phrases, which almost do not change in form and place they occur in and are applied by the speaker as classical diplomatic devices for structuring the discourse, namely welcoming the Secretary and the guest, expressing excitements or e.g. condolences towards the event under discussion.
The two attitude types most frequently found in the speeches are as of now considered to be connected to the University, Prague, 2021, pp. 1–4. URL: https://itat.ics. two features of the analyzed data. The ’Afect-inclination- upjs.sk/public/WAFNL2021OpenSessionProc.pdf . positive’ subcategory is suggested to reflect the most of [2] “Attitude.” in merriam-webster.com dictionary., formal expressions of attitudes following the diplomatic https://www.merriam-webster.com/dictionary/ protocol. Whereas the ’Judgement-propriety-negative’ attitude, 2022. Accessed: 2022-06-01. is suggested to reflect on the prevailing attitude in the [3] S. Bhattacharyya, V. Snašel, D. Gupta, A. Khanna, content of the speeches , which could be explained by Hybrid Computational Intelligence: Challenges and the cause of the meetings, the events discussed, and the Applications, volume 1, Academic Press, Chennai, diference in political orientation of the speakers. India, 2020.
Among the cases, that are considered doubtful and [4] B. Pang, L. Lee, Opinion mining and sentiment therefore could cause impreciseness of the future anno- analysis, Foundations and Trends in Information tations are: Retrieval 2 (2008) 1–135. URL: http://dx.doi.org/10.
(
(3) identifying appraisal by identifying and decoding [8] H. Nakayama, T. Kubo, J. Kamura, Y. Taniguchi, rhetorical devices in multi-word expressions such as X. Liang, doccano: Text annotation tool metaphors (e.g. “rats who are abandoning the ship”) for human, 2018. URL: https://github.com/ which requires taking into account context of the whole doccano/doccano, software available from text. https://github.com/doccano/doccano.
The first stage of annotation has provided pilot data for [9] M. Anisimova, Attitudes in the UNSC debates: annoanalyzing annotation specificities and challenges. The tated data, 2021. URL: https://www.merriam-webster. necessary changes, as well as clarifications to the anno- com/dictionary/attitude. tation scenario for the given task, should be considered.
E.g., considering broader context of the attitudes, annotating attitudes with additional set of tags, namely ’diplomatic’ and ’content’ to distinguish between diplomatic cliches and content attitudes, identifying the proper names in order to avoid annotating them as attitude entities, etc.