English. In this paper, we present the first available corpus of Italian political interviews with multimodal annotation, consisting of 56 face-to-face interviews taken from a political talk show. We detail the annotation scheme and we present a number of statistical analyses to understand the relation between these multimodal traits and language complexity. We also exploit the corpus to test the validity of existing studies on political orientation and language use, showing that results on our data are not as clear-cut as on English ones.1
In the context of a political interview, the host, typically a journalist, acts as a representative of the audience. This means that, if a politician manages to convince or deal with the criticism that the host addresses, then her/his trustworthiness, reliability and credibility will be easily established. In this situation, a politician is judged not only based on one’s arguments and rhetorical choices, but also on the attitude, self-confidence, and in general on an overall convincing behaviour. For example, if a politician seems to be conversationally dominant and manages interruptions to a satisfactory degree, it is more likely that the host, and therefore the audience, will be convinced by the arguments put forward by the interviewee. For this reason, analysing the combination of verbal and non-verbal elements in a political interview could be very interesting for scholars in political science and communication science, and in general to study consensus mechanisms. In this light, we present the first multimodal corpus of political 1Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). interviews in Italian, and analyze how the combination of verbal and non-verbal elements can shed new light into political agendas and politicians’ attitude. By ‘multimodal’ we mean that the corpus is composed of manual transcriptions of interviews broadcast on TV and annotated with information not only about the linguistic structure of the utterances but also about non-verbal expressions2.
The corpus, which we call PoliModal, addresses the need to make up for the lack of Italian linguistic resources for political-institutional communication and is annotated in XML following the standard for the transcriptions of speech TEI Guidelines for Electronic Text Encoding and Interchange3. In all transcripts, interviewers, interviewees and other guests’ turns have been enriched with the manual annotation of non-lexical and semi-lexical aspects such as breaks, interruptions, false starts, overlaps, interjections, etc. Furthermore, additional linguistic traits related to language complexity, use of pronouns and persons’ mentions have been automatically tagged, enabling an in-depth analysis of speakers’ attitude and communication strategy. In this work we present not only the corpus, which is made freely available at the link https://github. com/dhfbk/InMezzoraDataset, but also an analysis that, combining verbal and non-verbal elements, shows how these traits contribute to making an interview more or less convincing. 2
In recent years, political language has received
increasing attention, especially in the Anglo-Saxon
2According to
3P5: Guidelines for Electronic Text Encoding and
Interchange. See more https://tei-c.org/release/
doc/tei-p5-doc/en/html/TS.html#TSSAPA
and American world, where it is possible to have
free access to speech transcriptions from
government portals and personal foundation websites,
e.g. White House portal, William J. Clinton
Foundation, Margaret Thatcher Foundation. This has
fostered research on political and media
communication and persuasion strategies
However, not all languages are well represented
in this kind of studies. According to LRE Map4
there are currently 24 monolingual corpora for
Italian, two of which concern spoken language,
i.e. VoLIP
Concerning political corpora developed
specifically for conversation analysis, Bigi et al. (2011)
present a multimodal corpus of political debates at
the French National Assembly, on May 4th, 2010
and introduce an annotation scheme for a
political debate dataset which is mainly in the form of
video and audio annotations. Navarretta and
Paggio (2010) deal with the identification of
interlocutors via speech and gestures in annotated televised
political debates in British and American English.
Other papers have focused primarily on visual
aspects (gaze, gestures, facial expressions) of
communicative interaction during political talk shows
or parliamentary speeches
The most similar approach to ours is presented in Koutsombogera and Papageorgiou (2010). The authors analyse a Greek multimodal corpus of 10 face-to-face television interviews focusing on nonverbal aspects in order to study the attempts of 4LRE Map is a mechanism intended to monitor the use and creation of language resources by collecting information on both existing and newly-created resources, free available at http://lremap.elra.info/ persuasion and interruption during political interviews. Their work, however, is mainly aimed at studying the strategies for conversational dominance, and annotate specific traits accordingly. Our work, instead, is more general, includes a different set of tags and integrates also automatic linguistic features. 3
The PoliModal corpus includes the transcripts of 56 TV face-to-face interviews of 14 hours - taken from the Italian political talk show “In mezz’ora in piu`” broadcast from 24 September 2017 to 14 January 2018.The show follows a fixed format, with interviews conducted by a journalist, Lucia Annunziata, to a guest, typically a prominent figure in the political or cultural scene. A secondary guest may participate as well, usually a second journalist to comment on the debate. Each interview is done in the same limited time frame, 30 minutes, and no audience is present, so that applause and any other type of reactions are not included in the corpus.
The audio signal has been transcribed using a semi-supervised speech-to-text methodology (Google API + manual correction). All hesitations, repetitions and interruptions of the original interview have been included. The output has been further segmented into turns, and punctuation has been added, mainly to delimit sentence boundaries when they were not ambiguous.
It is important to note that, even if transcription
seems to be an objective task, it involves a
certain degree of interpretation. Indeed, the inclusion
of the punctuation necessary to make the writing
comprehensible, as well as the selection of
nonverbal messages and non-verbal expressions
(interjections, laughter, unfinished words, etc.) are
interpretative choices aimed at revealing a sense.5
Therefore, in the case of ambiguous sentences,
they have been identified manually, mainly
looking at the context of the enunciation. According to
5As
In PoliModal, annotation has been done using XML as markup language and following the TEI standard for Speech Transcripts in terms of utterances. The linguistic resource has currently 100,870 tokens and includes interviews to politicians covering all the Italian political spectrum (from the extreme right movement Casa Pound to the liberal and progressive Partito Radicale). Beside politicians, also a small number of people with different backgrounds (students, academics, judges, economists, etc.) has been interviewed and is therefore included in the corpus.
For each interview the following information was manually annotated and is included in the XML resource file:
(a) metadata: these include useful information for a quick identification of transcriptions, for example the tools used for the transcription, a link to the interview, the owner account, the title of the talk show, the date of airing, the guests, etc.
(b) pause: this tag is used to mark a pause
either between or within utterances. Speakers differ
very much in their rhythm and in particular in the
amount of time they leave between words, so the
following element is provided to mark occasions
where the transcriber judges that a speech has been
paused, irrespective of the actual amount of
silence. Several studies have converged on the
conclusion that we alternate between planning speech
and implementing our plans. Indeed, as shown in
(c) vocal: with this tag we mark any
vocalized but not necessarily lexical phenomenon, for
example non-lexical expressions (i.e. burp, click,
throat, etc.) and semi-lexical expressions (i.e. ah,
aha, aw, eh, ehm etc.). These traits have been
associated with the fact that linguistic planning is very
cognitively demanding, and it is difficult to plan an
entire utterance at once
(d) del: this tag covers different phenomena of
speech management, specifically false starts,
repetitions and truncated words. Since they are marked
in the TEI Guidelines as ‘editorially deleted’, the
corresponding tag is del. We include these in
our annotation since several past studies
(e) overlap: this phenomenon is present when
the speaker conveys (in a verbal or non-verbal
manner) that he/she is about to finish his/her turn
and the co-locutor starts speaking so that there
is a slight overlap of utterances. Overlaps can
be competitive, when the overlapper disrupts the
speech and can be perceived as intrusive by
dominating the conversation, and cooperative, when the
goal of the overlapper is to maintain the flow of
the turns and add to the conversation with further
comments
In this section, we analyse several linguistic dimensions that can be either automatically extracted or derived from the corpus annotation, and that can contribute to better understand typical traits of political communication. 4.1
We first group the politicians in our corpus into
political parties, and then analyse those that are
represented by least 3 politicians: Forza Italia, a
conservative center-right political party (3
interviews), Lega Nord, a right-wing political party
often targeting immigrants (5 interviews),
Movimento 5 Stelle, a populist citizens’ movement (3
interviews) and Partito Democratico, a
moderate centre-left political party (9 interviews). An
overview of the distribution of non-verbal traits in
the PoliModal corpus for each party is reported
in Fig. 1. Although the graph shows some
differences in the frequency of occurrences, they are
not statistically significant, also because of the
relatively small number of interviews considered in
the study. Also, the standard deviation for the
averages tends to be high, showing high differences
among interviewees of the same party. For
example, politicians of Lega Nord make on
average more pauses, but the range goes from 0.286
per turn (Roberto Maroni) to 0 (Luca Zaia).
Similarly, non-lexical and semi-lexical expressions,
marked as vocal, are on average more frequent for
PD politicians, but range from 1.25 per turn
(Enrico Letta) to 0.10 (Matteo Renzi). These results
show that differences pertain more to single
persons and conversational style than to political
orientation. An exception is given by overlaps, for
which the three politicians of M5Stelle
(Alessandro Di Battista, Luigi Di Maio, Giancarlo
Cancelleri) all show a frequency above average,
suggesting that it may be connected with the
communication strategy of the members of Movimento.
that in our case the hypothesis by Schoonvelde et
al. (2019) is not confirmed, with the three highest
ttr values belonging to politicians from three
different parties: Forza Italia (Mariastella Gelmini,
0.87 ttr), Lega Nord (Matteo Salvini, 0.82) and PD
(Michele Emiliano, 0.82).
A second analysis we carry out is related to
existing works about the use of linguistic features
related to political orientation. In particular, a
recent study by Schoonvelde et al. (2019) has
analysed more than 380,000 speeches from five
different Parliaments, and has proven that
ideologically conservative politicians use a less complex
language than liberal ones (this result is however
less clear for economic left-right ideology). Since
these findings were not tested on Italian political
documents, we carry out a comparison using the
collected transcripts. In order to analyse the
complexity of the language used by each politician we
computed the type-token ratio and the average
lexical density, i.e. the number of content words
divided by the total number of tokens. We do not
take into account the Gulpease index
Fig.2 shows the average type-token ratio and conceptual density per political party. There are almost no variations among the parties, with small standard deviations. This comparison suggests
A second hypothesis we want to test is the one introduced in the work by Cichocka et al. (2016), where the authors show that Republican presidents used a higher proportion of nouns than Democratic presidents, while there were no reliable differences in the use of verbs or adjectives. The authors suggest that, compared to liberals, conservative politicians are more inclined to use parts of speech that stress clarity and predictability (such as nouns) and reduce uncertainty and ambiguity (such as verbs or adjectives). We therefore compute the average number of nouns, adjectives and verbs per political party and compare them. Similar to the previous analysis, averages are all in the same range and there is no statistically significant difference among parties. However, some of the results are in line with Cichocka et al.’s study, with PD showing a slightly lower number of nouns on average (and Valeria Fedeli being the politician with the lowest noun ratio, 0.16). Also, Matteo Salvini and Luigi di Maio are the politicians with the highest use of nouns, 0.22 per token on average. A further evidence in favour of these results are the statistics obtained on the use of content words, in particular on the percentage of nouns, verbs, adverbs and adjectives, reported in Fig 3. We consider the five politicians with the highest number of turns in the corpus (see Table 1): Alessandro Di Battista (Movimento 5 Stelle), Carlo Calenda (PD), Matteo Renzi (PD), Angelino Alfano (Popolo delle Liberta`), Matteo Salvini (Lega). The figure confirms that Matteo Salvini is the politician using the most nouns on average, in line with the findings by Cichocka et al. (2016). Carlo Calenda, instead, is the politician that on average uses most verbs and adverbs, conveying more uncertainty and ambiguity than all the other politicians including Matteo Renzi.
The fact that the two studies considered do not find a clear confirmation in our corpus, where the differences among the parties are rather blurred, may have three possible explanations: i) this corpus may be too small to test the above hypotheses. Its expansion is indeed already in progress; ii) the hypotheses do not actually hold in our case, i.e. in the Italian political scene it is not true that liberals use more complex language and tend to use less nouns than conservatives; or iii) the four parties considered cannot be straightforwardly divided into liberals and conservatives, and there are different positions inside the same party. 4.3
A third analysis is aimed at studying the correlation between non-verbal traits and language complexity. We therefore focus on the interviews that have a minimal length of 50 turns. The list of politicians and corresponding count of annotated traits is reported in Table 1. Again, for complexity we consider type-token ratio and conceptual density.
We perform an analysis of the correlation between language complexity and the six non-verbal traits manually annotated in the interviews, normalised by the number of turns uttered by each politician. While type-token ratio (TTR) does not correlate with any of the manual traits, we found that lexical density shows a moderate negative correlation with repetitions (n=13, r=–0.51), truncations (r=–0.46) and non-lexical and semi-lexical expressions (r=–0.43). On the contrary, it has a moderate positive correlation with the average number of pauses (r=0.49). This result suggests that, among the manual traits, pauses are used as a linguistic device and are an indicator of a good control of the conversation. Therefore, they are more often used by politicians showing a high lexical density, i.e. the ability to convey concepts in a concise way, which is crucial especially during TV interviews. The other manually annotated traits, instead, seem to be more frequent in speeches that are less organised, for which the management of the discourse is less efficient.
Among the politicians considered in this study, Carlo Calenda makes on average the highest number of pauses (0.27 per turn on average, with a lexical density of 0.579), followed by Giulio Tremonti (0.16 pauses per turn, 0.585 lexical density). 5
In this work, we present PoliModal, the first freely-available multimodal corpus of political interviews, manually annotated with six non-verbal traits. The corpus covers 56 interviews, where each guest is associated with a role (for non politicians) or a political party. We also present a first statistical analysis of the traits and their association with language complexity and with the speakers’ political orientation.
In the future, we plan to start from the annotated material not only to extend the corpus, but also to investigate other aspects of political communication. For example, the choice to note nonverbal expressions is motivated by the will to study Guest Alessandro Di Battista
Carlo Calenda Matteo Renzi
Walter Veltroni
Simone Di Stefano
Pierluigi Bersani
Angelino Alfano
Giulio Tremonti
Matteo Orfini
Luigi Di Maio
Matteo Salvini 1
Matteo Salvini 2
Pier Carlo Padoan
the strategies of persuasion used by the
speakers. According to Poggi (2005), persuasion
strategies are multimodal constructs because politicians
– specifically in televised political interviews –
attempt to persuade their supporters not only by
their discursive style and argumentative speech,
but also through their personality and their
interactional behaviour. In the context of a political
interview, persuasion is related to conversational
dominance, i.e. a speaker’s tendency to control
the other speaker’s conversational actions over the
course of an interaction
Finally, since at the moment only one annotator
has performed the transcription, segmentation and
tagging task, we plan to compute inter-annotator
agreement in the near future. The annotation task
addressed so far falls – from a qualitative point
of view – in the first of the general types
identified by