In the context of a political interview, the host, typically a journalist, acts as a representative of the audience. This means that, if a politician manages to convince or deal with the criticism that the host addresses, then her/his trustworthiness, reliability and credibility will be easily established. In this situation, a politician is judged not only based on one's arguments and rhetorical choices, but also on the attitude, self-confidence, and in general on an overall convincing behaviour. For example, if a politician seems to be conversationally dominant and manages interruptions to a satisfactory degree, it is more likely that the host, and therefore the audience, will be convinced by the arguments put forward by the interviewee. For this reason, analysing the combination of verbal and non-verbal elements in a political interview could be very interesting for scholars in political science and communication science, and in general to study consensus mechanisms. In this light, we present the first multimodal corpus of political interviews in Italian, and analyze how the combination of verbal and non-verbal elements can shed new light into political agendas and politicians' attitude. By 'multimodal' we mean that the corpus is composed of manual transcriptions of interviews broadcast on TV and annotated with information not only about the linguistic structure of the utterances but also about non-verbal expressions2 .
The corpus, which we call PoliModal, addresses the need to make up for the lack of Italian linguistic resources for political-institutional communication and is annotated in XML following the standard for the transcriptions of speech TEI Guidelines for Electronic Text Encoding and Interchange 3 . In all transcripts, interviewers, interviewees and other guests' turns have been enriched with the manual annotation of non-lexical and semi-lexical aspects such as breaks, interruptions, false starts, overlaps, interjections, etc. Furthermore, additional linguistic traits related to language complexity, use of pronouns and persons' mentions have been automatically tagged, enabling an in-depth analysis of speakers' attitude and communication strategy. In this work we present not only the corpus, which is made freely available at the link https://github. com/dhfbk/InMezzoraDataset, but also an analysis that, combining verbal and non-verbal elements, shows how these traits contribute to making an interview more or less convincing.
In recent years, political language has received increasing attention, especially in the Anglo-Saxon and American world, where it is possible to have free access to speech transcriptions from government portals and personal foundation websites, e.g. White House portal, William J. Clinton Foundation, Margaret Thatcher Foundation. This has fostered research on political and media communication and persuasion strategies (Guerini et al., 2010;Esposito et al., 2015).
However, not all languages are well represented in this kind of studies. According to LRE Map4 there are currently 24 monolingual corpora for Italian, two of which concern spoken language, i.e. VoLIP (Alfano et al., 2014) and LUNA corpus (Dinarelli et al., 2009), and one multimodal, named ImagAct-ItalWorNet-Mapping (Bartolini et al., 2014); no entry includes an Italian corpus for the political domain. Furthermore, researchers in Italian politics have mainly focused on political communication in the verbal modality, evaluating monological discourse (Bolasco et al., 2006;Cedroni, 2010;Longobardi, 2010;Catellani et al., 2010;Bongelli et al., 2010;Zurloni and Anolli, 2010;Sprugnoli et al., 2016;Moretti et al., 2016) to study a politician's lexical, textual or rhetorical patterns. An exception is the work by Salvati and Pettorino (2010), that diachronically analyses some of the suprasegmental aspects of Berlusconi's speeches from 1994 to 2010. The corpus, however, is not available for further studies.
Concerning political corpora developed specifically for conversation analysis, Bigi et al. (2011) present a multimodal corpus of political debates at the French National Assembly, on May 4th, 2010 and introduce an annotation scheme for a political debate dataset which is mainly in the form of video and audio annotations. Navarretta and Paggio (2010) deal with the identification of interlocutors via speech and gestures in annotated televised political debates in British and American English. Other papers have focused primarily on visual aspects (gaze, gestures, facial expressions) of communicative interaction during political talk shows or parliamentary speeches (D'Errico et al., 2010).
The most similar approach to ours is presented in Koutsombogera and Papageorgiou (2010). The authors analyse a Greek multimodal corpus of 10 face-to-face television interviews focusing on nonverbal aspects in order to study the attempts of persuasion and interruption during political interviews. Their work, however, is mainly aimed at studying the strategies for conversational dominance, and annotate specific traits accordingly. Our work, instead, is more general, includes a different set of tags and integrates also automatic linguistic features.
The PoliModal corpus includes the transcripts of 56 TV face-to-face interviews of 14 hours -taken from the Italian political talk show "In mezz'ora in più" broadcast from 24 September 2017 to 14 January 2018.The show follows a fixed format, with interviews conducted by a journalist, Lucia Annunziata, to a guest, typically a prominent figure in the political or cultural scene. A secondary guest may participate as well, usually a second journalist to comment on the debate. Each interview is done in the same limited time frame, 30 minutes, and no audience is present, so that applause and any other type of reactions are not included in the corpus.
The audio signal has been transcribed using a semi-supervised speech-to-text methodology (Google API + manual correction). All hesitations, repetitions and interruptions of the original interview have been included. The output has been further segmented into turns, and punctuation has been added, mainly to delimit sentence boundaries when they were not ambiguous.
It is important to note that, even if transcription seems to be an objective task, it involves a certain degree of interpretation. Indeed, the inclusion of the punctuation necessary to make the writing comprehensible, as well as the selection of nonverbal messages and non-verbal expressions (interjections, laughter, unfinished words, etc.) are interpretative choices aimed at revealing a sense.5 Therefore, in the case of ambiguous sentences, they have been identified manually, mainly looking at the context of the enunciation. According to (Ducrot, 1995), in fact, it is not possible to understand a communicative act without knowing the context in which it occurs. The context is therefore essential to choose one of the possible interpretations of ambiguous expressions.
In PoliModal, annotation has been done using XML as markup language and following the TEI standard for Speech Transcripts in terms of utterances. The linguistic resource has currently 100,870 tokens and includes interviews to politicians covering all the Italian political spectrum (from the extreme right movement Casa Pound to the liberal and progressive Partito Radicale). Beside politicians, also a small number of people with different backgrounds (students, academics, judges, economists, etc.) has been interviewed and is therefore included in the corpus.
For each interview the following information was manually annotated and is included in the XML resource file:
(a) metadata: these include useful information for a quick identification of transcriptions, for example the tools used for the transcription, a link to the interview, the owner account, the title of the talk show, the date of airing, the guests, etc.
(b) pause: this tag is used to mark a pause either between or within utterances. Speakers differ very much in their rhythm and in particular in the amount of time they leave between words, so the following element is provided to mark occasions where the transcriber judges that a speech has been paused, irrespective of the actual amount of silence. Several studies have converged on the conclusion that we alternate between planning speech and implementing our plans. Indeed, as shown in (Henderson et al., 1966), participants to interviews typically show a cycle of hesitation and fluency, although the ratio of speech to silence varies among speakers.
(c) vocal: with this tag we mark any vocalized but not necessarily lexical phenomenon, for example non-lexical expressions (i.e. burp, click, throat, etc.) and semi-lexical expressions (i.e. ah, aha, aw, eh, ehm etc.). These traits have been associated with the fact that linguistic planning is very cognitively demanding, and it is difficult to plan an entire utterance at once (Lindsley, 1975). Therefore, hesitation pauses and similar vocal phenomena may be useful to perform a careful lexical retrieval, since past studies (Levelt, 1983) found that pauses occurred more often before low-frequency words than before high frequency ones.
(d) del: this tag covers different phenomena of speech management, specifically false starts, repetitions and truncated words. Since they are marked in the TEI Guidelines as 'editorially deleted', the corresponding tag is del. We include these in our annotation since several past studies (Simone, 1990;Bazzanella, 1992;Tannen, 1989) highlighted their importance in spontaneous speech, mentioning in particular the role of repetitions in controlling the in-progress textual design of speech (Voghera, 2001).
(e) overlap: this phenomenon is present when the speaker conveys (in a verbal or non-verbal manner) that he/she is about to finish his/her turn and the co-locutor starts speaking so that there is a slight overlap of utterances. Overlaps can be competitive, when the overlapper disrupts the speech and can be perceived as intrusive by dominating the conversation, and cooperative, when the goal of the overlapper is to maintain the flow of the turns and add to the conversation with further comments (Truong, 2013).
In this section, we analyse several linguistic dimensions that can be either automatically extracted or derived from the corpus annotation, and that can contribute to better understand typical traits of political communication.
We first group the politicians in our corpus into political parties, and then analyse those that are represented by least 3 politicians: Forza Italia, a conservative center-right political party (3 interviews), Lega Nord, a right-wing political party often targeting immigrants (5 interviews), Movimento 5 Stelle, a populist citizens' movement (3 interviews) and Partito Democratico, a moderate centre-left political party (9 interviews). An overview of the distribution of non-verbal traits in the PoliModal corpus for each party is reported in Fig. 1. Although the graph shows some differences in the frequency of occurrences, they are not statistically significant, also because of the relatively small number of interviews considered in the study. Also, the standard deviation for the averages tends to be high, showing high differences among interviewees of the same party. For example, politicians of Lega Nord make on average more pauses, but the range goes from 0.286 per turn (Roberto Maroni) to 0 (Luca Zaia). Similarly, non-lexical and semi-lexical expressions, marked as vocal, are on average more frequent for PD politicians, but range from 1.25 per turn (En-rico Letta) to 0.10 (Matteo Renzi). These results show that differences pertain more to single persons and conversational style than to political orientation. An exception is given by overlaps, for which the three politicians of M5Stelle (Alessandro Di Battista, Luigi Di Maio, Giancarlo Cancelleri) all show a frequency above average, suggesting that it may be connected with the communication strategy of the members of Movimento.
A second analysis we carry out is related to existing works about the use of linguistic features related to political orientation. In particular, a recent study by Schoonvelde et al. (2019) has analysed more than 380,000 speeches from five different Parliaments, and has proven that ideologically conservative politicians use a less complex language than liberal ones (this result is however less clear for economic left-right ideology). Since these findings were not tested on Italian political documents, carry out a comparison using the collected transcripts. In order to analyse the complexity of the language used by each politician we computed the type-token ratio and the average lexical density, i.e. the number of content words divided by the total number of tokens. We do not take into account the Gulpease index (Lucisano and Piemontese, 1988), which is the de-facto standard metric of readability in Italian, because it was meant for written documents and heavily relies on sentence length, a boundary that is not always present in transcripts.
Fig. 2 shows the average type-token ratio and conceptual density per political party. There are almost no variations among the parties, with small standard deviations. This comparison suggests that in our case the hypothesis by Schoonvelde et al. (2019) is not confirmed, with the three highest ttr values belonging to politicians from three different parties: Forza Italia (Mariastella Gelmini, 0.87 ttr), Lega Nord (Matteo Salvini, 0.82) and PD (Michele Emiliano, 0.82). A second hypothesis we want to test is the one introduced in the work by Cichocka et al. (2016), where the authors show that Republican presidents used a higher proportion of nouns than Democratic presidents, while there were no reliable differences in the use of verbs or adjectives. The authors suggest that, compared to liberals, conservative politicians are more inclined to use parts of speech that stress clarity and predictability (such as nouns) and reduce uncertainty and ambiguity (such as verbs or adjectives). We therefore compute the average number of nouns, adjectives and verbs per political party and compare them. Similar to the previous analysis, averages are all in the same range and there is no statistically significant difference among parties. However, some of the results are in line with Cichocka et al.'s study, with PD showing a slightly lower number of nouns on average (and Valeria Fedeli being the politician with the lowest noun ratio, 0.16). Also, Matteo Salvini and Luigi di Maio are the politicians with the highest use of nouns, 0.22 per token on average. A further evidence in favour of these results are the statistics obtained on the use of content words, in particular on the percentage of nouns, verbs, adverbs and adjectives, reported in Fig 3 . We consider the five politicians with the highest number of turns in the corpus (see Table 1): Alessandro Di Battista (Movimento 5 Stelle), Carlo Calenda (PD), Matteo Renzi (PD), Angelino Alfano (Popolo delle Libertà), Matteo Salvini (Lega). The figure confirms that Matteo Salvini is the politician using the most nouns on average, in line with the findings by Cichocka et al. (2016). Carlo Calenda, instead, is the politician that on average uses most verbs and adverbs, conveying more uncertainty and ambiguity than all the other politicians including Matteo Renzi. The fact that the two studies considered do not find a clear confirmation in our corpus, where the differences among the parties are rather blurred, may have three possible explanations: i) this corpus may be too small to test the above hypotheses. Its expansion is indeed already in progress; ii) the hypotheses do not actually hold in our case, i.e. in the Italian political scene it is not true that liberals use more complex language and tend to use less nouns than conservatives; or iii) the four parties considered cannot be straightforwardly divided into liberals and conservatives, and there are different positions inside the same party.
A third analysis is aimed at studying the correlation between non-verbal traits and language complexity. We therefore focus on the interviews that have a minimal length of 50 turns. The list of politicians and corresponding count of annotated traits is reported in Table 1. Again, for complexity we consider type-token ratio and conceptual density.
We perform an analysis of the correlation between language complexity and the six non-verbal traits manually annotated in the interviews, normalised by the number of turns uttered by each politician. While type-token ratio (TTR) does not correlate with any of the manual traits, we found On the contrary, it has a moderate positive correlation with the average number of pauses (r=0.49). This result suggests that, among the manual traits, pauses are used as a linguistic device and are an indicator of a good control of the conversation. Therefore, they are more often used by politicians showing a high lexical density, i.e. the ability to convey concepts in a concise way, which is crucial especially during TV interviews. The other manually annotated traits, instead, seem to be more frequent in speeches that are less organised, for which the management of the discourse is less efficient.
Among the politicians considered in this study, Carlo Calenda makes on average the highest number of pauses (0.27 per turn on average, with a lexical density of 0.579), followed by Giulio Tremonti (0.16 pauses per turn, 0.585 lexical density).
In this work, we present PoliModal, the first freely-available multimodal corpus of political interviews, manually annotated with six non-verbal traits. The corpus covers 56 interviews, where each guest is associated with a role (for non politicians) or a political party. We also present a first statistical analysis of the traits and their association with language complexity and with the speakers' political orientation.
In the future, we plan to start from the annotated material not only to extend the corpus, but also to investigate other aspects of political communication. For example, the choice to note nonverbal expressions is motivated by the will to study Poggi (2005), persuasion strategies are multimodal constructs because politicians -specifically in televised political interviewsattempt to persuade their supporters not only by their discursive style and argumentative speech, but also through their personality and their interactional behaviour. In the context of a political interview, persuasion is related to conversational dominance, i.e. a speaker's tendency to control the other speaker's conversational actions over the course of an interaction (Itakura, 2001), which is made evident through the kind of non-verbal expressions annotated in our corpus.
Finally, since at the moment only one annotator has performed the transcription, segmentation and tagging task, we plan to compute inter-annotator agreement in the near future. The annotation task addressed so far falls -from a qualitative point of view -in the first of the general types identified by (Mathet et al., 2015), in which the subjective interpretation is limited. Indeed, it deals with the "identification of units" (Krippendorff, 2018), in which the annotator, given a written or spoken text, must identify the position and boundary of linguistic elements (e.g. identification of prosodic or gestural units, topic segmentation). We therefore expect agreement to be at least fair, but we plan to measure it using standard metrics.