This paper presents a pilot study towards the creation of a monolingual writtenspoken parallel corpus in Italian, featuring two main novelties in the general landscape of spoken corpora: the alignment with the written counterpart of the same content and the spoken variety dealt with, represented by transcriptions of radio news broadcasting.
Nowadays, the contrast between written and spoken language does no longer represent a clear-cut opposition. The emergence of modern communication technologies such as radio, television and new (digital) media led to important changes in the analysis of the diamesic variation. Under this view, the opposition spoken vs. written language is reformulated in terms of a continuum with prototypical written and spoken language at the extreme poles and within which a cline of intermediate linguistic varieties can be recognised, mixing, to a different extent, features of the two. Nencioni (1976) defined the extreme poles of this continuum as the parlato-parlato (‘spoken-spoken’) variety, i.e. casual, spontaneous conversation, and the scritto-scritto (‘written-written’) variety, i.e. planned, formal, written language. Besides the typical contexts envisaging the use of spoken language—which require all participants to be present in the same environment, that the conversation is held in turns and that speakers make sure their messages are getting across—different contexts can be imagined: among them, the radio and television language which, despite being spoken, present traces of textual organisation recall
Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). ing the written language. Nencioni (1976) qualifies this variety of language use as parlato-scritto (‘spoken-written’), a label that emphasises its hybrid nature characterised by the co-occurrence of traits typical of both written and spoken language. From a different perspective, Ong (1982) refers to this variety as ‘secondary orality’, i.e. “an orality not antecedent to writing and print, as primary orality is, but consequent and dependent upon writing and print”.
In addition to this socio-linguistic interest, the
issue also bears relevance for computational
approaches as it has a substantial impact on the
perceived naturalness of human-machine interaction.
Indeed, one of the reasons why speech synthesis
applications still produce unnatural speech, apart
from bad prosody is that written language is
generally not suitable, i.e. comprehensible, direct and
effective, in spoken contexts
Our Corpus Italiano Parallelo Parlato Scritto (‘Spoken Written Italian Parallel Corpus’, henceforth CIPPS) features two fundamental novelties in the general landscape of spoken corpora: the alignment with a written counterpart of the same content and the type of spoken variety dealt with. 2
Notwithstanding the differences between written and spoken language styles and the impact it bears on human-machine interaction, little computational work has been devoted to develop data and methods for “transforming” a written text in a text suitable for a specific spoken context.
Previous works mostly deal with the transformation of spoken language into grammatically valid, correct written language that can be parsed by standard NLP tools—see for instance Marimuthu and Devi (2014) and Giuliani et al. (2014). However, the rise and spread of VR and AR applications seem to call for the need to appropriately tackle also the other direction, i.e. the transformation of written into (diamesically) appropriate spoken language, which presents different challenges1.
Few studies have been devoted to the automatic
transformation or generation of suitable spoken
language, mostly on Japanese. Among these,
Murata and Isahara (2001) describe an interesting
model to perform different kinds of paraphrasing
tasks, that is to transform sentences according to
1VR/AR is currently a hot topic especially in both
educational and industrial-training contexts
Kaji and colleagues (2004; 2005) worked on the transformation of written language to spoken language style in Japanese, approaching the issue as a lexical paraphrasing problem, for which they constructed an ad-hoc written–spoken web corpora focused on the connotational differences related to the suitability for orality of expressions. Their method learns predicate paraphrases from a dictionary and then uses the corpus to statistically determine whether an expression is suitable to be spoken.
More recently, Matsubara and Hayashi (2012) report about an application for generating spontaneous news speech in a news speech delivery service. They approach the issue as a text generation task and develop a rule-based system for automatically generating news speech scripts—to be read via speech synthesis—starting from newspaper articles. Their approach however focuses on a specific stylistic difference peculiar to Japanese hardly portable to other languages and does not involve any kind of parallel aligned data. 3
In this work we describe our first attempts at building a parallel written–spoken corpus that might ultimately be useful to train a system for the transformation of written text into text suitable to be spoken. We focus on two different language varieties within the spoken-written language continuum, mentioned in section 1, namely radio spoken language and newspaper written language. This focus was dictated both by the need to neutralize the effects possibly deriving from considering different topics, textual genres and/or communication contexts, and by the practical need of finding readily available data to run the pilot. Thus the present data-set is built by aligning newspaper articles, taken as representatives of the written– written variety and news broadcasting via radio, taken as representatives of the spoken–written variety. Given the goals defined above, our first step was to collect the materials for building the pilot data-set.
For the spoken data-set we chose the Lessico
di italiano Radiofonico corpus (LIR)
The written data-set was created by taking all news articles published in La Repubblica on the same dates4. Tables 1 and 2 report the figures of the data-sets.
In the case of the spoken corpus extensive extraction and cleaning work was required because the original transcriptions include many different genres (e.g. advertisements, interviews, entertainment,. . . ) and several different annotation tags.
From the selected days of the LIR corpus we needed to extract only the transcriptions of news text. The original texts in fact contain several types of annotations, all in a proprietary tagging format, and news are easily recognisable. So, for each day mentioned, we created a data-set by collating the news of the different radio broadcasters, 2Source: http://www.accademiadellacrusca.it/it/attivita/ less3ico-frequenza-dellitaliano-radiofonico-lir
The news transcriptions of the other broadcasters were too short for our purposes.
4source: https://ricerca.repubblica.it/ thus obtaining 6 spoken data-sets, one for each day. These were subsequently cleaned by using regular expressions that removed all annotation tags, which provided us with raw text data for the alignment experiment.
In Table 2 we can see the number of news extracted for each day and their average length in terms of tokens. Interestingly, but not surprisingly, we observe that newspaper articles on average are longer than radio news. 4
Once we gathered, cleaned and normalised the relevant data, we proceeded to align written and spoken texts on the basis of topic and semantic equivalence. Since the spoken transcriptions do not have an explicit marking of sentence boundaries, for the time being alignment is performed at text level; we leave sentence-level alignment for future work.
Given the six spoken data-sets and their corresponding written ones we experimented with two different methods to perform their alignment. One is based on the Jaccard index (Jaccard henceforth), the other method on cosine similarity (Cosine henceforth). Both algorithms followed one common preliminary step: for each data-set we took into consideration only nouns, verbs, adjectives and numerals, i.e. semantically heavy words.
The first method calculates similarity using the Jaccard index as a statistical index. In general, this coefficient measures the similarity of two samples through the ratio between the size of the intersection and the size of the union of the sample sets; so, in this case, the numerator is given by the overlap of words of the two documents, i.e. the number of relevant words present in both. The denominator instead is the sum of the relevant words of both documents. The computation can be represented as follows:
J (A; B) = joverlapping words in A, Bj jwords A + words Bj (1)
The range of acceptable values stands between 0 (for the couples of documents that have no words in common) and 0,5 (for the couples of documents with the highest similarity, i.e. with all relevant words in common).
The second method computes the cosine similarity between a vector representing all the relevant words in a spoken text and a vector representing a written text. Each vector contains a number of components identical to the amount of relevant words contained in the texts, the value of each component being the TFiDF value of the corresponding word in the represented text. Once all vectors were built, we compared each spokenvector with every written-vector and computed their cosine similarity. Finally, considering values of similarity in decreasing order we reorganised the pairs and completed document-alignment. The range of acceptable values for the Cosine method stands between 0 and 1, with values close to 1.0 indicating strong similarity. 4.1
The two methods illustrated above produced twelve output files, six for each method, all ranked on the basis of their similarity score in decreasing order. For each of them we considered the first one hundred spoken-written text pairs and manually evaluated their alignments on a binary scale with respect to their information content. News about the same topics, events or facts were considered good alignments. We decided to stop the evaluation at the first one hundred pairs, because after this threshold the recognised alignments were no longer significant (i.e. algorithms aligned pairs of documents with different topics).
On the 1200 manually assessed pairs we than calculated the accuracy of the two methods. We considered accuracy as the ratio between the number of aligned pairs in particular range of distance values and the total number of couples in the same range.
The graphics in Figures 1 and 2 show method accuracy for each range of similarity values, using both the 1995 and 2003 data. For example, in the range of values between 0,1 and 0,2, the Cosine method has an accuracy of 6% with the 1995 data and 22% with the 2003 data. As we advance in the higher similarity bands, we notice a growing trend for both methods, but while for Cosine we observe a gradual growth, the Jaccard method shows a faster rise. Moreover, we notice that most of the alignments occur in the lowest similarity range of value, while in the higher similarity ranges we found very few alignments (see Table 3 and 4 for details).
Remembering that the range of admissible values are different for the two methods let us focus on the results.
Cosine alignment evaluation Cosine for both data-sets has an accuracy of 100% in the range of values 0,8-0,7 and 0,6-0,5, while for the range 0,20,3 it has an accuracy of 6% for 1995’s data-sets and 22% for 2003’s data. Figure 1 shows a gap between 0.7 and 0.6 for 2003’s data. That is because, for this data-set, the cosine method did not assign values in this range. Overall, Cosine total accuracy is 61%, 53% on 1995 data and 69% on 2003 data.
Jaccard alignment evaluation In the range 0,30,2 the Jaccard method has an accuracy of 100% on both datasets; while for the 1995 data it drops to 53% in the range 0,2-0,1 and to 47% in the range 0,1-0,6. For the 2003 data in the range 0-2,01 the accuracy is 86%, which decreases to 44,8% in the range 0,1-0,07. Also in this case, as reported in Table 4, we have few alignments in higher distances despite the number of lower ones.
Overall, Jaccard total accuracy is 50%, 50% on 1995 data and 51% on 2003 data.
According to this evaluation, Cosine using TFiDF values is the best method for aligning our data.
Here is an example of text pairs with high cosine similarity values (0,7-0,8):
[Spoken]: [...] il diario di Paul Mccartney [...] rottura con i Beatles `e stato riconsegnato [...] al cantante il giorno dopo il concerto dei fori imperiali [...] Mccartney ha avuto modo di rileggere quel preziosissimo diario stracolmo di ricordi e ha confermato l’autenticita` [...] alcune frasi portano il segno della storia "Arriva John per discutere lo scioglimento della partnership" giugno millenovecentosettanta la fine dei Beatles [Written]: [...] il diario di Paul Mccartney [...] rottura con i Beatles e` stato riconsegnato [...] al cantante, il giorno dopo il concerto dei fori imperiali. [...] sir Paul ha avuto modo di rileggere quel preziosissimo diario stracolmo di ricordi, e ha confermato l’autenticita` dell’agenda. [...] alcune frasi portano il segno della storia: ‘‘arriva John per discutere lo scioglimento della partnership’’. giugno 1970, la fine dei Beatles. [...]
What follows instead is an example of a good alignment with lower cosine similarity values (0,30,2)5:
[Spoken]: se non mi attaccassero non mi difenderei [...] spiega Berlusconi [...] "Io sono un moderato" ripete il premier "Mi difendo da teoremi folli che non attaccano me ma il presidente del consiglio" [...]
[Written]: Berlusconi al contrattacco "Denuncero` chi mi offende". [...] E aggiunge che le accuse contro di lui si basano su "Teoremi folli". Teoremi ai quali [...] "Ho dato la risposta piu` moderata, contenuta e misurata che si potesse dare". [...]
The first example is also an example of high Jaccard similarity values (0,3-0,2).
In general, with both methods, the pairs of documents correctly aligned in the lower ranges of similarity show considerable differences in terms of lexical items and possibly linguistic structures, and thus represent a very interesting set of pairs for future investigation. Regarding higher ranges, we find a greater lexical overlap and a lower variation in linguistic structure. Comparing the pairs correctly aligned by the two methods we counted 77 identical ones, while the number of different pairs derived from Jaccard is 220, and from Cosine 260. In total we obtained 557 different correctly aligned pairs.
The final pilot CIPPS corpus consists of 557 text pairs corresponding to the correctly aligned and manually validated pairs of spoken and written 5For reasons of space the example texts have been arbitrarily shortened. documents resulting from both alignment methods. It can thus be taken as a gold-standard corpus of content aligned text pairs of news for the dates and years mentioned in section 3.1.
This section reports on our preliminary
contrastive analysis of CIPPS using Monitor-IT
Let us first compare the two sections of the CIPPS corpus. On the one hand, highly correlated features between the CIPPS written and spoken sections concern the distribution of nouns (both common and proper) and adjectives as well as verbal forms used in the third person singular; the correlation was calculated with the Spearman’s Correlation Coefficient (p-value 0.05). On the other hand, statistically significant different features across the spoken and written corpus sections detected with the Wilcoxon test (p-value 0,05) include specific verbal forms, deictic elements and determiners, prepositions and acronyms, as well as lexical richness (measured in terms of Token/Type Ratio). In particular, if verbal moods such as gerundive, subjunctive, infinitive and conditional are typically associated with written articles, the 1st and 2nd person of verbs in both singular and plural forms are typical of the spoken news reports. Demonstrative determiners and pronouns represent significant features of the spoken variety, whereas acronyms and lexical richness measured in terms of Token-Type Ratio characterise the written CIPPS section.
For what concerns the comparison of the
linguistic profiling results sketched above with what
we know from the literature about features of
spoken vs. written language, we observe that the
widely acknowledged fact that spoken language is
less complex than written language is declinated
here in quite a peculiar way. Differently from
the ‘spoken-spoken’ variety characterised by a
reduced number of nouns and consequently by a
lower noun/verb ratio (ranging between 0,80 and
1,
These findings, which need to be further elaborated and explored, confirm the hybrid nature of the spoken language variety represented in the CIPPS corpus, which is in line with the trend reported in the literature that the language of the radio shares features with both spontaneous oral and written language varieties. 6
In this paper we have presented our first experiments towards the creation of the CIPPS, a monolingual written-spoken parallel aligned corpus. The data for this pilot was drawn from existing corpora and archives, it was automatically aligned on the basis of two statistical methods and finally manually validated. To the best of our knowledge, this is the first attempt to build such a corpus and more research is needed to improve its potentials and increase its magnitude.
Among the open issues to be approached first is the lack of punctuation in the spoken part of the corpus, which makes automatic alignment with the written counterpart too coarse. As mentioned in the introduction, a corpus like ours might also be precious as a training set for the development of a system for transforming written into suitable spoken texts. Although little work has been done in this direction, the time is now ripe to tackle the challenge and we plan to start experimenting with both paraphrasing methods—as mentioned in section 1— and with monolingual machine translation, taking inspiration from Quirk et al. (2004) and Wubben et al. (2012). In this perspective, however, the first necessary step is to increase corpus size and improve alignment.
This work was partially supported by the 2year project ADA, Automatic Data and documents Analysis to enhance human-based processes, funded by Regione Toscana (BANDO POR FESR 2014-2020).