This work reviews recent publications addressing the Twitter translation task, and highlights the lack of appropriate corpora that represents the colloquial language used in Twitter. It also discusses the most well-know issues in the Twitter genre: the use of hashtags and the amount of OOVs, with especial focus in comparing the di erences between formal and colloquial texts. Resumen: Este trabajo resume las publicaciones recientes en el area de la traduccion automatica de tweets, destacando la falta de un corpus que represente el lenguaje coloquial presente en Twitter. Tambien se tratan los problemas mas conocidos del genero de Twitter: el uso de hashtags i la gran cantidad de palabras OOV, con especial enfoque en las diferencias entre tweets formales y coloquiales.
The success and increasing popularity of
microblogging has raised the need to analyse
and process its content. Traditional methods
for natural language processing fail when
applied over these texts. The reason
is not circumscribed to few nor simple
issues. Roughly, microblogs documents do
not follow the traditional structure of a
formal text or document, they use a number
of language variants, styles and registers
among other linguistic phenomena, and can
even include multimedia content as a way of
communication
Machine Translation (MT) is a hard task within the natural language processing eld. It has received considerable attention during the last decades, and it is still an active eld with many research challenges. As in other natural language processing tasks, it counts among its di culties the ambiguity of the language, and the need of corpora and a gold standard. The former can be addressed by analysing the context in which a sentence occur, while the second has been typically addressed by combining large amounts of general purpose data and smaller subsets of domain speci c datasets. The creation of a gold standard in MT requires the use of parallel data that helps to assess the quality of the output.
When addressing the automatic
translation within the microblogging
genre, one has to deal with the additional
di culty of having little or no context and
the fact that microblogs exhibit eeting
domains. Twitter is not di erent from
other microblogs, and has, in addition, its
own particularities. As described in
In this document, Section 2 summarises recent studies in this eld and di erent approaches followed to address these phenomena. Next, Sections 3 to 5 give a numerical analysis of 6 di erent corpora of tweets written in Basque, Catalan, and Spanish. The goal of this analysis is to sketch the content of the Twitter messages (tweets), highlight which are their principal characteristics and discuss the di erences between formal and colloquial tweets. 2
The automatic translation of tweets, in general, is more di cult than regular MT. Although the MT community has already addressed the translation of tweets, there are still few works in this area, mainly because of the lack of corpora, and especially those showing a fair representation of colloquial texts. The number of authors publishing content in multiple languages is not small, but their messages tend to be correct and well structured, in contrast to those posted by the gross of the users. 2.1
The availability of parallel corpora for
Twitter is growing but still scarce. The
following four works gathered parallel data
following diverse approaches, but them all
contain formal texts only.
In contrast, the following four works
deal with the noisy input from colloquial
texts, but either they do not belong to
the Twitter genre or they do not contain
parallel data.
Although the previous works addressed
di erent problems, they share a common
ground on the principal di culties of the
Twitter genre. First, the translation of
hashtags is an open issue that includes its
segmentation, identi cation and analysis of
its role in sentences
The increasing interest in the eld has
promoted the design of tools to create
especialised corpora. However, the human
translation of tweets also raises open
questions
All in all, the creation of synthetic corpus
to simulate these phenomena seem a feasible
approach
The next sections analyse six datasets of
tweets from the Tweet-Norm
A set of four datasets was obtained from the Tweet-MT corpora. It consists of 2 bitexts for Catalan{Spanish and Basque{Spanish language pairs. The four datasets contain both, the development and the test sets for each language: CAES.ca, CAES.es, EUES.eu and EUES.es. The tweets in these datasets were obtained from a sample of manually selected accounts of authors that tend to tweet in various languages, being namely public organisations and personalities. Hence, the content of the messages is mainly formal, i.e., they do not contain misspellings and do not abuse of the use of symbols.
The fth dataset, TNORM, was obtained from the Tweet-Norm corpus that gathered a random selection of geolocated tweets within the Iberian peninsula, excluding multilingual areas where other languages than Spanish are spoken. The corpus was processed to identify and annotate out-of-vocabulary words. Hence, it contains not only correct messages, but also colloquial ones. The dataset used in this work contains the two development sets and the test provided in the workshop.
The last dataset used in this work is TSM. It is a portion of the Social Media Corpus, and in particular the corpus of tweets in Spanish. It contains a EUES.eu
EUES.es # tweets # tokens avg. tokens/tweet # tweets # tokens avg. tokens/tweet # tweets # tokens avg. tokens/tweet general domain set of tweets randomly selected. So similarly to TNORM, it contains both formal and colloquial tweets. They were manually processed to classify them according to the language of the tweet and annotate di erent layers such as communication function, polarity, target, and topic. This process included some clean up of the twitter mark-up for privacy reasons. Hence, the author id and user mentions, hashtags and URLs were substituted with the labels @USER, #HASHTAG and [URL], respectively.
The six datasets were processed to
have similar characteristics: the tokens
that correspond to the author id and RT
(re-tweet) were removed when present, and
they were tokenised using an adaptation
to Spanish and Catalan languages of the
Twokenize tool
Although tweets are similar in length, a deeper analysis of their content shows remarkable di erences between the formal and the colloquial corpora. This section analyses the use of user mentions and URLs whereas Section 4 analyses the use of hashtags. Although dealing with user EUES.eu
EUES.es # URLs avg. URLs/tweet % URLs wrt. tokens # URLs avg. URLs/tweet % URLs wrt. tokens # URLs avg. URLs/tweet % URLs wrt. tokens
In contrast, the use of URLs seems to be
consistent across the two types of datasets.
The four bitexts contain almost the same
number of URLs, and we can nd almost
one URLs in each tweet. In return, TNORM
and TSM contain a remarkable small number
of URLs, less than 0:1% per tweet. Out of
curiosity, the majority of URLs in the bitexts
link to documents in the same language
as the tweet. Given that the selected
authors post multilingual messages, it seems
reasonable that they also link to the right
URL when available.
# hashtags
# hashtag types
# avg. hashtags/tweet
% hashtags wrt. tokens
# tweets > 1 hashtag
# hashtags
# hashtag types
# avg. hashtags/tweet
% hashtags wrt. tokens
# tweets > 1 hashtag
# hashtags
# hashtag types
# avg. hashtags/tweet
% hashtags wrt. tokens
# tweets > 1 hashtag
This section analyses the use of hashtags in
the datasets. This study and the next one in
Section 5 follow the procedure in
A more interesting issue is the translation of hashtags. In terms of the number of occurrences, each side of the bitexts contain a similar amount. However, the number of hashtag types in CAES.ca is much lower than the ones in CAES.es. A peer review of the hashtag sets reveals that the Spanish versions contain more written variants than their counterparts in Catalan. For instance, the hashtag \#revistapremsa" (Catalan) has four variants in the Spanish text: \#revista", 1The number of hashtag types in TSM is 1 because the corpus contains only the #HASHTAG label. TSM TNORM EUES.es EUES.eu CAES.es CAES.ca % tweets with a prologue % tweets with an epilogue % of # in a prologue % of # in an epilogue % tweets with a prologue % tweets with an epilogue % of # in a prologue % of # in and epilogue % tweets with a prologue % tweets with an epilogue % of # in a prologues % of # in a epilogues EUES.eu
EUES.es \#revistadeprensa", \#revistaprensa", and \#revistaprensa".
According to
On the OOV words in Twitter The use of out-of-vocabulary (OOV) words in Twitter has been claimed to be a hard issue. The reason is not only the high number of misspellings, symbols and orthographic errors, that could be partially tackled by using spell-checkers, but also the use of speci c lexica and lexical variants. For instance, the use of word combinations (e.g., in hashtags), the combination of di erent languages (especially in multilingual regions, but also English terms) and the unlimited ability of the microblogging sphere to invent new terms.
This section gives a numerical analysis of OOVs that occur in Twitter. In order to conduct this analysis, the datasets were processed to remove the user mentions and URLs, since them all are tokens that do not need to be translated. Some variants of the datasets were built. First, only the CAES bitext was used due the lack of a Language Model (LM) for Basque. Then, since the TNORM annotations provide the corrected forms for some OOV tokens (only spelling variants), they were used to build a new dataset TNORM-S were OOVs were substituted with the correct word when available. In addition, two di erent versions TNORM
TNORM-S were created out of each dataset. In the rst one (clean data), the hashtags were kept (the # symbol was removed) since they play an important role in the text, carry part of the semantics of the message and need to be translated in most of the cases. In the second dataset, all the hashtags were removed. The purpose of this second version is to highlight the impact of hashtags in the perplexity estimation of the texts.
Table 5 shows the results of this analysis. As expected, colloquial datasets contain a higher number of OOVs. The TNORM-S contains slightly a lower number of them in comparison to the non-normalised version, which indicates that the use of spell-checkers and the substituion of lexical variants in not enough to deal with OOVs. This is re ected in the gures on the perplexity of the datasets. The perplexity is high across all the datasets, and it slightly decreases after removing the hashtags from the data, indicating that the language used in the text is notable di erent from the LM. This can be ascribed to the fact that the LM was build using an out-of-domain corpus. In turn, removing the hashtags from the data decreases the amount of OOVs, and seems to have an impact only in the formal dataset, where half of the OOVs occur in the hashtags. However, their proportion is smaller when compared with the colloquial datasets.
For the sake of comparison, the same calculation was carried on using a LM trained on TNORM corpus, the only corpus publicly # OOV - clean data # OOV - no hashtags ppl - clean data ppl - no hashtags 11:08% 10:30% 591 591 11:26% 7:51% 735 669 available out of the two colloquial ones. The new LM was used to obtain the % of OOVs and perplexity estimations on CAES.es and TSM datasets. The results are shown in Table 6. The % of OOVs is higher in both cases, most probably due the small size of the corpus. However, the perplexity of the TSM dataset has decreased. This seems to indicate that the LM was able to capture a high proportion of the particular characteristics of colloquial tweets, and that these may be recurrent in the colloquial genre and do not appear in formal texts. 6
Conclusions and Further Work Twitter has its own particularities that makes it a hard genre to deal with. This work reviews recent publications that address the problem of Twitter translation. The number of works in this eld is still scarce due the lack of corpora, but also because of the lack of a gold standard and speci c evaluation methodologies that can help to assess the quality of a tweet translation. This work also discusses the most well-know issues in the Twitter genre: the use of hashtags and the amount of OOVs, with especial focus on comparing the di erences between formal and colloquial texts. The results obtained are preliminary, but they clearly show that these two registers are di erent not only from a linguistic point of view, but also in terms of tweet structure and content. Further work has to be done to align the hashtags and the OOVs in bitexts corpora and analyse the way their are translated. Also, the annotation layers of the TSM corpus enables the possibility to ne-grain the study, for instance, by focusing in the di erences between tweets with di erent communication functions. To conclude, no major di erences were found between languages, but this may be ascribed to the fact that the datasets were obtained from bitexts corpora.