This paper presents an AI experiment of translation into emoji conducted on a glossary from Dante Alighieri's Divine Comedy. The experiment is part of a project aiming to build up an automated emoji-based pivot language providing an interlingua as a tool for linguistic simplification, accessibility, and international communication: Emojilingo ( emojilingo.org). The present test involves human (Emojitaliano) and machine (Chat-GPT) translations in a comparative analysis in order to devise an automated integrated model highlighting emojis' expressive ability in transferring senses, clarifying semantic obscurities and ambiguities, and simplifying language. A first evaluation highlights Chat-GPT's ability to deal with a classic archaic literary vocabulary, also raising issues on managing criteria for better grasping the meanings and forms and about the multicultural extent of content transfer.
CEUR ceur-ws.org
Consisting today in 3,782 icons, regularly updated by Unicode Consortium,1 the emoji international catalogue contains signs for facial expressions, human gestures, lfags, food, people activities,
objects, numbers, and jobs, plants,
animals, symbols of travel,
places, geometrical forms.
While the visual content seems to be able to provide an encyclopedic list with universal significance, ideally capable of conveying language-independent meanings, the interpretation of emojis is, on the contrary, highly arbitrary. They are strongly subject to ambiguities and variations due to linguistic, cultural, and personal speciifcities.
LGOBE https://unifind.unior.it/resource/person/2569 (J. Monti); https://docenti.unimc.it/m.pierucci6 (M. L. Pierucci); https://www.dii.unipi.it/user/1906 (T. Uricchio)
0000-0003-1923-3974 (F. Chiusaroli); 0000-0001-6088-415X
and besides complementing written texts in online messages and posts as a nice means to express feelings and emotional statuses, emojis are also used to completely replace verbal language statements [1, 2].
sibility of using emojis as language to convey meanings through emoji-only translations. Notable examples include the popular Emoji Dick project, the translation into emoji of Moby Dick [3], or Wonderland [4], an emoji poster created in 2014 to reproduce the full story of Lewis Carroll’s Alice in Wonderland. These earliest experiments, however, lack codification and, as such, cannot be considered as a language, that is, a shared system in the
teract the natural polysemy of emojis.3
language can be represented by Emojitaliano [6].2 Emojitaliano is an emoji code originated from a crowdsourcing experiment initiated by a social community, specifically created to share a common emoji language able to coun
Story of a Puppet [7] (figure A.1), the structure and glossary of Emojitaliano have been afterwards usefully reapplied for the translation of texts of diferent genres such
res-2f30d44e-89c2-11e8-a7cb-00271042e8d9_%28Neologismi%29
as the technical declaratory prose of the Italian Constitu- The paper is organised as follows: section 2 introduces tion (figure A.2), the Manifesto of non-hostile communi- the Emojilingo project Parole di Dante, the subject being cation (figure A.6), the narrative prose of classic moral translations in emoji of 365 words (Parole di Dante) from tales (i.e., The Wolf and the Lamb, in figure A.5), Gia- Dante’s poem Divine Comedy. Section 3 presents the como Leopardi’s lyrical poem L’infinito 4 (The infinite, in AI translation experiment carried out with two versions ifgure A.4). of Chat-GPT (3.5 and 4) [15] on the 365 Dante’s words,
Emojitaliano is based on the assessment of conven- with a focus on the method and descriptions of some tional meanings and syntax, capable of guaranteeing the examples. Section 4 provides an evaluation of the results, sharing of sense by means of intersemiotic translation, also obtained through AI models and through a similarity beyond subjective interpretations.5 Emojitaliano pro- matrix, and the closing section includes conclusions and vides a grammatical structure and a shared vocabulary ideas on future work. which can be expanded and re-shared with each new translation[8].6
Recent experiments have opened new research hori- 2. Emojilingo: Parole di Dante
zons in evaluating the capability of large language
models (LLMs) to translate words or text into emojis. This The Emojilingo project is presented here as a follow
is predicated on the assumption that, given LLMs are up of Emojitaliano. The general idea is that, through
trained on extensive corpora sourced from the internet, the Emojitaliano community as control group, LLMs
they have been exposed to emojis and are able to grasp technologies can develop and speed up the processes
the semantics of emojis [9]. Recently, Text2Emoji [10] of translation, enable wider and easier dissemination of
was proposed as an automatic translator, based on a large the code, overcome the barriers of natural languages. The
text-emoji parallel corpus, created by prompting the LLM, Emojilingo.org website republishes some Emojitaliano
Chat-GPT
Among the experiments, a first attempt using Chat- (Dante’s words). It is well known that Dante’s vocabulary GPT to learn the Emojitaliano grammar was also carried may be dificult to understand for foreign speakers, and out in 2023 by the Emojitaliano research group. Assum- that a similar dificulty may occur for Italian speakers toing the fundamental role of a conventional syntax as a day, too, because of the many archaic or disused poetical basis for each shared code [12, 13], the aim was to verify words in the poem. Consequently, we believe that transthe ability of LLMs to learn and reapply the Emojitaliano lating this vocabulary into emojis can help mitigate these grammar rules to produce translations of Pinocchio on comprehension dificulties and facilitate understanding. its own [14]. Parole di Dante consists of 365 emojis which are the
In this paper we present a follow-up experiment of Emojitaliano translations of 365 Dante’s words. The automatic translation into emoji, focused on special vo- source, together with the original context and explanacabulary. Chat-GPT’s translations of an authorial lexicon tory comments, was published during 2021 as a daily sohave been tested and then compared to the corresponding cial media dissemination event by the Italian Accademia human solutions.The purpose is to test LLMs capabili- della Crusca.7 On that occasion, through the particities in autonomously rendering complex vocabulary, in pation by the Emojitaliano Twitter/X community,8 the the horizon of building a translation tool into emoji as a #emojitaliano and #scritturebrevi social community promeans of language simplification: the general project and duced emoji matches for all the 365 words, one per day. the conlang itself are named Emojilingo and available Parole di Dante (Dante’s words) is therefore a glossary of online on emojilingo.org. Dante Alighieri’s Divine Comedy translated into emojis, with the corresponding Italian words. 4https://www.scritturebrevi.it/?submit=Search&s=emojitaliano 5The Emojitaliano grammar and glossary are registered on @emojitalianobot on Telegram; 3998 members are signed up to @emojital- 7https://accademiadellacrusca.it/it/dante ianobot as of October 16th, 2024. 8The main accounts involved are @fchiusaroli and @stellissa/Maria 6https://www.scritturebrevi.it/emojitalianobot Stella Bottai.
During a university course exercise,9 the glossary was later provided with the English matches.10 A comparison between emoji solutions and English correspondences was discussed in the classroom, bringing out the feasibility of a chain translation method “text-emoji-text”. As a subsequent step, the research group tried a LLMs experiment of translation from Italian and English into emojis.
The resulting combination of human and AI translations, appropriately selected as will be shown below, is Parole di Dante in Emojilingo.11
In this section we present the translation experiment of the 365 Italian terms from Dante’s Comedy with ChatGPT. On the Emojilingo website, the translations chosen by Chat-GPT 4 (from the final evaluation explained in section 4.3) are available.12
The very large database on which the LLM architecture is based makes us assume that the machine knows the original text, belonging to the world literary canon, and we may also assume that it knows the English version of the work, as well as it will presumably have available multilingual Dante’s glossaries and commentaries. Unlike human translators, who translated on the basis of the explanations provided by the Accademia della Crusca, we decided that the only input to be given to the
10Sourced from https://dante.princeton.edu. 11https://emojilingo.org/parole_di_dante_about 12In addition, the full data is publicly available both on an online spreadsheet (https://docs.google.com/spreadsheets/d/ 13vkH3a-C0OpVTm9r5daFg_y0MN8lPASwGICaa72zaGg and on GitHub (https://github.com/EmojiLingo/emojilingo.github.io/tree/ main/_chatgpt)). machine would be author’s name and title of the work, instead. This approach allowed us to test Chat-GPT’s “autonomous” ability to handle this special lexicon directly.
In table 1 we present some examples of the translation experiment with Chat-GPT 3.5 and Chat-GPT 4 for the rendering of some terms, either rare or unusual, or now dismissed. The columns in the table are the following: • the original Dante’s term; • the original verse containing the word; • the explanation by the Accademia della Crusca (AC); • the crowdsourced Emojitaliano translation; • the Chat-GPT 3.5 (IT) translation; • the Chat-GPT 4 (IT) translation, and • its explanation by Chat-GPT 4 (IT) itself (translated into English for dissemination).
For our experiment we chose two models of Chat-GPT: Chat-GPT 3.5 (turbo-0125), and Chat-GPT 4 (0613) as our reference models, to examine the diferences in machine translations between the two models. In the second phase of the project, we compared and evaluated these two versions against the human translations (Emojitaliano) .
To automatically translate the words into emojis with Chat-GPT we adopted a zero-shot prompting approach ’
GPT-4 (IT) Explanation The pig emoji represents gluttony and greed, while the plate emoji recalls the act of overeating. These two emoji together symbolise a “lurk”-a spoiled and gluttonous individual.
The ‘baiulo’ is a professional figure who boasts of transporting something, so I chose the emoji of a worker (rural but representing the ethics of work) and a suitcase to represent the transportation.
An emoji depicting a person climbing or scaling, followed by a downward arrow to represent the action of falling or plummeting.
using OpenAI APIs.13 Despite the archaic and often obscure vocabulary, as already mentioned, no preliminary training was provided and the only context given in the prompt was the reference to the work’s title. The prompt was provided both in Italian and English (the latter using English terms), although the final evaluation was done using the Italian version. The English prompt is here provided: According to a preliminary evaluation, it is immediately apparent that both versions of Chat-GPT can provide interesting translation solutions of Dante’s words and are able to motivate their choices in a meaningful way.
One initial observation is that the translation solutions provided by the two Chat-GPT models often include multiple emojis, with Chat-GPT 3.5 doing so 88% of the time I will give you a word from Dante’s Divine and Chat-GPT 4 at 81%. In contrast the Emojitaliano Comedy and ask you to invent a trans- shows a higher tendency to use single emojis, doing so lation in emoji. Respond with a single in 49% of the cases. translation in 2 lines of plain text (with- The two Chat-GPT versions rarely provide the same out formatting): translations, except for some terms related to animals, such as ‘colubro’ or ‘lonza’. In some instances, particu- translation into emoji larly those involving realia (e.g., ‘eagle’, ‘angel’, ‘book’, - brief explanation of the choice. ‘galaxy’), the translations provided by Chat-GPT align The word is ‘{term}’. with those given by human translators.
In most cases, however, the solutions generated by
In the next section we will present some comments as Chat-GPT 4 difer, as do the accompanying explanations. well as some evaluation remarks. The diferences between the translations provided by the two versions of Chat-GPT are most often largely disparate. For example, the phrase ‘dolenti note’ is trans4. Evaluation lated by Chat-GPT 3.5 as and by Chat-GPT 4 as . Additionally, there are diferences in the order4.1. Initial Comments ing of emojis, as observed in the translation of ‘occhi di The following are some initial comments on the examples bragia’, where Chat-GPT 3.5 uses and Chat-GPT reported in Table 1. 4 uses . In other instances, while both versions include a common emoji, they are paired with diferent lurco All the translations use an animal to represent the additional emojis; for instance, ‘inanellare’ is translated negative qualities expressed in the text, likely due to a by Chat-GPT 3.5 as and by Chat-GPT 4 as ; plausible interference from the English word ‘lurk’. Chat- ‘colubro’ is rendered with the snake in all cases, but GPT 3.5 focuses on the environmental nocturnal context Emojitaliano adds the skull to convey the accurate instead of the vice of gluttony. The choice of a specific an- meaning of the poisonous animal, as derived from the Acimal, as the pig or the wolf, to convey negative semantic cademia della Crusca comment. In some cases, Chat-GPT values, reflects an Eurocentric view, which raises issues translations correctly grasp the core idea of the word but for the multilingual and multicultural reception. dismiss the figurative strength of the original: ‘intuarsi’, meaning ‘intimate and deep understanding’ and ‘interbaiulo The human translator reproduces the complex penetration between minds’, is in fact one of Dante’s meaning of the archaic word for “Emperor” as “bearer of original coinages (see also ‘infuturarsi’, etc.). Chat-GPT the sign of Empire”, while Chat-GPT translates it more 3.5 versions of ‘intuarsi’ as and appear not simply as the action of “carrying”, which simplifies but so poignant as the human literal solution seems more clarifies the direct meaning. expressing . Sometimes the Chat-GPT version succeeds in reproducing the sense more physically than accafi Both the human and Chat-GPT 4 translations the human one, as for ‘trasumanar’ ‘to rise above the huconvey the semantic value of rapid movement and ag- man’, by Chat-GPT 4 compared with the human gression, while the Chat-GPT 3.5 version emphasizes the version and Chat-GPT 3.5 translation . sarcastic tone used to depict despicable characters. The In a few cases, the translation solutions provided by issue of the symbolic representation of the animal icons both versions of Chat-GPT misinterpret the intended also emerges here. meaning of Dante’s word. For example, Chat-GPT 3.5 translates ‘zeba’ (meaning ‘goat’) as , erroneously conlfating it with a similar-looking word. Similarly, ChatGPT 4 translates the term as , misinterpreting it as ‘cattle dung’.
To evaluate Chat-GPT 4’s ability to suggest the best translation solutions we organised an evaluation task run by the model itself using the human crowdsourced translation, the Chat-GPT 3 and the Chat-GPT 4 ones.
Also, in this case we adopted a zero-shot prompting approach. The original Italian prompt is translated into English as follows:
lations from archaic Italian words extracted from Dantes’s Divina Commedia into emoji. I will provide you with: - The Italian word - Emoji translation A - Emoji translation B - Emoji translation C I ask you to tell me which translation into emoji do you prefer and why. Respond with 2 lines of plain text (without formatting) with the following info: - Choice: <emoji string> - Explanation: <Brief explanation of the choice> Here you are: - Italian word: term - Translation A: emoji1 - Translation B: emoji2 - Translation C: emoji3
to perform 10 retries and select the most frequent answer.
As the model was evaluating several translations ex aequo, we decided to reiterate the process until a diference was reached between the first and the second preferred translation. The results of this evaluation task are shown in table 2.
The data from the Chat-GPT 4 evaluation shows that Chat-GPT 4 has the highest preference score at 38.1%, followed closely by Emojiitaliano at 31.8%, and ChatGPT 3.5 at 30.1%. This suggests that Chat-GPT 4 was generally rated more favourably compared to Chat-GPT 3 and Emojitaliano.
The distribution of proportions is essentially symmetrical and balanced. The currently preferred translations have been compiled into a corpus validated as Emojilingo. 162 o n a i l a t i j o m E203 243 284 324 365 1–8
Chat-GPT3.5EN
Chat-GPT4EN
Chat-GPT4WIN 1.0 language: Emojilingo. Using a zero-shot prompting ap
proach, both Chat-GPT versions (3.5 and 4) provided CCChChhaCahathEta-t-Gam-tGG-tPGo-PPTGjTPTitP3Ta43T.l54.iW54aEEnIEIINNNTTomojitalia0000n0o....1.311116323Chat-GPT3.500000IT....1.411112269 Chat-GPT400000IT.....151111C38493hat-GPT3.5E00000N.....11211152642Chat-GPT4E00000N.....11211172283Chat-GPT4W00000IN.....11145375236 00000000........23457689 vifllttrsseebhCiaahmoenmoenaotlmrltcTnimiguioemaoantdhuatejnenrbiigulneidaooeudstltnysnAptrtilipdio,ifeaascsrc-Irlearnlrefafumoeenylmsasetojtnae.lcoiaeainaeonsteottcyWtaignauttnttineno,trbrewseecidaieanexv.hnsn,islsipestoaogneshWuefcniaslaaorcnacf,engncieeraetdmaxshdidspc3lieopa,stdet6arollncioaneasl.5btnosyeboentimtn.ewlilspahsenihnttutatrc,oohegiyaoonabsosrdedtdutwdseantacttoucgtssatpCorhhhiciaenufrrniveheonnxodtasegdairtsdsgmseartl.istuwl-aatseshalpIGocctsymoncieeieuttPomdhecshdottenTeudiihhisjenabmncieisflseeysicoiroont’raveotfsslhCnnoeuaomettesdadhlhwtnhcutibsEaersooDeanaiateimnnlrt-ndamtitceGsvmtenriohycfeaeacPttjtooarneynihahTt--sin’snoyeecs.t-versatility of the emoji code to convey the senses.
Figure 2: Similarity matrix between all models. Within this broad faculty of choice, however, some options seem quite critical, due to the dissimilarity of cultural values expressed by the languages, and by the 4.4. Similarity Matrix emojis themselves. That is, a main consequence of using AI for translation, also in emojis, is the reafirmaIn figure 1 we report the similarity matrix between all tion of the crucial challenge in international translation: Emojitaliano 365 values and the corresponding values the need for careful attention to specific cultural dimenprovided by all Chat-GPT engines (both for Italian and sions during localization [19]. Cultural values underlie English).14 In figure 2 we report the similarity matrix texts, words and languages, as, for example, a ‘pig’ is an between all model pairs. In both figures, we include the ‘occidental’ symbol for negative concepts as ‘dirt’ and selection of the Chat-GPT 4 evaluation agent presented ‘gluttony’ (as in ‘lurco’), while the animal has a totemic in section 4.3, using the label Chat-GPT 4 WIN. or sacred value elsewhere; likewise, colors, or gestures,
The similarity score between two strings, is computed take on cultural values according to societies and canusing the Levenshtein distance: .15 It is defined as the not be accorded univocal international meanings. The minimum number of single-character edits (insertions, choice of an icon as and international multilingual sign deletions or substitutions) required to change one word cannot override cultural peculiarities. Finally, cultural into the other. It is then normalized (i.e., divided by the vocabularies may vary on the basis of literary contexts maximum length of either strings). Finally the similarity and textual genres, often conveying suggestions related is obtained as 1 − . The similarity is 1 (black) when the to signifiers that are now lost. Given the conservative two strings are identical, and 0 (white) when they have structure of poetical language, emoji translations may no emojis in common. In the heatmap of figure 2 the therefore need to move beyond the broadness of interlinsimilarity between two models is computed as the mean gua to fully convey meanings by reproducing linguistic between all the 365 term-pairs similarities. signs ‘verbatim’ (es. ‘intuarsi’ ): that is, the literal solution, usually ruled out from the perspective of 5. Conclusion an international semantic code, becomes substantial to recover the cultural dimension of a literary text [20]. SpeIn this paper we presented a translation experiment into cial care is therefore required in selecting corresponding emojis using two versions of Chat-GPT, to compare them matches in emoji so that they do not conflict with recepwith a human version, already available, realized within tion in diferent countries and societies and so that they the framework of the Emojitaliano experience. The present do succeed in reaching the core content of the original, project focuses on an integrated translation program, that which is the main purpose of ‘the emojilingua’. combines both human (Emojitaliano) and automated ap- Future research will always need a human evaluation proaches, as a basis for a constructed emoji-based pivot of automated outcomes, carried on by a team with extensive expertise in cross-cultural perspectives, and with a deep understanding of cultural values of emojis. This will 14For clarity we include also Emojitaliano in the the first column, help to limit unrestricted creativity and ensure a wide wrehfeicrehnicsea.ll black as it is identical to the original values used as common comprehension of Emojilingo, and its highest 15https://en.wikipedia.org/wiki/Levenshtein_distance exportability.
A. Emojitaliano Works 1–8 ⌊⏳♾⌋ '⌊⌋ ⌊⛰⌋ , ➕ , ➰ ' . ☝ ' ➕ ' , ➕ ' ♾ , ➕ , ➕ ; 〰 ❤ ' . Figure A.4: L’“Infinito” in Emojitaliano La tramontana e il sole Si bisticciavano un giorno il vento della tramontana e il sole, ⌊❄⌋ '⌊‼◀⌋ ' l'uno pretendendo d'esser più forte , ➕ , dell'altro, ⌊⌋ '⌊◀⌋ ⌊⌋ ' . 2⃣ ⌊⌋ '⌊☝ ◀⌋ ⌊⌋ '⌊◀⌋ '⌊✌◀⌋ ' ⏩ ⌊⌋. , ⌊❄⌋ '⌊◀⌋, ⌊ ⌋ '⌊◀⌋ ;