English. Starting from the crowdsourcing experience of Pinocchio in Emojitaliano [1], the present paper intends to test Chat-GPT's ability to take on the Emojitaliano grammar and dedicated glossary to verify and reapply the Emojitaliano rules in order to produce translations on its own. A test of re-translation of Pinocchio is presented here. Italiano. A partire dall'esperienza in crowdsourcing di Pinocchio in Emojitaliano [1], il presente contributo intende testare la capacità di Chat-GPT di assumere la relativa grammatica e il glossario dedicato per verificare e riapplicare le regole della emojilingua allo scopo di svolgere traduzioni in proprio. Si presenta qui un test di ritraduzione di Pinocchio.
1. Introduction Melville), was the Emoji Dick project 1 by Fred Benenson
(2010). Starting from the English version of the novel,
Consisting today in over three thousand pictograms and each sentence was translated into an emoji version via
symbols, and regularly updated by Unicode Consortium, crowdsourcing. Each of Moby Dick’s 6,438 sentences has
the emoji international catalog contains signs for facial been translated 3 times by diferent Amazon Mechanical
expressions (smileys) and for human gestures, portraits Turk (MTurk) workers. The resulting emoji sentences
of people, plants and the animals, reproductions of food were then chosen by voting by another set of workers,
and objects for everyday activities and sports, symbols of and the most popular version of each sentence was
setravel and places. Whereas the visual content seems to lected for inclusion in the book. The outcome is a
wonderprovide an encyclopaedic catalog with a universal status, ful but inconsistent translation of the same terms
accordideally able to signify language-independent meanings, ing to the wisdom of the crowd in good sense, but without
the interpretation of emojis is, on the contrary, heavily any shared rules, structure or grammar, leading to the
arbitrary, subject to ambiguities and diferences due to impossibility of recovering the original text or meaning.
linguistic and cultural specificities [
Some eforts were made to develop an emoji based “Alice’s Adventures in Wonderland” by Joe Hale2 (2014).
language that could be shared among diferent cultural In this case, each word was directly translated into a
peoples. The first notable project that made an efort corresponding emoji. Consistency was thus guaranteed
of translating a classical novel (“Moby Dick” of Herman as the same word was translated with the same emoji,
introducing a de-facto lexicon. Nonetheless, no grammar
CLiC-it 2023: 9th Italian Conference on Computational Linguistics, structure is developed as the translation follows verbatim
Nov 30 — Dec 02, 2023, Venice, Italy the original text and its English-based word order.
* Corresponding author. In order to counteract the natural polysemy of emojis
†"Thf.ecsheiuasuatrhoolri@scuonnitmricb.uitte(Fd. eCqhuiaulslya.roli); tiberio.uricchio@unimc.it [
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License 2https://www.joehale.info/visual-poetry/wonderland.html CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) 3https://www.treccani.it/vocabolario/emojitaliano prehensible and readable in all languages of the world.
For this reason, Emojitaliano consists in a unique project that provides a grammatical structure and a shared vocabulary.
Emojitaliano is thus based on the assessment of
conventional meanings, capable of guaranteeing the sharing
of sense by means of intersemiotic translation [
The Wolf and the Lamb), Dante’s allegorical poetry of The Comedy, Giacomo Leopardi’s lyrical poem The infinite 5.
The process of such translation relied heavily on manual labor and human expertise, often time-consuming and subject to human limitations.
With the rapid advancements in artificial intelligence, a new era has dawned upon the world of translation.
Large Language Models (LLMs), such as BLOOM [
Each new translation experiment subsequent to
Pinocchio has been conducted by the community and also by
new groups, university and high school students in
par2. Preliminaries on Emojitaliano: ticular, and sometimes by single translators: everyone
lexicon and grammar was required to use the fixed grammar and, if already
present, the fixed vocabulary, and to reapply the simple
Emojitaliano consists of an iconic-based shared conven- rules for the creation of new vocabulary, starting from
tional code, first of all a simplified grammar of an iso- the semiotic value of the emoji. In this way, Emojitaliano
lating and analytical type, constructed on the model of has been able to benefit from an ever-growing
commu‘interlanguage’; it is anyway not a truly formalized alge- nity, capable of using creativity within a codified scheme
braic language, since it is adopted as a conventional code of rules. Hosted in a specific bot on Telegram
(@emojitalin a social media environment, as a ‘living human lan- ianobot) [
5www.scritturebrevi.org 6https://openai.com/research/gpt-4 7https://www.treccani.it/magazine/lingua_italiana/speciali/ ludolinguistica/Chiusaroli.html
Given the project’s goal of establishing an international
emoji code, we assumed that LLMs can be a useful tool
to speed up translation, as well as to spread the language
beyond the currently active community. For this
purpose, we chose GPT-4 as our reference model, reported
as currently the best general performing model in
language tasks [
Our first experiment aimed at observing the capability of the model in acting as a human translator of Emojitaliano, its capability to apply grammar rules, and its ability to use the already known tags by Unicode Consortium to creatively associate italian words, to memorize and enlarge the lexicon. We developed a prompt where we provided a passage from the first chapter of Pinocchio in Emojitaliano, initially without providing grammar and the lexicon, asking it to derive them, instead. Then we asked the model:
2. to translate on its own some sentences by apply
ing the derived rules; 3. to correct and refine the rules by comparing the personal version and the oficial one. 8Developed and updated by Federico Sangati: https://fede.sangati. me/project/emojitaliano/ 9On July 30, 2023 Figure 3: The role of the delimiters, automatically derived by 10https://chat.lmsys.org/?arena GPT-4. Best seen zoomed in the PDF version. 11The first answer to “What is Emojitaliano?” was generic and incorrect, that is: “Emojitaliano is a term used to describe the use of Italian words and phrases in combination with emojis to create a unique form of communication. It is a type of digital language In Fig. 1, we report the sentences from the first chapter that blends written words with visual symbols, allowing people that were given to the model, with the subsequent request to express emotions, thoughts, and ideas in a more dynamic and to derive the language rules. The output is reported in cpreeoaptlieveinwIatayl.yEamndojhitaaslibanecooims epaarctiocmulmarolyn pfeoaptuurlaeroafmsoocniaglymoeudniga Fig. 2. The interpretation of conceptual bracket signs in and messaging platforms. It has also led to the development of Emojitaliano was then requested, as considered a relevant new emojis that reflect Italian culture and traditions, such as the information missing in the first answer. The result is pizza, the espresso, and the gondola. The use of Emojitaliano is particularly interesting since a metalinguistic label is not limited to Italy, however, and it has spread to other countries assigned by the model, as can be seen in Fig. 3 where Italian is spoken or appreciated. It is an interesting example of how digital communication is transforming language and culture, and how people are adapting to new forms of expression in the digital age.”
sentences and lastly the whole first chapter of Pinocchio.
The first sentence translation is shown in Fig. 4. The most relevant considerations are that the model correctly applies the grammatical rules to translate a sentence for the first time. Notably, we can observe that it applies the principle of semantic reduction to the lexicon, e.g. in assigning the emoji ‘soon’ the meaning of ‘appena’ (‘as soon as’), based on the the already available pairings ‘subito’ (‘now’) and ‘quando’ (‘when’).
Having no access to the registered Emojitaliano glossary, the model succeeds in the translation by applying the common principles of semantic and rhetorical relations in lexicon for the choice of word vs. emoji pairings.
Interesting choices include the use of the ‘wheel’ symbol Figure 6: Further sentences and subsequent derivations of for the iteration verb ‘riottenere’ (‘to get back’) and the new rules. Best seen zoomed in the PDF version. use of the baby for ‘balbettare’ (‘stuttering’), together with the ‘speaker’ emoji for the speaking action.
The experiment continues by providing the oficial to realize the translation of chapters 1, 2, 3 of the text. version as a correction, and the request to derive the rules, The 1st chapter is reported in Fig. 8. One notable miswhich has the output shown in Fig. 5. The test involves take is that, at the end of the learning process, the model two further sentences and the subsequent derivation of appears to ignore the rule of the explicit subject in basic the rules, as reported in Fig. 6. The figurative expression sentences. An important rule of the Emojitaliano is that ‘gli era entrato addosso una gran paura’ (‘a great fear the subject is always to be expressed, as the verb does had come upon him’) is not simplified in ‘to get scared’, not have inflection. The personal pronoun appears in as it should be, but the literal meaning of ‘entrare’ as imperative sentences, instead. With regard to vocabu‘to come in’ is rendered through the ‘shoe’ emoji. As lary, the present model makes use of the emoji repertoire a relevant fact, in the derivation of the rule, the model updated to 2022: this implies the availability of somedoes not catch the meaning of the first point (the three times more appropriate choices compared to the past. emojis with the clip in Fig. 7) as ‘to be with’, since in The emoji catalog includes, for example, the ‘tree trunks’ Emojitaliano the ‘clip’ emoji is ‘with’ and ‘to have’ is ‘to for ‘legno, tronchi’, the ‘machine workshop’ for ‘bottega’, be + with’. which appears particularly suitable compared to the ofi
At the final stage of the test the whole set of the Emo- cial pairings for ‘falegname’ as ‘mechanic’ and the ‘robot’ jitaliano grammatical rules is provided, with the request for ‘puppet’. Among the right choices is the ‘volleyball player’ emoji for ‘tirare un colpo’. Although the model is familiar with the rule for the semantic plain translation of figurative language, it does not succeed in applying FBiegsutrseee8n: Tzohoem1setdchinapthteerPoDfPFinvoecrcshioion.translated by the model. it, as in ‘occhi fuori dal capo’, which is translated literally ‘eyes going out of the head’ instead of rendering the meaning of ‘to be shocked’ or ‘to be upset’.
A relevant issue emerged when Chat-GPT was asked formed a more extensive evaluation by building a dataset
to translate some specific words: for example, ‘legno’ of text pairs Italian-Emojitaliano and asking the model
(‘pezzo di legno’, ‘piece of wood’) is translated with for the translation.
the ‘fir’ (’abete’) emoji and, somewhere later, with the We constructed the dataset by considering the first 3
‘wooden door’ emoji. This is against one of the main chapters of Pinocchio [
We also noticed that grammar and rules mistakes can Italian text and the relative human translation. Each
senbe corrected by the model upon casually reminding rules tence is given to the model for translation independently
in long interactions. The model leaned to progressively from the others.
forget the rules and, thus, a restart of the session was To perform the evaluation, we constructed a textual
required after a few sentences. We believe that this is prompt where the grammar and the basic rules are
caredue to the limited window of attention of LLMs and the fully explained in Italian, where we include as the
trainencoding of emoji that require several tokens for each of ing set, the first chapter as given examples of
translathem. tion. The remaining two chapters are used as the test set.
Measuring quantitatively the quality of the translation
is more challenging than the typical translation tasks
4. Performance evaluation metrics, given the creative use of emojis and their
combinations in expressing a meaning. Given the low number
According to our preliminary exploration, we established of samples, we resorted to human evaluation and the
that GPT-4 is able to derive the semantic rules and trans- use of GPT-4 as a judge following [
Emojitaliano was born thanks to the free dedication and
commitment of an enthusiast devoted Twitter social
community, then also of student groups, willing to share the
goal of building an emoji-based artificial language model,
to be used as a communicative code across language
barriers [