In this paper, we examine the competition between pairs of adjectives in Italian that are antonyms of the same term: one is a “morphological antonym” formed by negative prefixation, the other is a “lexical antonym” with no morphological relationship with the term in question. We consider pairs of adjectives that are reported as antonyms in lexicographic resources and extract the nouns that can be modified by both adjectives from a large corpus. We select a set of 8 nouns for each pair that present higher, lower, and comparable frequencies combined with each antonym respectively and then we perform two experiments with a LLM. Firstly, we perform experiments for masked-token prediction of the adjective, to study the correlation between prediction accuracy and the frequency of the noun-antonym pair. Secondly, we perform a polarity-flip experiment with a multilingual LLM, asking to change the adjective into its positive counterpart. Our results point to the conclusion that the lexical antonym seems to have a narrower lexical coverage and scope than the morphological antonym.
CEUR ceur-ws.org A
with opposite meanings. In their canonical form, two
antonyms’ meanings can be represented as the poles of a
semantic continuum [
In Italian, given a word (e.g., the adjective felice ‘happy’), antonyms can either be realized via prefixation of that word (e.g., infelice ‘unhappy’) or through an independent lexeme (e.g., triste ‘sad’). In our work, we refer to these types of antonyms as morphological antonym and lexical antonym, respectively. A word in the lexicon may have both a morphological and a lexical antonym, only one of them, or neither. In this paper, we are interested in triplets of adjectives where a positive adjective (e.g., felice) presents two possible antonyms (or “co-antonyms”), one formed morphologically by prefixation (e.g., infelice), one morphologically independent 3). (e.g., triste) (Figure 1). felice infelice triste lexical, for the same word. lexical antonyms; explicitly in the case of morphological antonyms, by adding a prefix with a negative, contradicting value. Considering their diferent morphological structure, one possible hypothesis on their lexical competition is that the morphological antonym has a more restricted semantics, representing the negation of the semantics of its adjectival base, while the lexical antonym has a broader semantic coverage, as it is morphologically independent from its positive counterpart (see Section
In this paper, we are interested in studying the factors literature on antonyms (Section 2), there is no empirical that govern the selection of the morphological antonym vs. the lexical one. These two types of antonyms express in-depth study that investigates the competition between morphological and lexical antonyms in single languages, “negative” semantics with respect to the opposite, “pos- including Italian. Studies on antonyms do identify the itive” term in diferent ways:
implicitly in the case of two types of antonyms but generally do not address the factors influencing the preference for one type over the other intralinguistically.
Attribution 4.0 International (CC BY 4.0).
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License in a masked-token prediction task (Section 5.2); and, finally, by performing a substitutability task within the only one of the antonyms possesses, preventing their full same context by switching the polarity of the context sen- synonymy and interchangeability. tence with a SOTA multilingual LLM, in order to study Justeson and Katz [12] take a diferent approach. Using when the adjective is switched to the positive un-prefixed the Brown Corpus and Deese’s antonym dataset, they adjective or to another, positive but morphologically un- were among the first to study antonymy based on a correlated one (Section 5.3). pus. They found that antonymic terms co-occur more frequently than expected, confirming a syntagmatic relationship between them (in addition to the paradigmatic 2. Background one). This syntagmatic relationship was confirmed by a more extensive work carried out in subsequent years: Whereas the exploration of the competition between mor- Jones [11] collected 56 antonym pairs, analyzing jourphological and lexical antonyms, addressed in this paper, nalistic texts, to identify eight discursive functions of has not gained much attention so far, the literature on antonym co-occurrence. antonymy in general is abundant, especially in relation A study that does address the competition between to the English language. forms is Aina et al. [13], who studied syntactically
A term’s antonym is related to it according to three negated adjectives and morphological antonyms (e.g., main characteristics: polarity, gradability and canon- not happy vs unhappy, respectively). Using distributional icity. The first two characteristics refer to the position- semantics they found that a syntactically negated ading of the two antonymic terms w.r.t. the two poles jective is more similar to the positive adjective than to (polarity) of a graded (gradability) scale [3], along its lexical antonym. Additionally, they show that the which free positions can be occupied by other similar morphological antonym is less similar to its lexical base but diferently graded terms. The scale is based on a spe- compared to the similarity between a negated adjective cific property that the two terms share. For example, in and its non-negated counterpart. the pair long-short, the two antonyms share the property Last but not least, a very recent typological study [14] of “length”, defining the start and end of an axis whose examines 37 antonym pairs across 55 languages, focusing poles are defined by two terms, with long representing on antonym formation. When a derived form is attested, the “unmarked” base term of the opposition (this is why it typically applies to the member of the pair with lower we ask for “how long” rather than “how short” something valence or lesser magnitude. Antonyms related to core would last [4]). property concepts (dimension, age, value, color) tend to
However, there are cases where antonymic pairs be expressed through distinct lexical forms (resulting in are formed with potentially competiting antonyms, lexical antonyms), while those related to peripheral proplike friendly-unfrendly vs. friendly-hostile: friendly- erty concepts (physical property, human property, speed) unfriendly is placed on a scale that defines a greater or are generally encoded using derived forms (namely morlesser degree (gradability) of a property, while friendly- phological antonyms). Although the study is insightful hostile are certainly in opposition but belong to two scales and inspiring in a number of ways, the specific question of incompatible properties. In terms of their gradability, of which reasons underlie the preference for lexical or therefore, it seems that the morphological antonym is morphological forms in a single language remains unan“more gradable” than the lexical one. swered.
Canonicity, according to Paradis and Willners [5], defines two semantically related and conventionalized terms as a pair in the language. It is a gradual property 3. Morphological vs. Lexical and can be possessed to a greater or lesser extent. A high degree of canonicity translates into a high degree Antonyms of semantic-lexical embedding in memory and leads to A morphological antonym (e.g., inattivo ‘inactive’ conventionalization in usage. from attivo ‘active’) is immediately recognizable as a
Psycholinguistic studies also suggest that canonical negative term due to the presence of the negative prefix antonyms derive from the speakers’ experience with the un- ‘in-/un-’, while the nature of the opposition is less language: the two terms are inseparable, one elicits the immediate with a lexical antonym (e.g., passivo ‘pasother [6, 7, 8, 9, 10]. When a term has two structurally sive’), because one has to identify the property shared by diferent but semantically similar antonyms (Figure 1), the two opposing terms. canonicity is influenced by factors such as learned pref- Moreover, these two ways of forming an antonym erence for specific pairings, the speaker’s familiarity due from the same term create an asymmetric system: while to exposure, and diferent nuances of meaning [ 11]. Out one of the two terms in lexical opposition (e.g., attivo of context, the antonyms may appear equivalent, but ‘active’) has its morphological antonym (e.g., inattivo ‘inwithin context, a specific meaning may be activated that active’), the other term (e.g., passivo ‘passive’) does not a wider range of ways of not being friendly (such as being (e.g., *apassivo, *impassivo, ‘*unpassive’). This imbalance aloof) whereas hostile is fairly specific” (Murphy 2003: is due to the greater emphasis that language places on 202). So, the morphological antonym would be more everything that requires more precise specifications [ 15]. polysemous, while the lexical counterpart would have The situation is not always perfectly equivalent interlin- a narrower scope. guistically: for example, a form like *invero to indicate Given these two competing hypotheses, we aim to the opposite of vero ‘true’ is not attested in Italian, while empirically verify: it is possible in English (untrue).
However, there are cases in both Italian and English • whether the lexical antonym is more frequent where the two competitors have diferent nuances: for than the morphological antonym; example, infelice-triste / unhappy-sad cover diferent con- • whether the morphological antonym is actually texts in that triste and sad convey a stronger emotional less polysemous than the lexical antonym. meaning, while infelice and unhappy encompass certainly In order to do this, we design a set of experiments. We strong but less intense feelings.[16] ifrst select antonym pairs; we calculate their frequency
Regarding distribution in usage, lexical antonyms of co-occurrence with nouns to have a defined context; are predominant for more basic meanings, supporting the then, we perform two tasks: (i) masked-token prediction Principle of Least Efort theorized by Zipf [ 17], which sug- and (ii) polarity flip. gests that we expect the most used concepts to be coded with short and simple words: basic terms therefore tend to have structurally simple antonyms even when more 4. Dataset Construction complex morphological antonyms would be possible. For example, for a pair like alto-basso ‘tall-short’, there is no 4.1. Antonym Pair Selection morphological counterpart that can be associated with either term: neither *inalto ‘*untall’ nor *inbasso ‘*unshort’ For our study, we decided to focus on adjectives, as this exist. These are canonical antonyms referring to basic class is the most suitable for investigating antonymy, language concepts: in cases like this, but not exclusively, given that it includes content words that normally expreference for simpler and more immediate words blocks press qualities. Moreover, adjectives are semantically the potential formation of a morphological antonym. simpler compared to other word classes as they usually
According to Murphy [4], culturally salient concepts describe a single property that may be or may not be
necessitate clear and concise linguistic expressions. For present to a greater or lesser degree ( Jones et al. [
The competition between the two terms of the texts.1 antonym pair, i.e., the situation in which the usage con- Finally, we created our initial dataset by pairing each text of both terms is nearly the same and allowing for a adjective with its corresponding morphological antonym certain degree of substitutability, is still debated. and a possible lexical antonym.
According to one hypothesis, since the morphological The morphological antonym was formed by using one antonym is the “perfect” negation of a specific lexical of the three possible prefixes productively used in Italian base, it should occur in more restricted contexts (i.e., a to create antonyms, namely: dis-, s-, in- [20]2. subset of the contexts of its positive counterpart) and The lexical antonym was chosen randomly among all should therefore have a narrower semantics (cf. [18, possible options, taking into account synonymy with the 19, 15]). So, morphological antonyms would be less morphological antonym and semantic neutrality. This polysemous. On the other hand, the lexical antonym, means that the lexical antonyms were selected to be idenot sharing identical lexical properties with the opposed ally substitutable with the morphological ones in as many term, should occur in broader contexts and thus be more contexts as possible, and roughly possessing the same polysemous.
However, Murphy [4], examining the English triple friendly-unfriendly-hostile, notes that “The two antonyms are hardly equivalent, though, since unfriendly describes
nature of our study. 2The prefix in- is the most productive and the most widespread. It often adapts phonetically to the bases it attaches to, forming the allomorphs im-, ir-, and il- via assimilation [20]. number of senses according to the dictionary (cf. Table 2). 4.3. Lexical Context Definition and The synonymy between the two types of antonyms was Example-Sentence Extraction further confirmed using Il grande dizionario dei Sinonimi e dei Contrari, Zingarelli 2013 [21].
Summing up, the antonym pairs examined are: • infelice ‘unhappy’ - triste ‘sad’ • impreciso ‘imprecise’ - approssimativo ‘approximative’ • scorretto ‘incorrect’- sbagliato ‘wrong’ • imprudente ‘imprudent’ - avventato ‘reckless’ • insuficiente ‘insuficient’ - scarso ‘scant’ resulting in 5 triplets (base, morph_ant, lex_ant).
with nouns in the itTenTen20 corpus, a large web corpus of written Italian, searched through the SketchEngine platform https://www.sketchengine.eu/.
The analysis of the occurrences highlighted that the two types of antonyms display partially diferent collocational preferences (see Appendix A, Table 3).
Overall, we can split the antonymic adjective-noun couples in two groups according to whether the cooccurrence is: Subsequently, for each antonym pair, eight nouns with diferent co-occurrence frequencies were selected. Specifically, we considered both nouns that typically occur with one of the antonyms (e.g., matrimonio infelice), falling within the domain of collocations, and (more generic) nouns whose co-occurrence frequencies with the lexical and the morphological antonym are very similar (e.g., donna infelice and donna triste) (cf. Table 1).
Noun + Adjective matrimonio infelice ‘unhappy marriage’ matrimonio triste ‘sad marriage’ donna infelice ‘unhappy woman’ donna triste ‘sad marriage’
The latter case is especially interesting for our current purposes, as it represents possible ground for “competition”, namely a situation where the context of use is nearly the same and allows for a certain substitutability between the two terms of the antonym pair.4 • (i) polarization towards one of the two ad- After defining the noun list, for each noun we ranjectives: in these cases, we can speak of fairly domly selected 10 sentences containing the noun folstable distributional preferences, falling within lowed by the morphological antonym and ten sentences the realm of collocations or idiomaticity (e.g., mat- containing the same noun followed by the corresponding rimonio preferably selecting infelice rather than lexical antonym from itTenTen20. This was done for all triste); eight nouns and for all five antonym pairs, resulting in a • (ii) similar with the two adjectives, indicating 800-sentence dataset.
potential substitutability of the two antonyms in the same contexts (e.g., donna selecting both infe- 5. Experiments lice and triste with similar relative frequencies).
Both groups are relevant to explore the context of use of the two types of antonyms, although, for our current purposes, we specifically targeted the nouns in group (ii), namely nouns that occur with both adjectives, suggesting a certain degree of competition between the two antonyms: see sentence 1, where infelice ‘unhappy’ can be replaced by the lexical antonym triste ‘sad’.
1. Un ritratto preciso ma discontinuo che ci restituisce l’immagine di una donna infelice, umiliata, afranta, ma non distrutta, non arresa alla sorte 3 3‘A precise but fragmented portrait that gives us the image of an unhappy, humiliated, distraught woman, but not destroyed, not resigned to fate’.
In Section 3 we outlined two possible hypotheses regarding the competition between the two terms of the antonym pair and we selected the following as a working hypothesis: the morphological antonym, being formed by a negative prefix applied to a specific lexical base, would have more restricted usage contexts (possibly a subset of the contexts of the lexical base), and therefore be less polysemous than the lexical one; on the other hand, the lexical antonym, not sharing morphological structure with the opposing term, would semantically cover some or all of its meanings along with other independent meanings, resulting in broader usage contexts and greater polysemy.
To verify this hypothesis, we performed 2 sets of experiments: (i) masked-token prediction, to estimate the 4For a detailed view of the selected adjectives and nouns, as well as their co-occurrence frequencies, see Appendix A. probability of occurrence of one antonym or the other according to a native Italian language model; (ii) polarity lfip , to transform the collected sentences from a negative meaning to a positive meaning.
We used bert-base-italian-xxl-cased language model5 to perform a token prediction task by masking the antonym present in each sentence. The model was asked to predict the probability of occurrence between the two possible antonyms; then, we took the alternative with the highest probability according to the model as the model prediction.
Our analysis started with the identification of adjectives and their possible antonyms, which, as mentioned (Sec- 5.3. Polarity-flip tion 4.1), have been chosen on the basis of their possible substitutability within the same context. For this reason, In this task, we asked a SOTA LLM, GPT-4o, to transform ifrst of all, the various dictionary definitions of antonyms the sentences extracted from the dataset, both those conwere taken into consideration. We counted how many taining the morphological antonym and those containing senses are associated to each antonym in the Zingarelli the lexical antonym, into positive sentences. dictionary [22], taking the number of senses reported as We used the same prompt for all antonym pairs, a first proxy of semantic broadness. parametrising the antonyms and sentences presented,
As a second proxy, the semantic coverage of each by asking the model to flip the sentence from a negative antonym was taken into account. We conducted an anal- sense to a positive one, always by changing the adjective ysis of the lexical variety of each group’s context in the accompanying the target noun.6 selected sentences, by calculating the token/type ratio We then fetched the new adjective generated by the for each group. We report the results in Table 4. As can model, and calculated when the new, positive adjective be seen, no relevant diferences were found according to coincided with the lexical base, and when not. these features. The rationale behind this is that the senses of an antonym can be separated through the various positive 5.2. Masked-token Prediction terms with which it can be changed. We expected that sentences containing the morphological antonym would According to our hypothesis, in this task we expect that be turned into positive using their lexical base more ofthe predicted antonym will have a higher prediction ac- ten than their lexical counterparts, indicating a narrower curacy in the sentences with the highest occurrence of semantics. the adjective with the selected words (represented by high relative frequency). In contexts with similar frequencies, we expect that accuracy should be similar for 6. Analysis and Results both antonyms, showing a genuine competition between the two, as the language model should not have a specific As regards the number of meanings listed in the dictiopreference. nary for the two terms of the antonym pair (Table 2),
We previously said that the words that form the these are almost the same, indicating that the recognized antonym pairs can be considered synonymous. In fact, senses of each antonym alone are not decisive to defull synonyms are rather rare (Murphy [4], among oth- termine the selection between one or the other. As for ers), also because languages tend to avoid synonymy by lexical variability (see Appendix B, Table 4), token/type diferentiation in terms of meaning or distribution [ 23]. ratio also fails to reveal a significant diference between Therefore, the two terms of the pair are better regarded as morphological and lexical antonyms; in only three cases near-synonyms, meaning that one term can cover almost does the token/type ratio of lexical antonyms exceed all the meanings of the other but not all of them. that of morphological antonyms. This seems to indicate
To evaluate the factors that lead to the choice of one that factors other than contextual variability underlie the antonym over the other, we decided to observe how a na- preference for one or the other. tive Italian language model pre-trained for masked-token Let us now consider Table 3, Appendix A. In addition to prediction model behaves in terms of the probability of the co-occurrences of nouns with the two antonyms, the occurrence of an antonym in a given context. In this accuracy for the two tasks for each noun-adjective pair is respect, see Niwa et al.[24], who used BERT to predict also provided (further divided into noun-morphological antonyms in specific contexts: experiments on Japanese antonym and noun-lexical antonym). The distribution slogans showed a top-1 accuracy of 29.3% and a top-10 of nouns in this table follows a specific order: nouns at accuracy of 53.8%, with human evaluations confirming the extremes exhibit occurrence frequencies polarized that over 85% of predicted antonyms were appropriate, 5dbmdz/bert-base-italian-xxl-uncased demonstrating the method’s efectiveness in capturing 6For detailed information on the prompt used in this study, please contextually relevant antonyms. refer to the Appendix B.
Couple infelice-triste impreciso-approssimativo scorretto-sbagliato imprudente-avventato insuficiente-scarso senses polarity
flip
markable given the potential purely morphological (rather than semantic) bias due to the derivational relatedness of the morphological antonym, that could be predicted to favour its replacement by the target positive adjective to some degree. towards either the lexical antonym or the morpholog- range of senses (see, e.g., Murphy [4]). ical antonym. Some nouns form with the adjective a We believe that these results, that contradict our inifairly stable collocation, while other nouns form freer ex- tial hypothesis, open up new avenues for future research pressions. For the purposes of this study, particularly in in this area, despite the limitations of the present study, relation to the analysis of competition, the central nouns which has an exploratory nature and a narrow empiriwith similar frequencies are of greater importance. cal coverage. Indeed, only 5 adjectives were analyzed,
Upon examining the occurrences and accuracy, we exclusively belonging to core vocabulary. Another shortobserve that the values are comparable. coming is that, unlike for English, there are no in-depth
As for masked-token prediction, the consistent higher studies on antonyms in Italian. However, we want to values of the lexical antonym indicate higher predictabil- stress the importance of conducting studies on languages ity and/or higher degree of idiomaticity, which contra- other than English to avoid the well-known Anglocentric dicts our working hypothesis that lexical antonyms dis- bias. play a broader semantic coverage. Hopefully, our results will be challenged by further
Finally, as for the polarity flip, consistently with the studies in the future, which might even overturn our conmasked-token experiments, the sentences containing the clusions entirely, if a larger data set were considered. Furmorphological antonym were turned into positive using thermore, it would be interesting to investigate whether the lexical base fewer times than their lexical counter- the results obtained for Italian are also found for other parts, suggesting that the latter may have a more re- languages that present both lexical and morphological stricted semantic spectrum, contrary to our initial hy- antonyms, including languages with a diferent morphopothesis.7 logical system. With a view of deepening the analysis methodologically, it would be interesting to focus on additional linguistic factors that might drive the choice 7. Conclusion and Future Work between lexical and morphological antonyms, such as semantic networks or word frequency, and to expand the testing to the psycholinguistic dimension.
What is sure is that the relationship between morphological and lexical antonyms is more complex than previously thought and that the choice of one type of antonym over another depends on a variety of interconnected factors that are still to be fully unveiled.
Our study investigated the diferences and competition between two types of antonyms, morphological and lexical, focusing on a computational account of their context of use. While a lexical analysis did not prove decisive, experiments on masked-token prediction and polarity lfip, aimed at approximating their semantic coverage, indicate that, unlike what is suggested in some studies on antonymy, the lexical antonym seems to have a narrower lexical coverage and scope, supporting the view that it is actually the morphological antonym, despite its closer relationship with the lexical base, that displays a wider [3] E. Sapir, Grading, a study in semantics., Philosophy [20] C. Iacobini, Prefissazione, in: M. G. e F. Rainer (Ed.), of Science 11(2) (1944) 93–116. La formazione delle parole in italiano, Tübingen: [4] M. L. Murphy, Semantic relations and the lexicon, Niemeyer, 2004, pp. 97–163.
Cambridge University Press, 2003. [21] G. Pittàno, Il grande dizionario dei Sinonimi e dei [5] C. Paradis, C. Willners, Antonymy and negation - Contrari, Zanichelli, 2013.
the boundedness hypothesis, Journal of Pragmatics [22] N. Zingarelli, Zingarelli 2024. Vocabolario della lin38 (2006) 1051–1080. gua italiana, Zanichelli, 2023. [6] D. S. Palermo, J. J. Jenkins, Word association norms: [23] M. Aronof, Competition and the lexicon, in: E. Angrade school through college, Minneapolis: Univer- nibale, C. Iacobini, M. Voghera (Eds.), Livelli di analsity of Minnesota Press, 1964. isi e fenomeni di interfaccia, Roma: Bulzoni, 2016. [7] J. Deese, The structure of associations in language [24] A. Niwa, K. Nishiguchi, N. Okazaki, Predicting and thought, Baltimore: Johns Hopkins University antonyms in context using BERT, in: A. Belz, Press, 1965. A. Fan, E. Reiter, Y. Sripada (Eds.), Proceedings [8] W. G. Charles, G. A. Miller, Contexts of antonymous of the 14th International Conference on Natural adjectives, Applied Psycholinguistics 10 (1989) Language Generation, Association for Computa357–375. tional Linguistics, Aberdeen, Scotland, UK, 2021, pp. [9] D. J. Hermann, G. Conti, D. Peters, P. H. Robbins, 48–54. URL: https://aclanthology.org/2021.inlg-1.6.
R. J. S. Chafin, Comprehension of antonymy and doi:10.18653/v1/2021.inlg- 1.6. the generality of categorization models, Journal of Experimental Psychology: Human Learning and
Memory 5 (1979) 585–597. [10] D. Gross, U. Fischer, G. A. Miller, The organization of adjectival meanings, Journal of Memory and
Language 28 (1989) 92–106. [11] S. Jones, Antonymy: a corpus-based approach,
Routledge, 2002. [12] J. S. Justeson, S. M. Katz, Co-occurrences of antonymous adjectives and their contexts, Computational Linguistics 17 (1991) 1–20. URL: https://aclantholo gy.org/J91-1001. [13] L. Aina, R. Bernardi, R. Fernández, Negated adjectives and antonyms in distributional semantics: not similar, in: Proceedings of the Fifth Italian Conference on Computational Linguistics, IJCoL, 2019, pp. 57–71. URL: http://journals.openedition.org/ijcol/4 57. doi:https://doi.org/10.4000/ijcol.457. [14] M. Koptjevskaja-Tamm, M. Miestamo, C. Börstell,
A cross-linguistic study of lexical and derived antonymy, Linguistics (2024). URL: https://doi. org/10.1515/ling-2023-0140. doi:doi:10.1515/li ng- 2023- 0140. [15] F. Vicario, Note sull’ordine degli elementi in coppie
di verbi antonimi., Linguistica XLIII 43 (2003) 3–12. [16] V. Muehleisen, Antonymy and Semantic range in English, Ph.D. thesis, Northwestern University,
Evanston, 1997. [17] G. K. Zipf, Human behavior and the principle of
least efort, Cambridge Addison-Wesley, 1949. [18] R. von Jhering, Der Zweck im Recht, Leipzig: Bre
itkopf and Hӓrtel, 1883. [19] K. E. Zimmer, Afixal negation in english and other languages: an investigation of restricted productivity, Word. Journal of the Linguistic Circle of New York 20 (1964).
infelice
lexical antonym
triste approssimativo sbagliato avventato scarso system_message = '''In una frase l'aggettivo originale è stato sostituito da un token [MASK]. Tu devi riscrivere la frase facendo minimi cambiamenti e sostituire l'aggettivo mascherato con un altro aggettivo, in modo che la frase risulti volta al positivo.
Il tuo output deve essere SOLO un json nel seguente formato e con i seguenti campi: {"new_sentence": "<tua nuova frase>", "new_adj": "<l'aggettivo con cui hai sostituito [MASK] nella nuova frase>"}''' user_message = f'''Frase originale: "{masked_sent}" (aggettivo originale: {agg})''' {system_message = f'''In a sentence, the original adjective has been replaced by a [MASK] token. You need to rewrite the sentence making minimal changes and replace the masked adjective with another adjective, so that the sentence is positively oriented.
Your output must be ONLY a json in the following format and with the following fields:\n''' + '''{"new\_sentence": "< your new sentence>", "new\_adj": "<the adjective with which you replaced [MASK] in the new sentence"}''' user\_message = f'''Original sentence: "{masked\_sent}" (original adjective: {agg})'''}