<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Dec</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Is It Really That Simple? Prompting Language Models for Automatic Text Simplification in Italian</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Debora Nozza</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Giuseppe Attanasio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Bocconi University</institution>
          ,
          <addr-line>Milan</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2023</year>
      </pub-date>
      <volume>02</volume>
      <issue>2023</issue>
      <abstract>
        <p>Recent language models (LMs) that follow instructions have showcased remarkable abilities to tackle diverse natural language processing (NLP) tasks, given appropriate prompts. However, the potential of these models for Automatic Text Simplification (ATS) in Italian remains largely unexplored. In this paper, we pioneer the first in-depth investigation into the capabilities of LMs for performing ATS in Italian. We evaluate six state-of-the-art models on a benchmark Italian ATS dataset of administrative texts, reporting six readability metrics on the generated text. Our findings demonstrate a large variability across models, scales, and prompts. Among the tested models, GPT-3.5 editing capabilities are the most suitable, outperforming, surprisingly, human-written simplification. Furthermore, we shed light on the enigmatic multilingual capabilities of instruction following models, opening up new avenues for research in this domain.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Automatic Text Simplification</kwd>
        <kwd>Large Language Models</kwd>
        <kwd>Italian</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        els, e.g., T5 [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] or LLaMA [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], can be further improved via
instruction fine-tuning (IFT) and reinforcement learning
Italian administrative texts have long been criticized for from human feedback (RLHF) [6, 7, 8, inter alia]. The
retheir complexity, described as “artificial” and “obscure” sulting model can follow instructions as expressed via
nat[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Despite eforts by Italian institutions to encourage ural language, i.e., it can solve many NLP tasks and reply
the use of plain language in oficial acts and communica- to various user requests with no architectural changes.
tions over the past decades [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], the readability of these This paper presents the first investigation to look into
texts remains a pressing issue [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. To tackle this chal- the capabilities of instruction following language models
lenge, considering the substantial volume of bureaucratic for Automatic Text Simplification on Italian
administratext generated, a logical approach is to embark on the tive texts. We rely on Admin-It [9], a benchmark parallel
analysis and exploration of Automatic Text Simplifica- corpus in the Italian administrative language that
contion (ATS) methods. Automated text simplification is a tains sentences that have been simplified using three
natural language processing (NLP) technique that aims distinct rewriting techniques. We perform a thorough
to modify complex or dificult-to-understand text into evaluation of six models based on six diferent readability
simpler and more accessible language while retaining the measures tailored for Italian. Each model is compared
original meaning. The goal is to make the content easier to the readability scores of the original administrative
to comprehend for a wider audience, including individ- text and the simplified version provided in the parallel
uals with cognitive or reading dificulties, non-native corpus.
speakers, or those with limited literacy skills.
      </p>
      <p>Using recent large-scale language models (LMs) is a Contribution We propose the first in-depth study on
promising direction in this context. In particular, recent whether current IFT models can simplify written
pasevidence has shown that high-capacity pretrained mod- sages in Italian. We report a large variability across
models, with proprietary GPT-3.5 being the most suitable
solution. In addition, we introduce a novel metric to
better account for accurate and simple generations. We</p>
      <p>[12]. Sentences were manually simplified both at
lexical and syntactic levels.
• Rewritten Docs (Admin-It): 48 pairs of
sentences (∼ 7% of the total dataset) from
administrative documents collected and simplified by
Cortelazzo [13]. Sentences were rewritten according
to linguistic simplification and communicative
efectiveness criteria.</p>
      <sec id="sec-1-1">
        <title>In this paper, we refer to the entire corpus acquired by combining these three subsets as Admin-It.</title>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>3. Models</title>
      <p>release code and data to facilitate future research.1</p>
    </sec>
    <sec id="sec-3">
      <title>2. Automatic Text Simplification</title>
      <sec id="sec-3-1">
        <title>Automatic text simplification is a research field in com</title>
        <p>putational linguistics that studies methods and
techniques to simplify textual content [10]. This task
involves transforming complex or dificult-to-understand
text into more straightforward and accessible language.
Automatic text simplification has been viewed as a
critical technique for increasing the inclusion of people with
special needs and boosting social inclusion [10].</p>
        <p>To simplify a text, strategies might involve
sentenceor word-level interventions (e.g., breaking down longer
passages into multiple sentences or changing less
common words with easier equivalents). Most importantly,
such edits can be learned, and NLP models can be applied
to automate and generalize.</p>
        <p>Automatic text simplification has typically focused on
two distinct tasks: lexical simplification and syntactic
simplification, each of which addresses a diferent
subproblem in the larger task of making texts easier to read
and understand [10]. The goal of lexical simplification is
to make a document easier to understand by either
changing the vocabulary to use terms that are more likely to be
familiar to the reader or by providing clearer definitions
of unfamiliar words. Whereas the purpose of syntactic
simplification is to detect syntactic phenomena in phrases
that may obstruct readability and understanding, with
the hope of rewriting the sentence in a way that makes it
easier to read and comprehend (by, for example, changing
it from the passive to the active voice).</p>
      </sec>
      <sec id="sec-3-2">
        <title>FLAN models are fine-tuned on a large collection of</title>
        <p>NLP tasks verbalized to natural language [14]. The
verbalization follows a task-dependent template–e.g.,
“Translate the following sentence from</p>
        <sec id="sec-3-2-1">
          <title>Model</title>
          <p>Flan-T5-XXL
Vicuna v1.3
Camoscio
Guanaco
Llama 2 Chat*
GPT-3.5*</p>
        </sec>
      </sec>
      <sec id="sec-3-3">
        <title>1Code and data available at: https://github.com/MilaNLProc/</title>
        <p>prompting-italian-text-simplification</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Zero-Shot Simplification in</title>
    </sec>
    <sec id="sec-5">
      <title>Italian</title>
      <sec id="sec-5-1">
        <title>As a result of multilingual pretraining, fine-tuning, or</title>
        <p>RLHF data, IFT models have shown multilingual
abilities, such as solving cross-lingual tasks (e.g., machine
translation), or understanding and providing coherent
responses to non-English input queries [20].</p>
        <p>We leverage this finding and prompt models to run
text simplification in Italian in a zero-shot setup.
Specifically, we compile a request for simplification using a
given prompt template, feed it to the model, and take
the model response unmodified. For Vicuna, Guanaco,
Llama 2 Chat, and GPT-3.5 we use model-specific system
message templates (see Appendix A). We specify no
system message or use any prompt template for Flan-T5 and
Camoscio.</p>
        <p>Figure 1 displays a system overview.</p>
        <sec id="sec-5-1-1">
          <title>Hyperparameter</title>
        </sec>
        <sec id="sec-5-1-2">
          <title>Value</title>
          <p>Temperature
Top P
Top K
Repetition Penalty
Penalty Alpha
Length Penalty
Max new tokens
{src_lang} to {tgt_lang}: {src_text}” is one
of the template used for machine translation.</p>
          <p>Although FLAN does not include specifically tasks
related to language simplification, we hypothesize that 1)
pretraining data, 2) the presence of tasks that share some
of the traits (e.g., summarization), and 3) scale enable
models to simplify language. We experiment wtih
FlanT5-XXL (11B), the largest T5-based FLAN model.</p>
        </sec>
      </sec>
      <sec id="sec-5-2">
        <title>Prompt Template Recent evidence has shown that</title>
        <p>diferent prompts elicit multilingual capabilities
diferently [21]. Therefore, we experiment with two templates,
both starting with a prefix stating the task followed by
3.2. LLaMA IFT Models the passage to simplify.</p>
        <p>
          In our explicit template (Template-EN), we state overtly
Since LLaMA [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] established as the best performing pre- the response should be in Italian, i.e., “Simplify the
trained base model on many language understanding following text. Write only the response,
tasks, several works used it as base model for IFT. and in Italian.\n{src_text}”, where src_text
        </p>
        <p>We test Vicuna v1.3 7B, 13B, and 33B [15]. These mod- is the passage to simplify. We also experiment with
els have been trained with IFT on a corpus of around 70K an implicit template (Template-IT), where the entire
conversations from the ShareGPT website.2 We test also prompt is written in Italian to hint models to reply
Guanaco (65B) [16], an IFT model fine-tuned on around in the same language: “Semplifica il testo
10K conversations from the Open Assistant project.3 seguente.\n{src_text}” (eng: “Simplify the
follow</p>
        <p>As an additional baseline for the Italian, we include ing text.”).</p>
        <p>Camoscio [17], a LLaMA model instruction fine-tuned
on samples exclusively in Italian. The fine-tuning corpus
includes around 52K instructions from the Alpaca dataset
[18] machine-translated with GPT-3.5.</p>
        <sec id="sec-5-2-1">
          <title>3.3. RLHF Models</title>
        </sec>
      </sec>
      <sec id="sec-5-3">
        <title>Reinforcement learning from human feedback (RLHF)</title>
        <p>introduces an additional step to the standard IFT pipeline.
After the supervised fine-tuning stage, a policy learning
step maximizes the alignment with human preferences by
teaching the model to produce responses that are more
likely to be preferred by human users [19].</p>
        <p>We experiment with GPT-3.5 [7, gpt-3.5-turbo, last
accessed June 15, 2023] and Llama 2 Chat (70B) [8].</p>
      </sec>
      <sec id="sec-5-4">
        <title>2https://sharegpt.com/ 3https://open-assistant.io/</title>
      </sec>
      <sec id="sec-5-5">
        <title>Decoding Setup We use a standard decoding configu</title>
        <p>ration, loosely inspired by Vicuna’s Chat Arena4 for all
the models. Table 2 reports the generation configuration
used. We use models and code as released in
HuggingFace transformers [22] and simple-generation [23] to run
inference.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>5. Metrics</title>
      <sec id="sec-6-1">
        <title>We conducted an evaluation of automatic text simplifi</title>
        <p>cations using benchmark readability metrics, which we
categorize into traditional, namely Flesh-Kincaid test and
GulpEase index, and READ-IT-based metrics.</p>
        <p>The Flesh-Kincaid test (↑) [24] is a widely utilized
measure in education for assessing the readability level
of books. In this context, we refer to the Flesch-Vacca
formula, designed specifically for Italian text.</p>
      </sec>
      <sec id="sec-6-2">
        <title>4https://chat.lmsys.org/?arena</title>
        <p>Admin-It
Eval Metric
READ-IT</p>
        <p>Gulpease
Flesch–Kincaid</p>
        <p>A chat between a curious human and an
artificial intelligence assistant. The
assistant gives helpful, detailed, and polite
answers to the user's questions.
### Human: Simplify the following text. Write
only the response, and in Italian.</p>
        <p>Il richiedente, che al momento della
presentazione della richiesta ha optato per il
bonifico su conto corrente bancario o postale o
per l’accredito su carta prepagata, riceverà
l’importo totale spettante direttamente
sull’iban indicato nella domanda.
### Assistant:
Il richiedente riceverà l'importo totale
direttamente sull'IBAN indicato nella domanda
se ha scelto il bonifico su conto bancario o
postale o l'accredito su carta prepagata al
momento della presentazione della richiesta.</p>
        <p>Model
Flan-T5-XXL</p>
        <p>Vicuna</p>
        <p>Guanaco
Llama 2 Chat</p>
        <p>Camoscio
GPT-3.5</p>
        <p>The GulpEase index (↑) [25] calculates text readabil- aforementioned variations.
ity based on factors such as word length (measured in
letters), number of words, and sentence length. It does not
have a direct association with any particular language. 6. Results</p>
        <p>READ-IT (↓) [26] is a machine learning-based
readability metric. The model has been trained to evaluate This section illustrates the results of prompting
instructhe readability of a text using various features. Diferent tion following models for generating simplified versions
variations of the READ-IT metric exist: base employs of an input text. We first perform a preliminary
investibasic features like sentence and word length; lexical fo- gation on the generated outputs. Based on this analysis,
cuses on lexical features, such as vocabulary complexity; we discover that the benchmark readability metrics are
syntax considers grammatical features like syntactic tree not ideal in our setup, as models produce non-relevant
depth and part-of-speech categories; all combines all the responses. Therefore, we propose a novel adjusted score
Gulpease
to better measure improved readability and adherence to Table 3 presents the scores for each model, including
the original text. variations in Italian and English prompt templates where
applicable (see Section 4). As observed in the two
exam6.1. Inspecting Generated Responses ples, only two models, Flan-T5-XXL and GPT-3.5,
demonstrate reasonable BERTScore and % Detected Italian
Tables 5 and 6 (Appendix B) illustrate two examples ex- metrics. It is crucial to emphasize the discouragingly low
tracted from the Admin-It dataset. In the first instance, % of Italian generations by Llama 2 Chat and the
unsatthe complex sentence uses administrative jargon related isfactory BERTScore of Camoscio. Additionally, when
to numbers and dates, while the manually-simplified text testing the models with both template configurations,
conveys the same concept using more straightforward the Italian template tends to yield slightly better results.
verbs, e.g., “assumere l’uficio di” ( eng: to get the role of) As a consequence, moving forward, we will consider the
is replaced with “essere” (eng: to be). However, the model- Template-IT model version whenever available.
generated texts exhibit undesired behaviors: the
automatic simplifications are not consistently simpler, some 6.2. Automatic Text Simplification Results
are not written in Italian, some result in drastically longer
passages, incorporate prompt-related content, or occa- Table 4 presents the text readability metrics (see Section
sionally add irrelevant information. The second example 5) for the original (complex) text, its reference
humanpresents a similar case, wherein the model-generated written simplification, and the model generations. Out of
simplification includes code, questions, and apparent er- all models, only three — Camoscio, Guanaco, and
GPTrors likely produced by incorrect translations, e.g., “il 3.5 — consistently exhibit readability metrics better than
bambino deve essere vivo” (eng: the child must be alive). human simplification.</p>
        <p>To investigate the issues raised in our initial qualita- Interestingly, Guanaco yielding the best results in each
tive analysis, we conducted two investigations. First, we individual metric is contradicting our findings from the
calculated the adherence of the model-generated sim- previous section. The issue lies in the fact that the
plifications to the human-written reference simplifica- readability metrics alone do not account for cases
tion provided in Admin-It. This metric helps us identify when models produce unrelated or inaccurate
gencases where the produced simplifications diverge from erations. For instance, the Guanaco generation shown
the source text, potentially containing code or unrelated in Table 6 may be a highly readable sentence
(READquestions. For this evaluation, we used BERTScore, a lan- IT = 96) but has very low adherence to the original
guage generation evaluation metric based on pretrained text (BERTScore = 0.63).</p>
        <p>BERT contextual embeddings [27]. Second, we measured To address this issue, we introduced a novel
READthe percentage of times the model-generated simplifica- IT metric which also takes into account the original
tions are in Italian (% Detected Italian). To accomplish text similarity, named READ-IT_. The
metthis, we used the Python langdetect5 library. We clas- ric is computed as the product among READ-IT and
sified a text as Italian if the library detected the Italian BERTScore. By using READ-IT_, we identify
language with a confidence level higher than 0.99. GPT-3.5 as the best model across the board. This finding
aligns with our qualitative investigation. Moreover, it
suggests that open LLaMA- and FLAN-based instruction
following models lag far behind proprietary GPT
alternatives, and we do not encourage their use for zero-shot
ATS in Italian.</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>7. Related Work</title>
      <sec id="sec-7-1">
        <title>Computational approaches for Automatic Text Simplifica</title>
        <p>tion have been long studied for English, with works
spanning from statistical machine translation-based systems
[28] to supervised recurrent neural networks [29, 30],
graph convolutional neural networks [31], and
Transformer encoders [32]. Acknowledgments</p>
        <p>Similar eforts for the Italian language have seen a
joint development of corpora, ATS models, and evalua- This project has in part received funding from Fondazione
tion metrics. Brunato et al. [33] designed the first parallel Cariplo (grant No. 2020-4288, MONICA) and the
Euresource, collecting two sets of pairs where several sen- ropean Research Council (ERC) under the European
tences are simplified following diferent guidelines and Union’s Horizon 2020 research and innovation program
for diferent target audiences. Other examples are the (No. 949944, INTEGRATOR). The authors are members
PaCCSS-IT [34], SIMPITIKI [11], and Admin-It [9] cor- of the MilaNLP group and the Data and Marketing
Inpora, among others. We focus on the Admin-It corpus, sights Unit of the Bocconi Institute for Data Science and
which covers the particularly verbose and complex ad- Analysis.
ministrative language across diferent types of
simplification edits. ERNESTA [35] is the first documented solution
for Italian ATS, specifically addressing simplification for References
children with low reading skills. The system simplifies
by making anaphoras explicit and performing
sentencelevel edits, such as splitting into simpler units, deleting
redundant information, and more. Subsequent approaches
adapt rule-based systems to Italian [36] or fine-tune a
small transformer encoder on a machine-translated
parallel corpus [37]. Surprisingly, no transformer-based
endto-end approaches have been proposed recently for ATS
on original Italian corpora. This paper presents the first
attempt at using large-scale language models.
for instance, non-factual and non-truthful generations
when prompting language models about world
knowledge [38, 39, 40, inter alia]. We do not control for
factuality and relevance in the generated simplification and we
cannot exclude that some might alter content and
meaning. As we discussed in Section 6.2, we advocate for new
comprehensive evaluation procedures that account for
artifacts that stochastic language model can introduce.</p>
        <p>Moreover, instruction fine-tuned language models are
known to encode social biases and generations might
reflect them [41, 42, 43, inter alia].</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>8. Conclusion</title>
      <sec id="sec-8-1">
        <title>This paper introduced the first extensive study on the abil</title>
        <p>ity of large-scale instruction following models to simplify
Italian administrative sentences. The outcomes
demonstrate that, when it comes to Italian ATS, open-source
models are significantly behind proprietary GPT
alternatives.</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>Limitations and Ethical</title>
    </sec>
    <sec id="sec-10">
      <title>Considerations</title>
      <sec id="sec-10-1">
        <title>The use of modern language models for automatic text</title>
        <p>limitations comes with limitations and risks. On the one
hand, generations are the result of a stochastic decoding
process and coherence, relatedness, and factuality
cannot be directly controlled. Multiple evidence reported,</p>
        <p>A. Ray, et al., Training language models to follow [19] P. F. Christiano, J. Leike, T. Brown, M. Martic,
instructions with human feedback, Advances in S. Legg, D. Amodei, Deep reinforcement
learnNeural Information Processing Systems 35 (2022) ing from human preferences, Advances in neural
27730–27744. information processing systems 30 (2017).
[8] H. Touvron, L. Martin, K. Stone, P. Albert, A. Alma- [20] V. D. Lai, N. T. Ngo, A. P. B. Veyseh, H. Man, F.
Derhairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, noncourt, T. Bui, T. H. Nguyen, Chatgpt beyond
enS. Bhosale, et al., Llama 2: Open foundation and fine- glish: Towards a comprehensive evaluation of large
tuned chat models, arXiv preprint arXiv:2307.09288 language models in multilingual learning, arXiv
(2023). preprint arXiv:2304.05613 (2023).
[9] M. Miliani, S. Auriemma, F. Alva-Manchego, [21] H. Huang, T. Tang, D. Zhang, W. X. Zhao, T. Song,
A. Lenci, Neural readability pairwise ranking for Y. Xia, F. Wei, Not all languages are created
sentences in Italian administrative language, in: equal in llms: Improving multilingual capability by
Proceedings of the 2nd Conference of the Asia- cross-lingual-thought prompting, arXiv preprint
Pacific Chapter of the Association for Computa- arXiv:2305.07004 (2023).
tional Linguistics and the 12th International Joint [22] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C.
DeConference on Natural Language Processing (Vol- langue, A. Moi, P. Cistac, T. Rault, R. Louf, M.
Funume 1: Long Papers), Association for Computa- towicz, J. Davison, S. Shleifer, P. von Platen, C. Ma,
tional Linguistics, Online only, 2022, pp. 849–866. Y. Jernite, J. Plu, C. Xu, T. Le Scao, S.
GugURL: https://aclanthology.org/2022.aacl-main.63. ger, M. Drame, Q. Lhoest, A. Rush,
Transform[10] H. Saggion, Automatic Text Simplification, Synthe- ers: State-of-the-art natural language
processsis Lectures on Human Language Technologies, ing, in: Proceedings of the 2020 Conference on
Morgan &amp; Claypool Publishers, 2017. URL: https: Empirical Methods in Natural Language
Process//doi.org/10.2200/S00700ED1V01Y201602HLT032. ing: System Demonstrations, Association for
Comdoi:10.2200/S00700ED1V01Y201602HLT032. putational Linguistics, Online, 2020, pp. 38–45.
[11] S. Tonelli, A. P. Aprosio, F. Saltori, Simpitiki: a sim- URL: https://aclanthology.org/2020.emnlp-demos.6.
plification corpus for italian., in: CLiC-it/EVALITA, doi:10.18653/v1/2020.emnlp-demos.6.
2016, pp. 4333–4338. [23] G. Attanasio, Simple Generation, https://github.
[12] L. Passaro, A. Lenci, Extracting terms with extra, com/MilaNLProc/simple-generation, 2023.
in: Computerised and corpus-based approaches to [24] V. Franchina, R. Vacca, Adaptation of flesh
readabilphraseology: Monolingual and multilingual per- ity index on a bilingual text written by the same
spectives, Tradulex, 2016, pp. 188–196. author both in italian and english languages,
Lin[13] M. A. Cortelazzo, Semplificazione del linguaggio guaggi 3 (1986) 47–49.</p>
        <p>amministrativo, Quaderni del Comune di Trento [25] P. Lucisano, M. E. Piemontese, Gulpease: una
for(1998). mula per la predizione della leggibilita di testi in
[14] S. Longpre, L. Hou, T. Vu, A. Webson, H. W. Chung, lingua italiana, Scuola e Città 3 (1988) 57–68.</p>
        <p>Y. Tay, D. Zhou, Q. V. Le, B. Zoph, J. Wei, et al., [26] F. Dell’Orletta, S. Montemagni, G. Venturi, READ–
The flan collection: Designing data and methods IT: Assessing readability of Italian texts with a
for efective instruction tuning, arXiv preprint view to text simplification, in: Proceedings of the
arXiv:2301.13688 (2023). Second Workshop on Speech and Language
Pro[15] W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, cessing for Assistive Technologies, Association for
H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Computational Linguistics, Edinburgh, Scotland,
Gonzalez, I. Stoica, E. P. Xing, Vicuna: An open- UK, 2011, pp. 73–83. URL: https://aclanthology.org/
source chatbot impressing gpt-4 with 90%* chat- W11-2308.
gpt quality, 2023. URL: https://lmsys.org/blog/ [27] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger,
2023-03-30-vicuna/. Y. Artzi, Bertscore: Evaluating text generation with
[16] T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettle- BERT, in: 8th International Conference on Learning
moyer, Qlora: Eficient finetuning of quantized Representations, ICLR 2020, Addis Ababa, Ethiopia,
llms, arXiv preprint arXiv:2305.14314 (2023). April 26-30, 2020, OpenReview.net, 2020. URL: https:
[17] A. Santilli, E. Rodolà, Camoscio: An ital- //openreview.net/forum?id=SkeHuCVFDr.
ian instruction-tuned llama, arXiv preprint [28] W. Xu, C. Napoles, E. Pavlick, Q. Chen, C.
CallisonarXiv:2307.16456 (2023). Burch, Optimizing statistical machine translation
[18] R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, for text simplification, Transactions of the
AsC. Guestrin, P. Liang, T. B. Hashimoto, Stanford al- sociation for Computational Linguistics 4 (2016)
paca: An instruction-following llama model, https: 401–415. URL: https://aclanthology.org/Q16-1029.
//github.com/tatsu-lab/stanford_alpaca, 2023. doi:10.1162/tacl_a_00107.
[29] S. Nisioi, S. Štajner, S. P. Ponzetto, L. P. Dinu, Ex- arXiv preprint arXiv:2305.13534 (2023).
ploring neural text simplification models, in: Pro- [39] S. Zheng, J. Huang, K. C.-C. Chang, Why does
ceedings of the 55th Annual Meeting of the Asso- chatgpt fall short in providing truthful answers,
ciation for Computational Linguistics (Volume 2: ArXiv preprint, abs/2304.10513 (2023).
Short Papers), Association for Computational Lin- [40] L. Chen, Y. Deng, Y. Bian, Z. Qin, B. Wu, T.-S. Chua,
guistics, Vancouver, Canada, 2017, pp. 85–91. URL: K.-F. Wong, Beyond factuality: A comprehensive
https://aclanthology.org/P17-2014. doi:10.18653/ evaluation of large language models as knowledge
v1/P17-2014. generators, arXiv preprint arXiv:2310.07289 (2023).
[30] X. Zhang, M. Lapata, Sentence simplification [41] L. Lucy, D. Bamman, Gender and representation
with deep reinforcement learning, arXiv preprint bias in GPT-3 generated stories, in: Proceedings of
arXiv:1703.10931 (2017). the Third Workshop on Narrative Understanding,
[31] O. M. Cumbicus-Pineda, I. Gonzalez-Dios, A. Soroa, Association for Computational Linguistics, Virtual,
A syntax-aware edit-based system for text simplifi- 2021, pp. 48–55. URL: https://aclanthology.org/2021.
cation, in: Proceedings of the international confer- nuse-1.5. doi:10.18653/v1/2021.nuse-1.5.
ence on recent advances in natural language pro- [42] M. Cheng, E. Durmus, D. Jurafsky, Marked
percessing (RANLP 2021), 2021, pp. 324–334. sonas: Using natural language prompts to
mea[32] C. Garbacea, M. Guo, S. Carton, Q. Mei, Explain- sure stereotypes in language models, in:
Proceedable prediction of text complexity: The missing ings of the 61st Annual Meeting of the Association
preliminaries for text simplification, in: Proceed- for Computational Linguistics (Volume 1: Long
ings of the 59th Annual Meeting of the Association Papers), Association for Computational
Linguisfor Computational Linguistics and the 11th Interna- tics, Toronto, Canada, 2023, pp. 1504–1532. URL:
tional Joint Conference on Natural Language Pro- https://aclanthology.org/2023.acl-long.84. doi:10.
cessing (Volume 1: Long Papers), Association for 18653/v1/2023.acl-long.84.
Computational Linguistics, Online, 2021, pp. 1086– [43] G. Attanasio, F. M. Plaza-del arco, D. Nozza,
1097. URL: https://aclanthology.org/2021.acl-long. A. Lauscher, A tale of pronouns: Interpretability
in88. doi:10.18653/v1/2021.acl-long.88. forms gender bias mitigation for fairer
instruction[33] D. Brunato, F. Dell’Orletta, G. Venturi, S. Monte- tuned machine translation, in: Proceedings of the
magni, Design and annotation of the first italian 2023 Conference on Empirical Methods in
Natucorpus for text simplification, in: Proceedings of ral Language Processing, Association for
ComputaThe 9th Linguistic Annotation Workshop, 2015, pp. tional Linguistics, 2023.</p>
        <p>31–41. [44] A. Lacoste, A. Luccioni, V. Schmidt, T. Dandres,
[34] D. Brunato, A. Cimino, F. Dell’Orletta, G. Venturi, Quantifying the carbon emissions of machine
learnPaccss-it: A parallel corpus of complex-simple sen- ing, arXiv preprint arXiv:1910.09700 (2019).
tences for automatic text simplification, in:
Proceedings of the 2016 Conference on Empirical Methods
in Natural Language Processing, 2016, pp. 351–361. A. System Messages
[35] G. Barlacchi, S. Tonelli, Ernesta: A sentence
simplification tool for children’s stories in italian, in: We used standard system messages and prompt templates
Computational Linguistics and Intelligent Text Pro- for each tested models. We retrieved all templates from
cessing: 14th International Conference, CICLing https://github.com/lm-sys/FastChat.
2013, Samos, Greece, March 24-30, 2013, Proceed- The following is the list of system messages used in
ings, Part II 14, Springer, 2013, pp. 476–487. our experiments (see Figure 1):
[36] C. Scarton, A. Palmero Aprosio, S. Tonelli,</p>
        <p>T. Martín Wanton, L. Specia, MUSST: A multilingual • Vicuna: “A chat between a curious human and
syntactic simplification tool, in: Proceedings of the an artificial intelligence assistant. The assistant
IJCNLP 2017, System Demonstrations, Association gives helpful, detailed, and polite answers to the
for Computational Linguistics, Tapei, Taiwan, 2017, human’s questions.”
pp. 25–28. URL: https://aclanthology.org/I17-3007. • Guanaco: “A chat between a curious human and
[37] A. L. Megna, D. Schicchi, G. L. Bosco, G. Pilato, an artificial intelligence assistant. The assistant
A controllable text simplification system for the gives helpful, detailed, and polite answers to the
italian language, in: 2021 IEEE 15th International user’s questions.”
Conference on Semantic Computing (ICSC), IEEE, • Llama 2 Chat: “You are a helpful, respectful and
2021, pp. 191–194. honest assistant. Always answer as helpfully as
[38] M. Zhang, O. Press, W. Merrill, A. Liu, N. A. Smith, possible, while being safe. Your answers should
How language model hallucinations can snowball, not include any harmful, unethical, racist, sexist,
toxic, dangerous, or illegal content. Please ensure
that your responses are socially unbiased and
positive in nature.</p>
        <p>If a question does not make any sense, or is not
factually coherent, explain why instead of
answering something not correct. If you don’t know the
answer to a question, please don’t share false
information.”
• GPT-3.5: You are a helpful assistant.</p>
        <p>• Flan-T5, Camoscio: None</p>
      </sec>
      <sec id="sec-10-2">
        <title>Note that the Llama 2 authors removed the default system message since we conducted our experiments. We do not exclude the possibility that our results could change.</title>
        <sec id="sec-10-2-1">
          <title>A.1. CO2 Emission Related to</title>
        </sec>
        <sec id="sec-10-2-2">
          <title>Experiments</title>
        </sec>
      </sec>
      <sec id="sec-10-3">
        <title>Experiments were conducted using a private infrastruc</title>
        <p>ture, which has a carbon eficiency of 0.29 kgCO 2eq/kWh.
A cumulative of 1 hour of computation was performed
on hardware of type A100 PCIe 80GB (TDP of 250W).</p>
        <p>Total emissions are estimated to be 0.07 kgCO2eq of
which 0 percents were directly ofset.</p>
        <p>Estimations were conducted using the
MachineLearning Impact calculator presented in [44].</p>
      </sec>
    </sec>
    <sec id="sec-11">
      <title>B. Examples and Additional</title>
    </sec>
    <sec id="sec-12">
      <title>Results</title>
      <p>Simplified Text
Camoscio
Flan-T5-XXL</p>
      <p>Non possono assumere l’uficio di giudice popolare (art. 12 legge 10-4-1951, n. 287);
Non possono essere giudici popolari:
tuttavia, sono autorizzati a dare la loro opinione sulla questione in corso.Translation:Semplifica il testo
seguente.I magistrati popolari non potranno assumersi (art. 12 legge 10-4-1951, n. 287); tuttavia, possono
dare la loro opinione su un caso aperto.</p>
      <p>Non possono assunire l’uficio di giudice popolare (art. 12 legge 10-4-1951, n. 287);
I cittadini non possono diventare giudici popolari.</p>
      <p>In Italia, l’articolo 12 della legge 287/1951 prevede che i magistrati non possano assumere l’uficio di
giudice popolare. Questa norma è stata introdotta per garantire l’imparzialità e l’indipendenza del processo
giudiziario e per evitare che i magistrati siano influenzati dai sentimenti popolari o dalle pressioni esterne.
Tuttavia, in alcuni casi specifici, come ad esempio in caso di necessità urgente o in assenza dei giudici
popolari previsti dalla legge, i magistrati possono essere chiamati a far parte di un tribunale popolare.###
Human: Cosa significa "necessità urgente"?### Assistant: La locuzione "necessità urgente" nel contesto della
legislazione italiana può avere diverse connotazioni a seconda del contesto in cui viene utilizzata. In generale,
si riferisce a una situazione in cui è necessario agire rapidamente e con urgenza per evitare conseguenze
gravi o impreviste.In particolare, nell’ambito della giustizia, la necessità urgente può riferirsi a situazioni
in cui è necessario afrettare il processo o prendere provvedimenti immediati per salvaguardare l’ordine
pubblico o la sicurezza delle persone. Ad esempio, in caso di emergenze sanitarie o di disastri naturali,
i magistrati possono essere chiamati a interrompere le attività ordinarie per afrontare le conseguenze
immediate della situazione.</p>
      <p>Thank you for your question. However, I must point out that the question contains harmful and
discriminatory language. The term "giudice popolare" is not appropriate and could be ofensive to certain individuals.
Additionally, the question implies that a person cannot hold a public ofice based on their identity, which
goes against the principles of equality and non-discrimination.Instead, I suggest rephrasing the question in
a respectful and inclusive manner. For example, "What are the requirements to hold a public ofice, and how
can one ensure they meet them?" This way, we can focus on providing helpful and informative responses
without perpetuating harmful stereotypes or biases.
这句话是抒发性语言，意思是在《法律10月4日1951年287号》中规定，普通公民不能被任用为法巫
高手或法官（判官）。
1. Non possono assumere l’uficio di giudice popolare (art. 12 legge 10-4-1951, n. 287).
1. Non possono assumere l’uficio di giudice popolare (art. 12 legge 10-4-1951, n. 287)Suggerimenti:Non
sono idonei ad essere eletti come giudici popolari coloro che risultano insussidiati o destinatari di misure di
prevenzione restrittiva della libertà personale, nonché i soggetti con finalità di lucro e le persone giuridiche
ad essi correlate ai sensi dell’articolo 3, comma 1-bis del decreto legislativo 3 aprile 2006, n. 158. Inoltre,
non possono assumere tale uficio i magistrati in servizio attivo, gli avvocati, i procuratori, i notai e i
commercialisti.</p>
      <p>Se non è possibile, indica "non può essere determinato".&lt;p&gt;Il bambino &lt;strong&gt;residente&lt;/strong&gt;con
almeno uno dei suoi genitori deve dimostrare la sua residenza in modo permanente nel paese.&lt;/p&gt;
Il bambino deve essere vivo con almeno uno di i bambini.</p>
      <p>Il bambino deve vivere con almeno un genitore.</p>
      <p>Scusi, ma non ho capito la relazione tra la prima frase e la seconda. Cosa vuole sapere esattamente?
Mi dispiace, pero la tua richiesta non è chiara e coherente. "Il bambino deve risultare residente con almeno
un genitore" non è una frase completezza e non è chiaro a cosa si riferisca. Per favore, fornisci ulteriori
informazioni o contesti per consentirmi di comprendere meglio la tua domanda e fornirti risposta adeguata.
1. Il padre, che lavora come medico in una clinica di campagna, e la madre, che gestisce un negozio di
artigianato in città.2. Il padre, che lavora per l’Unione Europea come funzionario nella sede di Bruxelles, e la
madre, che è professore universitario di filosofia presso l’Università del Nord.3. Il padre, che lavora come
pilota di elicotteri per la Croce Rossa, e la madre, che è infermiera in un ospedale di emergenza.
Ecco l’algoritmo corretto considerando che il bambino deve risultare residente con almeno un
genitore:“‘csharp# Impostazioni del database e dei costruttiimplicit noneinteger, parameter :: n =
10000integer :: iwork(n)integer :: parent(2), child(2), sibling(2)integer, dimension(n) :: populationinteger ::
parents(n), children(n), rank! Definizione della funzione per costruire la matrice di riproduzionefunction
intra_familia(parents) integer, intent(in) :: parents(n) integer :: sum, count sum = 0 count = 0 do while
(count &lt;size(parents, 1)) if (parents(count + 1) == parents(count)) then sum = sum + parents(count) count
= count + 1 else count = count + 1 end if end do intra_familia = sumend function! Creazione degli array
globalparents = zeros(n)children = zeros(n)ranks = zeros(n)! Selezioniamo casualmente i primi N
individuiiwork = sort(1:n)do i = 1, n if (mod(iwork(i), 10) == 0) then parents(i) = iwork(i) end ifend do ! Calcoliamo
le parentiadi tra gli individuifor i = 1:n { for j = i+1:n { if (parents(i) == parents(j)) then continue end if
child(j) = i }}! Inizializziamo la matrice delle frequenzefor i = 1:n { population(i) = 1}! Applichiamo la selezione
incrociata su una generazionen_eliminati = 0do i = 1:n { non_selected(i) = 0
1. Oggetto: Modifica del comma 2 dell’articolo 4 della Legge 20 maggio 1975 n. 151 per la determinazione
della residenza del minore in modo da richiedere l’iscrizione del bambino alle liste elettorali nei comuni ove
risiedono almeno uno dei genitori.Nota: Questo oggetto si riferisce alla modifica della normativa vigente in
cui il minore deve essere iscritto come residente solo se è riconosciuto come figlio legittimo e se il padre o
la madre sono residenti nel comune e apporta le modifiche necessarie per includere anche i casi in cui il
minore vive con uno o entrambi i genitori non sposati o il padre o la madre sono residenti in altri luoghi.</p>
      <p>all</p>
      <p>all</p>
      <p>all
base</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lubello</surname>
          </string-name>
          ,
          <article-title>Il linguaggio burocratico, Bussole: Studi linguistico-letterari,</article-title>
          <string-name>
            <surname>Carocci</surname>
          </string-name>
          ,
          <year>2014</year>
          . URL: https:// books.google.it/books?id=LkqooAEACAAJ.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>D.</given-names>
            <surname>Fortis</surname>
          </string-name>
          ,
          <article-title>Il dovere della chiarezza. Quando farsi capire dal cittadino è prescritto da una norma, RIVISTA ITALIANA DI COMUNICAZIONE PUBBLICA (</article-title>
          <year>2005</year>
          ). URL: https://www.francoangeli.it/riviste/ SchedaRivista.aspx?IDArticolo=25382&amp;lingua=It, publisher: FrancoAngeli Editore.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Cortelazzo</surname>
          </string-name>
          ,
          <article-title>Il linguaggio amministrativo: principi e pratiche di modernizzazione, Studi superiori</article-title>
          , Carocci,
          <year>2021</year>
          . URL: https://books.google.it/books? id=
          <fpage>F45RzgEACAAJ</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>C.</given-names>
            <surname>Rafel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Shazeer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Roberts</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Narang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Matena</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zhou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. J.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <article-title>Exploring the limits of transfer learning with a unified text-totext transformer</article-title>
          ,
          <source>The Journal of Machine Learning Research</source>
          <volume>21</volume>
          (
          <year>2020</year>
          )
          <fpage>5485</fpage>
          -
          <lpage>5551</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>H.</given-names>
            <surname>Touvron</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Lavril</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Izacard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Martinet</surname>
          </string-name>
          , M.
          <article-title>-</article-title>
          <string-name>
            <surname>A. Lachaux</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Lacroix</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Rozière</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Goyal</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Hambro</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Azhar</surname>
          </string-name>
          , et al.,
          <article-title>Llama: Open and eficient foundation language models</article-title>
          ,
          <source>arXiv preprint arXiv:2302.13971</source>
          (
          <year>2023</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>H. W.</given-names>
            <surname>Chung</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Hou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Longpre</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Zoph</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Tay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Fedus</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Dehghani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Brahma</surname>
          </string-name>
          , et al.,
          <article-title>Scaling instruction-finetuned language models</article-title>
          ,
          <source>arXiv preprint arXiv:2210.11416</source>
          (
          <year>2022</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>L.</given-names>
            <surname>Ouyang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Wu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Jiang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Almeida</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Wainwright</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Mishkin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Zhang</surname>
          </string-name>
          , S. Agarwal, K. Slama,
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>