In this paper, we present our approach for the CLEF 2025 SimpleText Task 1, which addresses both sentence-level and document-level scientific text simplification. For sentence-level simplification, our methodology employs large language models (LLMs) to first generate a structured plan, followed by plan-driven simplification of individual sentences. At the document level, we leverage LLMs to produce concise summaries and subsequently guide the simplification process using these summaries. This two-stage, LLM-based framework enables more coherent and contextually faithful simplifications of scientific text.
In the past decade or so, a new form of learning has emerged. Instead of science being only accessible through journals or formal education, the general public is now privy to an enormous wealth of material through the Internet. Whether it be directed self-studying, or casual social media perusal, nearly everyone is now able to access scientific information. An important caveat remains, however, and that is when a resource is accessible for free and without rigorous moderation, misinformation will invariably run rampant. This is why now more than ever, the need for reliable, easy-to-understand scientific-based text and content has taken the forefront.
This is where automated text simplification comes in. With the volume of scientific text at hand,
rewriting the same exact content in layman terms manually is intractable. Resources have been
expended towards automating this task, and over the course of 20 years, automatic text simplification
has progressed significantly [
The capabilities of Large Language Models have made them a game-changer for automatic text simplification. Unlike previous methods, LLMs can achieve a deeper semantic interpretation of source text, allowing them to not only simplify vocabulary and syntax but also to summarize and restructure information for clarity. Though their internal representations of knowledge are opaque, their practical application as a powerful tool for generating simplified text is undeniable, paving the way for more efective simplification systems.
The SimpleText lab [
In this paper, we tackle both sentence-level text simplification as well as document-level using a tiered approach. For the sentence-level, an LLM first generates a simplification strategy regarding a sentence, and then the LLM is tasked to perform that strategy to generate a simplified version. For the document-level, the LLM generates a summary of the text as a whole and then uses that summary as part of the prompt that guides the simplification process.
The paper is organized as follows: Section 2 provides a small literature review of the text simplification task; section 3 provides details of the approach undertaken for each of the subtasks; section 4 showcases the results and provides a brief discussion; section 5 discusses possible future work and the conclusions we have derived from these experiments.
Scientific text simplification aims to enhance the accessibility and comprehensibility of technical content for non-expert audiences, including patients, educators, and the general public. This task has been explored at both sentence and document levels, with increasing interest in using neural and large language model (LLM)-based methods.
Ondov et al. [
Recent work has also examined the role of LLMs in improving user comprehension and reducing
cognitive load. Guidroz et al. [
Fang et al. [
These findings support our motivation to investigate structured prompting methods to enhance control, coherence, and factuality in the simplification of scientific texts.
A variety of automatic evaluation metrics are employed to assess the quality of generated or simplified text. In this work, we consider four commonly used metrics: SARI, BLEU, BERTScore (F1) and FleschKincaid grade level (FKGL), each capturing diferent aspects of text quality to compare LLM-based simplification strategies.
SARI (System output Against References and against the Input sentence) [
BLEU (Bilingual Evaluation Understudy) [
BERTScore [
Flesch–Kincaid Grade Level (FKGL) [
The focus of task 1 is to study the performance of simplification systems in both sentence-level and document-level settings.
Inspired by recent advances in plan-driven sentence simplification [
The task is structured into two stages. In the first stage, the model is prompted to select an appropriate simplification strategy from a predefined set: rephrase, delete, split, ignore, or merge. In the second stage, the model is prompted to generate the corresponding simplified sentence based on the selected strategy[Appendix B.1].
In this task, we leverage large language models (LLMs) for summary-guided document simplification [
First, the model is prompted to produce a clear and concise summary of the input complex document[Appendix B.2]. This summary serves as a semantic scafold to guide the simplification process. In the second step, the same model is prompted to simplify the original document using the generated summary as contextual guidance[Appendix B.3]. This strategy enhances coherence and faithfulness while reducing the risk of over-simplification.
The plan_guided_llama system demonstrates efective sentence-level simplification on the 37 aligned Cochrane-auto abstracts, achieving a strong SARI score of 42.33 and reducing lexical complexity (8.52) while maintaining reasonable BLEU and FKGL values. This indicates that the model balances simplification and content preservation. Detailed results are shown in Table 1. e r o c S y t i x e l p m o C l a c i x e
L 8.89 8.71 8.52 Method
The plan_guided_llama model demonstrates strong simplification capability on the 217 Plain Language Summaries test set, achieving a SARI score of 42.98, which reflects balanced simplification performance. However, its BLEU score (6.33) is notably low, suggesting limited surface-level overlap with references. The model produces simplified outputs with a significantly lower FKGL (7.82) compared to the source (13.29), indicating improved readability. The compression ratio (0.48) and low exact copy rate (0.00) suggest aggressive simplification, while the deletions proportion (0.71) is higher than additions (0.18), pointing to a tendency to simplify by removal. Full results are shown in Table 2.
Overall, the results demonstrate that the plan-guided LLaMA system efectively simplifies complex biomedical text at sentence-level while maintaining readability and informativeness, with a trade-of in exact lexical overlap. 4.2. Evaluation of Task 1.2: Document-level Scientific Text Simplification The results in Table 3 show that the llama_summary_simplification system achieves a SARI score of 40.32, indicating moderate simplification quality. While the BLEU score (7.63) is comparatively low—suggesting some divergence from reference phrasing—the FKGL score of 9.56 reflects improved readability relative to the source. The compression ratio of 0.59 and deletion proportion of 0.70 suggest aggressive content reduction, contributing to simplification but potentially at the cost of semantic ifdelity. 1.00 0.00 0.00 n o i t r o p o r P s n o i t i d d
A 0.00 0.16 0.31 0.00 0.60 0.70 e r o c S y t i x e l p m o C l a c i x e
L 9.05 8.65 8.50 e r o c S y t i x e l p m o C l a c i x e
L 8.89 8.71 8.49 Method
The llama_summary_simplification method demonstrates efective simplification on the 217 Plain Language Summaries, achieving a strong SARI score of 42.92 and a reduced FKGL of 9.94. The lexical complexity score of 8.55 indicates simplification in vocabulary. However, a relatively low BLEU score of 5.32 and Levenshtein similarity of 0.39 suggest reduced semantic similarity to the reference (Table 4).
Overall, the summary guided method achieves consistent simplification across both the datasets, balancing lower complexity with reduced semantic fidelity.
1.00 0.00 0.00 n o i t r o p o r P s n o i t i d d
A 0.00 0.29 0.24 n o i t r o p o r P s n o i t e l e
D 0.00 0.63 0.75 e r o c S y t i x e l p m o C l a c i x e
L 9.05 8.65 8.55 While the direct LLM-based simplification approach slightly outperforms the summary-guided method in terms of standard metrics such as SARI, BLEU, and BERTScore, the summary-guided approach ofers distinct benefits. By first generating a high-level summary and then using it to steer the simplification process, this method can produce more coherent and purpose-driven simplifications. The separation of summarization and simplification stages allows the model to better focus on key ideas, potentially reducing redundancy and irrelevant elaborations.
Moreover, the reduced token length in the summary-guided method suggests more concise output, which can be beneficial in applications where brevity and focus are important. Although the readability score (FKGL) is slightly higher, indicating marginally more complex language, the summary-guided approach may improve factual alignment and structural cohesion.
This study explores the efectiveness of large language models (LLMs) in text simplification across both sentence and document levels. At the sentence level, our plan-driven approach, which prompts the model to select an explicit simplification strategy before generation, yields improved performance in fluency and readability compared to direct simplification. At the document level, we propose a summary-guided simplification pipeline that, while slightly underperforming in standard metrics, ofers qualitative advantages in conciseness and coherence by leveraging intermediate summarization as contextual scafolding.
Our work demonstrates that incorporating structural planning and summarization can enhance LLM-based simplification, especially for longer and more complex texts. Future research will focus on 1Only SARI is the oficial evaluation metric for Task 1. Other metrics are reported for supplementary analysis. improving the structural prompting framework by introducing an iterative loop that refines prompts based on automatic evaluation metrics, serving as a built-in feedback mechanism.
We thank the Data Science at Georgia Tech (DS@GT) CLEF competition group for their support. This
research was supported in part through research cyberinfrastructure resources and services provided by
the Partnership for an Advanced Computing Environment (PACE) at the Georgia Institute of Technology,
Atlanta, Georgia, USA [
During the preparation of this work, the authors used ChatGPT and Gemini for grammar and spelling check, as well as assistance in the code for the conducted experiments. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.
You a r e a s e n t e n c e s i m p l i f i e r .
Given a document , a s e n t e n c e from t h a t document , and t h e n e x t s e n t e n c e f o r c o n t e x t , c h o o s e an i n t e r n a l s i m p l i f i c a t i o n s t r a t e g y from t h e f o l l o w i n g o p t i o n s : ’ r e p h r a s e ’ , ’ d e l e t e ’ , ’ s p l i t ’ , ’ i g n o r e ’ , ’ merge ’ .
Then o u t p u t ONLY t h e s i m p l i f i e d s e n t e n c e , b a s e d on yo u r c h o s e n s t r a t e g y .
Document : The e c o n o m i c r e p o r t showed a s i g n i f i c a n t downturn i n t h e l a s t q u a r t e r .
You a r e g i v e n a complex document . Your t a s k i s t o w r i t e a c l e a r and c o n c i s e summary t h a t c a p t u r e s t h e e s s e n t i a l i n f o r m a t i o n , main arguments , and key f i n d i n g s .
G u i d e l i n e s : − Do not i n c l u d e minor d e t a i l s or examples u n l e s s c r u c i a l t o t h e main i d e a . − Focus on t h e o v e r a l l message and s t r u c t u r e o f t h e document . − Use s i m p l e and a c c e s s i b l e l a n g u a g e . − The summary s h o u l d be u n d e r s t a n d a b l e w i t h o u t r e a d i n g t h e o r i g i n a l document . ### Document : { document } ### Summary :
Listing 1: Prompt for LLM-Based Summary-Guided Document Simplification You a r e g i v e n a complex document and i t s summary . Your t a s k i s t o r e w r i t e t h e complex document i n a s i m p l e r , c l e a r e r way w h i l e e n s u r i n g t h e meaning a l i g n s with t h e p r o v i d e d summary . G u i d e l i n e s : − Keep t h e r e w r i t t e n v e r s i o n f a i t h f u l t o both t h e o r i g i n a l document and i t s summary . − Use simple , a c c e s s i b l e v o c a b u l a r y and s e n t e n c e s t r u c t u r e s . − Avoid i n t r o d u c i n g new i n f o r m a t i o n not p r e s e n t i n t h e o r i g i n a l document . − R e t a i n t h e key i d e a s , s t r u c t u r e , and i n t e n t c a p t u r e d i n t h e summary . ### Complex Document : { document } ### Summary : { summary } ### S i m p l i f i e d Document :