This paper presents our system for Aspect Sentiment Quad Prediction (ASQP) shared task in Portuguese hotel reviews. Our system leverages text generation-based language models to extract quadruples consisting of aspect categories, aspect terms, sentiment terms, and polarities from customer reviews. Our methodology focuses on a structured text generation paradigm that encodes the relationship between aspects and sentiments through a custom output format. Experimental results on the ABSAPT-2025 shared task dataset demonstrate the eficacy of our approach in handling the complexities of sentiment analysis in low-resource languages. The proposed model achieves competitive performance compared to the baseline and obtains an F1-score of 45.66% (rank 1) in the ASQP-PT 2025 [1, 2] shared task.
ABSA has evolved from simple sentiment classification to increasingly fine-grained analysis. Early
approaches such as those presented in SemEval tasks [
Json data
Preprocessing
Contextual data Test Sample 1 Test Sample 2 ...
Test Sample n test data
Evaluate
Trained LLM "{review}"="Furtaram a minha mala no hotel com todas as minhas coisas dentro e o hotel não se responsabilizou por nada. Nem desconto na tarifa eu tive."
FormatInput: """Please extract the list of categories, aspects, sentiments, and polaritiesin the following review: {review}""" Output:"""general is negative because hotel is Furtaram a minha mala [ssep] general is negative because hotel is não se responsabilizou
Sample 1 Sample 2 ...
Sample n
train data
trainitnugni/ngfineconverts all ABSA subtasks into text generation problems. Similarly, [
Multilingual approaches to sentiment analysis have gained popularity with the advent of
crosslingual pre-trained language models. Pretrained models such as mBERT [
Recent advances in NLP have demonstrated the efectiveness of framing structured prediction tasks
as sequence-to-sequence problems. [
As illustrated in Figure 1, our approach implements an end-to-end sequence-to-sequence workflow for Aspect Sentiment Quad Prediction. The process begins with raw JSON data containing Portuguese hotel reviews and their annotations, which undergoes preprocessing to normalize text and structure the training examples. We transform this data into contextual examples where each input-output pair follows a specific pattern. As shown in the workflow diagram, this creates paired training samples where the model learns to generate structured outputs from unstructured review text. The example demonstrates how the Portuguese review "Furtaram a minha mala no hotel com todas as minhas coisas dentro e o hotel não se responsabilizou por nada. Nem desconto na tarifa eu tive." is transformed into the structured output "general is negative because hotel is não se responsabilizou," capturing the aspect category, polarity, aspect term, and sentiment term in a single coherent structure.
The processed data is used to fine-tune a pre-trained language model (mT5), which is then evaluated against test samples following the same format. This unified approach ofers several advantages over pipeline methods, particularly in capturing the interdependencies between aspect categories, terms, sentiments, and polarities.
We conceptualize Aspect Sentiment Quad Prediction (ASQP) as a structured sequence transduction problem. Given an input review text = {1, 2, ..., } consisting of tokens, our objective is to generate a set of quadruples = {1, 2, ..., }, where each quadruple = (, , , ) comprises an aspect category ∈ , an aspect term (a contiguous span in ), a sentiment term (another span in ), and a sentiment polarity ∈ { , , }.
The fundamental challenge lies in capturing the complex interdependencies among these four elements while maintaining computational eficiency. Rather than decomposing ASQP into subtasks—as has been common in prior ABSA research—we adopt a holistic generative approach that leverages the semantic understanding capabilities of large pre-trained language models.
The T5-portuguese model [19] represents a language-specific adaptation of the original T5 architecture, specifically pre-trained on a large corpus of Portuguese text. Unlike multilingual variants, this model concentrates its entire parameter capacity on a single language, potentially ofering more nuanced linguistic representations for Portuguese-specific phenomena. The model follows the encoder-decoder architecture of the original T5: • Encoder: Transforms the input review text into contextualized representations • Decoder: Autoregressively generates the structured output containing sentiment quadruples The T5-portuguese model benefits from: • Focused pre-training on Portuguese linguistic patterns • Enhanced handling of Portuguese morphology and syntax • Better representation of Portuguese-specific semantic nuances • Domain adaptation to Portuguese web content
This language-specific pre-training theoretically provides advantages in capturing the subtleties of sentiment expression in Portuguese, which often employs complex verbal constructions and rich adjectival morphology that difer significantly from other Romance languages.
In parallel, we evaluate the multilingual T5 model [20], specifically the base variant containing approximately 580M parameters. The mT5 architecture follows the same encoder-decoder paradigm but has been pre-trained on mC4, a massive multilingual corpus covering 101 languages including Portuguese.
The decision to include mT5 is motivated by several factors: • Its substantial exposure to Portuguese data during pre-training • Its proven cross-lingual transfer capabilities, allowing it to leverage patterns learned from highresource languages • The potential for more robust representations through cross-lingual knowledge sharing We hypothesize that while mT5 allocates only a fraction of its parameter capacity to Portuguese, the cross-lingual transfer capabilities may compensate by adapting knowledge from related high-resource languages like Spanish and French.
For input encoding, we prepend a task-specific prompt to each review: Please extract the list of categories, aspects, sentiments and polarities in the following comment: "{review_text}".
For output encoding, we design a structured format that explicitly captures the relationships between elements in each quadruple: {category} is {polarity} because {aspect} is {sentiment} [ssep] ...
Where [ssep] is a special separator token used to delineate multiple quadruples. This format ofers several advantages: 1. It expresses the logical relationship between aspects and sentiments 2. It maintains a consistent structure that the model can learn to reproduce 3. It allows for variable numbers of quadruples per review 4. It reduces the need for complex output parsing mechanisms This formulation transforms ASQP into a direct text generation problem, allowing us to leverage the language model’s implicit knowledge without introducing task-specific architecture modifications
For model optimization, we employ a cross-entropy loss function at the token level: | | ℒ = − ∑︁ log (|<, ; )
=1 where represents the target output sequence, represents the input sequence, and represents the model parameters.
We fine-tune all parameters of the pre-trained mT5 model using the AdamW optimizer with a learning rate of 3 × 10 −4 and a linear warmup schedule over 10 percent of training steps. This approach allows the model to adapt its pre-trained representations to the specific linguistic patterns of Portuguese sentiment expression.
We conduct experiments on the ABSAPT-2025 dataset, which consists of hotel reviews written in Portuguese. The dataset contains annotations for aspect categories, aspect terms, sentiment terms, and polarities. Each review may contain multiple aspect-sentiment quadruples, with an average of 2.7 quadruples per review.
The reviews cover various aspects of hotel accommodations, including service, cleanliness, location, and value. The dataset is particularly challenging due to: • The linguistic complexity of Portuguese, with its rich morphology • The domain-specific terminology related to hospitality • The implicit sentiment expressions common in review language • The variable length and structure of user-generated content
ABCD Baseline
We implement our approach using the Hugging Face Transformers library. The model configuration is based on the google/mt5-base architecture, which contains approximately 580 million parameters. Both input and output sequences are limited to a maximum length of 512 tokens. We train the model using a batch size of 8 for 10 epochs. Optimization is performed using the AdamW optimizer with a weight decay of 0.01, and a learning rate of 3 × 10 −4 , incorporating 10% warmup steps. To improve memory eficiency, we use mixed precision training with the bfloat16 format. All training is conducted on an NVIDIA GPU with CUDA support. Additionally, dropout is applied at a rate of 0.1 for regularization, and gradient clipping with a maximum norm of 1.0 is employed to prevent exploding gradients.
At inference time, we employ beam search decoding with beam width 3, temperature 0.1, and top-p sampling with p=0.9. This configuration balances between deterministic output (necessary for consistent structured prediction) and diversity (useful for capturing varied expressions).
The model’s generated text is post-processed using a custom parser that: 1. Splits the sequence on the [ssep] token to identify individual quadruples 2. Extracts the category, polarity, aspect, and sentiment components using regular expressions 3. Locates the exact span positions of aspect and sentiment terms in the original text 4. Handles special cases such as implicit sentiments (marked as “NULL”) and generic aspect references
The resulting structured data is converted to the competition’s required JSON format, including position information for aspects and sentiment terms.
Tables 1 and Table 2 present the performance of our system with diferent backbone architectures for the quadrupled extraction task. Table 1 shows our internal experimental results on a validation split derived from the training data, allowing us to compare diferent model configurations and training strategies. Table 2 presents the oficial competition results as evaluated by the organizers on the held-out test set, demonstrating our system’s performance in the competitive context.
As shown in Table 1, our internal validation experiments explored various configurations of model architecture and training strategies. We evaluated both T5-portuguese and mT5 models across diferent instruction prompting approaches and data augmentation techniques. For instruction prompting, English instructions consistently outperformed both no instructions and Portuguese instructions across both model variants. The T5-portuguese model achieved its highest F1-score of 0.52 with English instructions, matching the best performance of the mT5. This counterintuitive finding—that English instructions work better than Portuguese instructions for a Portuguese language task—suggests that the models’ pre-training regimes may have better prepared them to follow English instructions. Our data augmentation experiments showed that augmentation strategies preserving the structural characteristics of the source data (“With source”) outperformed less constrained augmentation methods ("Without source"). However, neither augmentation approach improved upon the best non-augmented configuration, indicating that the quality of augmented data requires further refinement to provide meaningful benefits.
Table 2 reveals that our submission (Team ABCD) using the method with main configurations including English intruction prompt designs with T5-portuguese model achieved the top ranking in the oficial ASQP task evaluation, with an F1-score of 0.4566 compared to the baseline’s 0.4055. The performance gap is particularly pronounced in recall (0.4706 vs. 0.3678), suggesting our approach’s superior ability to identify relevant quadruples across diverse review contexts.
The diference between our internal validation scores (up to 0.52 F1) and the oficial competition score (0.4566 F1) highlights the challenges of domain adaptation and the potential presence of distributional shifts between training and test data. This phenomenon is common in shared tasks and underscores the importance of robust evaluation practices that account for potential overfitting to validation data.
In this paper, we present our system, which leverages the power of sequence-to-sequence modelling with pre-trained language models to extract quadruples consisting of aspect categories, aspect terms, sentiment terms, and polarities from customer reviews. Experimental results on the ABSAPT-2025 shared task dataset demonstrate the eficacy of our approach in addressing the complexities of sentiment analysis in low-resource languages. The proposed model achieves competitive performance without requiring complex architectural modifications or parameter-eficient fine-tuning techniques, establishing a strong baseline for future research in multilingual ASQP. For future work, we plan to fine-tune large language models to enhance overall performance and explore data augmentation techniques to expand the training set.
This research is funded by Vietnam National University HoChiMinh City (VNU-HCM) under grant number C2024-26-02.
During the preparation of this work, we used ChatGPT and Grammarly to check grammar and spelling and edit the content for clarity and coherence. After using these tools, we reviewed and edited the content as needed and took full responsibility for the publication’s content. Computational Linguistics, 2020, pp. 7871–7880. [16] G. Paolini, B. Athiwaratkun, J. Krone, J. Ma, A. Achille, R. ANUBHAI, C. N. dos Santos, B. Xiang, S. Soatto, Structured prediction as translation between augmented natural languages, in: International Conference on Learning Representations, 2021. URL: https://openreview.net/forum?id= US-TP-xnXI. [17] Y. Hao, S. He, W. Jiao, Z. Tu, M. Lyu, X. Wang, Multi-task learning with shared encoder for non-autoregressive machine translation, arXiv preprint arXiv:2010.12868 (2020). [18] R. Shin, C. Lin, S. Thomson, C. Chen, S. Roy, E. A. Platanios, A. Pauls, D. Klein, J. Eisner, B. Van Durme, Constrained language models yield few-shot semantic parsers, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 7699–7715. [19] D. Carmo, M. Piau, I. Campiotti, R. Nogueira, R. Lotufo, Ptt5: Pretraining and validating the t5 model on brazilian portuguese data, arXiv preprint arXiv:2008.09144 (2020). [20] L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Rafel, mt5: A massively multilingual pre-trained text-to-text transformer, 2021. URL: https://arxiv.org/abs/2010. 11934. arXiv:2010.11934.