This paper presents the contribution of the PMOTE-UC-CUJAE team to the 2025 edition of the Rest-Mex shared task, focused on sentiment polarity classification in Spanish-language tourist reviews related to Mexico's Magical Towns. We propose a strategy based on probabilistic data augmentation techniques, employing covariance matrix estimation via the Ledoit-Wolf method, alongside Lasso regression and Elastic Net, applied to CLS embeddings generated using the RoBERTa-base-bne model.The resulting balanced datasets were used to train lightweight multilayer perceptron (MLP) classifiers, avoiding the need for computationally intensive transformer finetuning. Despite technical limitations that restricted our participation to a single subtask, the results demonstrate significant improvements over the baseline in key metrics such as Macro F1 for polarity. Notably, our models achieved balanced performance across all sentiment classes, including those with fewer examples, confirming the efectiveness of probabilistic oversampling in imbalanced contexts.These findings highlight the potential of probabilistic data augmentation methods for multilingual sentiment analysis tasks and reinforce the feasibility of eficient, transformer-free solutions in resource-constrained environments.
One of the most significant challenges in supervised classification problems is class imbalance, which occurs when one or more classes are represented by considerably fewer examples compared to others. This situation is common in real-world applications such as fraud detection, medical diagnosis, fault monitoring, risk analysis, and sentiment analysis—where minority classes are often the most critical. When models are trained on imbalanced data, they tend to optimize overall accuracy at the expense of properly detecting minority classes, thus compromising the practical usefulness of the system.
The field of sentiment analysis is not exempt from this phenomenon. Despite recent advances in
transformer-based techniques, there remains a need for methods capable of generating semantically
coherent synthetic data in high-dimensional spaces to efectively address imbalance. Classical
oversampling methods, such as SMOTE [
Most of these approaches rely on interpolation between -nearest neighbors to generate new samples.
While this is generally efective in low-dimensional domains, their performance tends to degrade when
dealing with high-dimensional embeddings. To overcome this limitation, a new oversampling strategy
was proposed in 2024 based on shrinkage estimation of the covariance matrix using the Ledoit–Wolf
method—known as SMOTE-COV HD [
Probabilistic approaches have thus emerged as competitive alternatives, modeling the statistical
distribution of minority classes based on their embeddings. This work is situated within Subtask 1 of
the Rest-Mex 2025 competition [
In this context, we propose a strategy based on probabilistic data augmentation using covariance matrix estimation via the Ledoit–Wolf method, as well as Lasso regression and Elastic Net. Our approach extracts CLS vector representations from the RoBERTa-base-bne model and generates new synthetic instances for the minority classes from multivariate distributions. These balanced representations are then classified using a lightweight multilayer perceptron (MLP), which avoids the need to fine-tune large transformer models, thereby reducing computational costs and ofering a viable solution for resource-constrained environments.
1 to 5). purposes.
are: The proposed approach follows a systematic sequence of steps to perform sentiment polarity classification on tourist reviews using probabilistic data augmentation and lightweight models. The process is detailed as follows: 1. Dataset Preparation: The dataset is examined to identify the relevant columns for each task, particularly the text column and the label column indicating the sentiment polarity (ranging from
Each review is tokenized using the pre-trained model roberta-base-uncased.
The [CLS] token embedding (a 768-dimensional vector) is
extracted for each review. The label column is mapped to numerical indices for classification
3. Data Augmentation: If class imbalance is detected, one of three probabilistic oversampling
techniques is applied to generate synthetic examples for the minority classes. The three variants
• Ledoit–Wolf (LW) shrinkage: The covariance matrix is estimated with shrinkage to
reduce estimation error [
Σ LW = (1 − ) +
where is the sample covariance matrix, is a structured target matrix (such as a scaled
identity matrix), and is a shrinkage parameter.
• Lasso Regression: The synthetic instances are generated based on regression coeficients
obtained by minimizing [
A lightweight Multilayer Perceptron (MLP) is trained on the balanced dataset. The data is split into 80% for training and 20% for testing. During training, the loss and accuracy are monitored, and Macro F1-scores are calculated on the test set. 5. Model Persistence: After training, the MLP weight dictionary and a JSON file mapping the class labels to indices are saved for later use.
and embedding pipeline and predicting sentiment classes. 6. Deployment: These files are then used to classify new review data, replicating the tokenization This pipeline allows eficient classification without transformer fine-tuning while leveraging semantically coherent synthetic samples to improve the performance of the model under severe class imbalance.
A detailed comparison between our three submitted systems (labeled as HM) and the reference baseline system (labeled as BL) for Subtask 1 of Rest-Mex 2025 is shown in Table 1. This subtask focuses on sentiment polarity classification of tourist reviews. The table reports the overall performance in terms of Macro F1 and Accuracy, along with F1 scores for each sentiment class (from 1 to 5), providing a clearer picture of how each method handles class imbalance.
Run PMOTE-UC-CUJAE_1 PMOTE-UC-CUJAE_2 PMOTE-UC-CUJAE_3 Baseline
Place HM HM HM BL
Method Ledoit-Wolf
Lasso ElasticNet Baseline
Macro F1 Accuracy F1_C1 F1_C2 F1_C3 F1_C4 F1_C5 0.4087 0.3783 0.3708 0.1584 61.41% 61.57% most balanced performance. • Elastic Net performed well in classes 2 (0.3464) and 3 (0.3275), though with slightly lower F1 for class 1. • Lasso regression also maintained consistent F1s above 0.28 for minority classes, highlighting its reliability in similar tasks.
These results confirm that the three probabilistic data augmentation methods allowed the MLP classifiers to learn more representative patterns across all sentiment categories, particularly for underrepresented ones. Unlike the reference system, our models achieved consistent results without relying solely on the dominant class.
This work presents a probabilistic data augmentation framework designed to address polarity classification in highly imbalanced contexts, such as the Rest-Mex 2025 task. By using regularized covariance estimators (Ledoit–Wolf), Lasso regression, and Elastic Net, we generated synthetic examples for minority classes based on the statistical structure of RoBERTa CLS embeddings, improving distribution without introducing semantic noise.
The experimental results validate the efectiveness of this strategy: the Ledoit–Wolf-based variant achieved the highest Macro F1 score (0.4087), significantly outperforming the reference system (0.1584) and showing balanced performance across all polarity classes. Although the Lasso and Elastic Net models also yielded competitive results, the superior performance of Ledoit–Wolf highlights the benefits of directly modeling global covariance structure in high-dimensional spaces.
Beyond classification accuracy, this approach greatly reduces computational costs. Unlike transformer ifne-tuning, which requires extensive computational resources, our system uses a lightweight MLPbased classifier, making it a practical solution for real-world sentiment analysis scenarios in tourism and related domains.
Future work will focus on: • Extending the approach to other multilingual or multimodal datasets where class imbalance is also critical. • Exploring block-structured or hierarchical covariance estimation to improve the scalability of the
Ledoit–Wolf method. • Incorporating semantic validation or diversity constraints during synthetic generation to enhance coverage and reduce redundancy. • Evaluating the approach in few-shot or zero-shot learning contexts, where synthetic generation may be crucial in the absence of labeled data.
This study reinforces the role of probabilistic data augmentation as a robust, scalable, and interpretable strategy to tackle imbalanced sentiment classification in complex domains.
We would like to express our sincere gratitude to the organizers of Rest-Mex 2025 for designing and coordinating this valuable shared task. Competitions of this nature not only promote research in lowresource languages and real-world applications, but also foster academic collaboration, methodological exchange, and the development of practical solutions in the field of Natural Language Processing. We especially appreciate the opportunity to participate and contribute to the scientific community through this platform.
We also thank the National Program of Science and Technology PN223LH004: Automation, Robotics and Artificial Intelligence, of the Ministry of Science, Technology and Environment of Cuba, for supporting this work under project PN223LH004-038: Theoretical contributions to AI in the management of complex data problems.
We declare that the present manuscript has been written entirely by the authors and that no generative artificial intelligence tools were used in its preparation, drafting, or editing.
The results and oficial rankings of the shared task can be accessed through the following link: • Rest-Mex 2025 Results