The widespread availability of online reviews and the pervasive role of social media have reshaped how travelers plan vacations. In this context, user-generated content has become a key resource, strongly influencing consumer behavior. In tourism, sentiment-driven decision-making is especially relevant, making sentiment analysis a valuable tool for extracting insights from reviews and social media posts. However, adapting these techniques to domain- and language-specific contexts remains challenging. To address this, Rest-Mex was developed as an evaluation framework for analyzing tourism-related texts in Mexican Spanish, a variety often underrepresented in NLP resources. This paper presents the Sintax Surfers team's approach to the Rest-Mex task, which involved training a decision tree-based model to predict polarity, location (town), and type of place from online reviews. Our model achieved accuracies of 68.49% for town, 71.40% for polarity, and 95.99% for type of place, demonstrating the efectiveness of lightweight, interpretable models for domain-specific sentiment analysis in under-resourced language varieties.
The proliferation of online reviews and the ubiquitous presence of social media have profoundly
transformed how travelers make vacation-related decisions. Today, consumers heavily rely on
usergenerated content when planning their trips. Tourists frequently prioritize peer reviews over traditional
factors such as price or location [
This dynamic has led to an exponential increase in user-generated data, particularly in the form
of travel reviews. These reviews now serve as a critical source of information for consumers and a
powerful marketing tool for businesses [
Rest-Mex arises as a response to this challenge, representing the first evaluation forum since 2021 (with
subsequent editions in 2022 and 2023) aimed explicitly at text analysis in the tourism domain, focusing
on Mexican Spanish [
The remainder of this paper is organized as follows: first, we describe the dataset used for the task; next, we outline the architectures proposed for each subtask; and finally, we present the results obtained on the evaluation dataset.
In this section, we describe the dataset used for the Sentiment Analysis Task and the architectures used to tackle this task.
The dataset comprises a collection of 208,501 documents in Spanish, each containing a title and its corresponding review. Each document represents a user-generated review of a tourist attraction, hotel, or restaurant located in Mexico. The reviews were sourced from the TripAdvisor website, and the dataset was provided with pre-assigned labels.
As a first step in our analysis, we examined the word count distribution across the corpus. Table 1 presents summary statistics obtained after concatenating the title and review for each document. "Text length" refers to the total number of characters, including letters, digits, spaces, and punctuation marks, while "word count" denotes the number of words per document.
As part of our exploratory analysis, we investigated the distribution of word frequencies within the corpus. Figure 1 displays a word cloud representing the most frequent tokens. As is commonly observed in natural language datasets, the most frequent terms are stop words, such as “de,” “y,” “la,” “el,” “en,” “que,” “es,” “un,” “a,” and “muy,” among others.
Subsequently, we examined the distribution of instances across the diferent classes. Figure 2 illustrates the class distribution within the training dataset. Among the classification tasks, the Type class (Figure 2b) exhibits the most balanced distribution across its categories. In contrast, the Polarity (Figure 2a) and Town (Figure 2c) classes show a marked imbalance. Notably, the Town class includes a high number of distinct labels (a total of 40), which may introduce additional challenges during the model training process.
Metric mean std min 25% 50% 75% max
To prepare the text for processing with transformer-based architectures, the following steps were applied to the reviews: • Conversion to lowercase • Typo correction • Removal of whitespaces • Removal of special characters In some cases, the title and the review text were concatenated into a single input; such instances are explicitly indicated.
In this section, we describe the training procedure for each subtask. The main strategy involved using pre-trained transformer-based models for the feature extraction phase, followed by a dense neural network layer for the classification phase.
In the Polarity subtask, we employed the RoBERTa-base-bne model [19], a variant of the RoBERTa architecture [20]. RoBERTa-base-bne is a transformer-based masked language model specifically designed for the Spanish language.
To prepare the input text, the RoBERTa model requires tokenizing documents using its own tokenizer, so it was use. The architecture of the proposed neural network is based on the RoBERTa base model, whose output embeddings are concatenated with a fully connected neural layer consisting of 512 units. This is subsequently followed by an additional dense layer of 256 units, which serves to further refine the learned representations before passing them to the final classification head.
To improve the model’s generalization capabilities and reduce the risk of overfitting, a dropout rate of 0.2 was applied to the fully connected layers. The training process was conducted using the AdamW optimization algorithm, with a learning rate set to 0.00002. The model was trained for a total of 8 epochs, a setting chosen to balance convergence and computational eficiency.
This architecture aims to leverage the powerful contextual representations produced by RoBERTa, while the additional dense layers facilitate task-specific adaptation and improved discriminative performance.
(a) Polarity class.
(b) Type class.
(c) Town class.
For the Type subtask, we also employed the RoBERTa-base-bne model [19], as mentioned in the previous section the use of the RoBERTa-base-bne model requires the RoBERTa tokenizer [20]. The overall architecture consists of a sequence of processing steps: input text is first tokenized using the RoBERTa tokenizer, followed by feature extraction through the RoBERTa-base-bne model. The output embeddings are then passed to a fully connected layer for classification. The classification layer comprises five neurons with a softmax activation function to predict class probabilities.
he training configuration included a batch size of 16, a dropout rate of 0.2 applied to the output of the RoBERTa model, and the AdamW optimizer [21] with a learning rate of 0.0001. The model was trained for a total of 9 epochs.
For the Town subtask, three diferent strategies were explored: Baseline Strategy: As a baseline, the BETO [22] model was used with minimal modifications. A classification head was added to the BETO model, with the number of output neurons corresponding to the number of target classes. The model was fine-tuned using a batch size of 8, for 3 epochs, with the AdamW optimizer. No further changes were made to the dataset.
Undersampling Strategy: In this approach, the dataset was undersampled so that each class contained approximately the same number of samples, equal to the average number of samples across all classes (or the nearest feasible number). The BETO model was then trained using the same hyperparameters as in the baseline strategy.
Combined Input Strategy: In this strategy, the Title and Review fields were concatenated into a single text input. This combined text was then used for both training and inference with the BETO model.
Only the best performance was reported in the Results section
The organizers of the Rest-Mex challenge proposed a custom evaluation metric to assess model performance. This metric, defined in Equation 4, combines the results of diferent subtasks through a weighted average of F1 scores. The components of the metric are defined as follows: () = ∑︀|=|1 () || • is a forum participant system. • = {1, 2, 3, 4, 5}. • () is the F-measure value for the class obtained by the system . • represents the list with all Magical Towns. • () represents the measure obtained by the system for the Town class. () = 2 · () + () + 3 · () 6 • () represents the result given by Equation 1. • () represents the result given by Equation 2.
• () represents the result given by Equation 3.
In addition to the Rest-Mex evaluation metric, we also report Accuracy, Precision, and F1 score to assess our models’ performance. These metrics provide a broader understanding of model behavior and help evaluate both the overall efectiveness and the reliability of the predictions.
• () represents the measure obtained by the system for the Attractive class. • () represents the measure obtained by the system for the Hotel class. • () represents the measure obtained by the system for the Restaurant class. () = () + () + ()
3 () = ∑︀=1( ) () ( ) Where: Where: Where: Where: (1) (2) (3) (4)
The test dataset consists of a total of 89,166 pairs of titles and their corresponding reviews. Both the title and the review text were joined to create the input required by the pre-trained models. As explained in previous sections, the Rest-Mex organizers proposed an equation to evaluate model performance. According to the reported results [17], the Track Score is: () = 64.123.
Table 2 presents the results reported by the Rest-Mex organizers. The model that achieved the lowest F1 score was the one developed for the Town subtask. This outcome may be attributed to the high number of distinct classes in the corpus, as well as the pronounced class imbalance between the most and least frequent labels.
A similar issue appears in the Polarity subtask, where the F1 score is also low. Although the number of classes in Polarity is significantly smaller compared to Town, the substantial imbalance among its classes suggests that class distribution plays a more critical role in model performance than the number of classes alone.
This may also explain why the accuracy scores for both subtasks are relatively similar, despite the diferences in F1 score. It is likely that, in the Polarity task, the model disproportionately favored majority classes, leading to inflated accuracy at the expense of minority class performance due to the more severe imbalance.
In the Type subtask, the best performance was achieved across all metrics. This suggests that class imbalance may significantly afect the other tasks. Additionally, the relatively small number of unique labels in this subtask could further contribute to the high performance observed.
Sentiment analysis is a valuable tool for tourists, as it enables quick access to client opinions about services, locations, products, and more. In recent years, natural language processing techniques—particularly attention-based models—have emerged as efective solutions for tasks of this nature.
In this paper, we presented our proposed method and the results obtained in the Rest-Mex Sentiment Analysis Task. The results reveal significant challenges in the Town and Polarity subtasks, where the large number of labels and pronounced class imbalance negatively afected both the training and classification phases. In contrast, the Type subtask yielded the highest performance across key evaluation metrics, including accuracy, precision, and F1 score.
As future work, we propose exploring advanced fine-tuning strategies and rebalancing techniques, such as EDA, back-translation, or the generation of synthetic instances, to enhance performance, particularly in the Town and Polarity subtasks. Additionally, further investigation could focus on the use of data augmentation methods, including EDA, Random Oversampling (ROS), and synthetic data generation approaches. Other rebalancing strategies, such as oversampling and undersampling, as well as the adoption of alternative architectures for model training, could also be explored.
The authors gratefully acknowledge the financial support provided to the first author by SECIHTI which supported his graduate studies during the development of this work.
We declare that the present manuscript has been written entirely by the authors and that no generative artificial intelligence tools were used in its preparation, drafting, or editing. A. P. López-Monroy, J. Martínez-Miranda, H. Pérez-Espinosa, A. Y. Rodríguez-González, Overview of rest-mex at iberlef 2021: Recommendation system for text mexican tourism (2021). [15] M. Á. Álvarez-Carmona, Á. Díaz-Pacheco, R. Aranda, A. Y. Rodríguez-González, D. Fajardo-Delgado, R. Guerrero-Rodríguez, L. Bustio-Martínez, Overview of rest-mex at iberlef 2022: Recommendation system, sentiment analysis and covid semaphore prediction for mexican tourist texts, Procesamiento del Lenguaje Natural 69 (2022) 289–299. [16] M. Á. Álvarez-Carmona, Á. Díaz-Pacheco, R. Aranda, A. Y. Rodríguez-González, V. Muñiz-Sánchez, A. P. López-Monroy, F. Sánchez-Vega, L. Bustio-Martínez, Overview of rest-mex at iberlef 2023: Research on sentiment analysis task for mexican tourist texts, Procesamiento del Lenguaje Natural 71 (2023) 425–436. [17] M. Á. Álvarez-Carmona, Á. Díaz-Pacheco, R. Aranda, A. Y. Rodríguez-González, L. Bustio-Martínez, V. Herrera-Semenets, Overview of rest-mex at iberlef 2025: Researching sentiment evaluation in text for mexican magical towns, volume 75, 2025. [18] J. Á. González-Barba, L. Chiruzzo, S. M. Jiménez-Zafra, Overview of IberLEF 2025: Natural Language Processing Challenges for Spanish and other Iberian Languages, in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2025), co-located with the 41st Conference of the Spanish Society for Natural Language Processing (SEPLN 2025), CEUR-WS. org, 2025. [19] A. G. Fandiño, J. A. Estapé, M. Pàmies, J. L. Palao, J. S. Ocampo, C. P. Carrino, C. A. Oller, C. R. Penagos, A. G. Agirre, M. Villegas, Maria: Spanish language models, Procesamiento del Lenguaje Natural 68 (2022). URL: https://upcommons.upc.edu/handle/2117/367156#.YyMTB4X9A-0. mendeley. doi:10.26342/2022-68-3. [20] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov,
Roberta: A robustly optimized bert pretraining approach, arXiv preprint arXiv:1907.11692 (2019). [21] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, 2017. URL: https://arxiv.org/abs/ 1412.6980. arXiv:1412.6980. [22] J. Cañete, G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, J. Pérez, Spanish pre-trained BERT model and evaluation data, in: PML4DC at ICLR 2020, 2020.