This paper addresses the binary classification of clickbait in Spanish tweets using three distinct approaches to improve detection. The datasets were provided as part of the TA1C challenge in IberLEF 2025, and the phases of pretraining, data preparation, fine-tuning, and evaluation were structured to train the models. Techniques such as undersampling and oversampling were used to handle class imbalance, aiming to generate a balanced dataset for model training. The three models evaluated were: RoBERTuito with manual features, RoBERTuito with class weighting, and Llama 3.2 - 3B. The results indicated that Model 3, based on Llama 3.2 - 3B, achieved the best performance, with an F1-score of 95.04%, outperforming the other models presented in this work. Furthermore, the model's robustness in a competitive environment was highlighted, as it ranked sixth in the TA1C challenge with a score of 80.115% on the evaluation dataset. While Model 1 showed competitive performance, its reliance on manual features limited its generalization capacity. Model 2 exhibited overfitting, emphasizing the importance of improving balancing and generalization techniques. This study demonstrates the efectiveness of advanced architectures like Llama 3.2 for the clickbait detection task and highlights areas for improvement in future implementations.
As part of the background related to the linguistic analysis approach for detecting clickbait in Spanish,
two studies directly addressing this topic have been identified (Robles, 2020; Loayza, 2024) [
Robles S. [
Various studies have addressed the problem of clickbait detection from multiple approaches, incorporating both traditional feature extraction techniques and advanced deep learning models. The following is a chronological review of the most representative works in this field, focusing on those related to the use of Spanish and deep learning for headline classification.
The study by Omidvar et al. [
More recently, Broscoteanu et al. [
Later, Gamage et al. [
For the development of this work, we used the datasets provided by the organizers of Task 1 in the
TA1C: Clickbait Detection and Spoiling in Spanish competition [
TA1C_dataset_detection_train TA1C_dataset_detection_dev_gold TA1C_dataset_detection_test
2002 497
798 203
2800 700 700
This study follows a structured methodology consisting of pre-training, data preparation, fine-tuning, and evaluation phases for the binary classification of clickbait on Twitter. The pipeline used in the implementation of the models for this study is shown in Figure 1.
The pipeline comprises several stages. First, data preprocessing is performed to remove as much information as possible that may introduce noise into the classification model. To this end, URLs, mentions, emojis, hashtags, etc., were handled. Subsequently, class balancing techniques are applied, relevant features are extracted, and the model is trained using diferent techniques depending on the case.
To evaluate the model’s performance, the metrics of accuracy, precision, recall, and F1-score were considered, with the latter being the primary metric. To assess the performance of the fine-tuned model, the dataset was split into 80% for training and 20% for testing.
We initially approached the clickbait detection task using traditional machine learning techniques. In particular, we trained an Extra Trees Classifier with a resampling strategy that combined random Input Output
Preprocessing Read and load
data Clean dats
Evaluate model
Calculate the weight of classes
Tokenization Regularization
Random OverSampling Insertion of synonyms
Splitting data Models training
Config Model Model training oversampling and undersampling to address class imbalance. This model achieved an F1-score of 0.416. We then implemented a CatBoost Regressor under similar conditions, which improved performance to an F1-score of 0.664. However, both results remained below the levels required for a robust solution. Given these limitations, we shifted our focus to transformer-based models, which have demonstrated superior performance in natural language processing tasks.
For this study, we selected the three best-performing transformer architectures. The pipeline in Figure 1 is common to the three evaluated models, although each presents specific variations in data processing, feature extraction, and model architecture. The following section describes the specific details of each implemented model.
In this work, an initial dataset was used, consisting of a training dataset whose structure is detailed in Table 1. The unequal distribution between the classes was a common challenge in the context of binary classification, as the minority class, "Clickbait," was less represented compared to the majority class. To address this imbalance, two main techniques were used: undersampling and oversampling. The undersampling technique involved reducing the size of the majority class, while oversampling was implemented to increase the number of samples from the minority class. In this case, oversampling generated 702 duplicates of the "Clickbait" class, increasing its representation within the dataset. As a result of these techniques, the final dataset consisted of 3,000 examples, with a balanced 50% distribution for each class.
The model was a combination of representations generated by the RoBERTuito uncased model described
by Pérez et al. (2022) [
The manually extracted features were as follows: 1. Clickbait word count: The number of times a word from the list appears in the text. 2. Clickbait ratio: The percentage of words in the text that are part of the clickbait list. 3. Boolean presence: A binary indicator that determines whether the text contains at least one clickbait word. 4. Numbers: A binary indicator indicating whether the text contains numeric digits. 5. Exclamations: A binary indicator indicating whether the text contains exclamation marks (!). 6. Question marks: A binary indicator indicating whether the text contains question marks (?). 7. Uppercase letters: The proportion of uppercase letters in the text.
8. Length: The total number of words in the text.
Once these eight manual features were extracted, they were combined with the 768 dimensions generated automatically by the RoBERTuito model. The concatenation of both representations resulted in a final 776-dimensional vector, which was used for the binary classification of the texts.
The RoBERTuito uncased model corresponds to a variant of the RoBERTa architecture. This model was pre-trained using a corpus of 500 million Spanish-language tweets, providing it with an excellent ability to capture specific patterns from the Twitter platform, such as mentions, emojis, hashtags, and diverse content. Additionally, RoBERTuito has demonstrated its efectiveness in previous classification tasks, such as hate speech detection in the SemEval 2019 Task 5, HatEval dataset, sentiment and emotion analysis in the TASS 2020 dataset, and irony detection in the IrosVa 2019 dataset.
The dataset used, like in the previous model, consists of the training dataset shown in Table 1. In the
preprocessing stage, as in the previous model, URLs and representations of mentions, emojis, hashtags,
and diverse content were left to the RoBERTuito uncased tokenizer. To address this imbalance, the class
weighting technique was used instead of increasing the data through data augmentation techniques.
The class weights were calculated using the compute_class_weight function from sklearn [
Subsequently, random oversampling was used, randomly duplicating 499 examples from the minority class to raise its representation to 1,500 examples. As a third strategy, semantic data augmentation was applied based on synonym replacement using WordNet, with a modification rate of 30%, focusing only on words longer than four letters. This technique generated 600 additional synthetic examples, increasing the "Clickbait" class to a total of 2,100 entries. The model was quantized to 4-bit (QLoRA) with alpha=32 over 7 transformer modules for eficiency, with 2.94% of trainable parameters (97M of 3.31B).
We evaluated the efectiveness of the models based on the metrics previously mentioned, along with the training and validation losses. Once the data was read and processed, the class weighting obtained is as stipulated in Table 2.
Model 2 Model 3
Class 0 Class 1 Class 0 Class 1 Taking into account this data, it is noted that the model will indeed give more weight to errors in class 1 because it is the class with fewer data. Model 1 was trained for 5 epochs with a decay rate of 0.01, while models 2 and 3 were trained for 10 epochs with the same decay rate and early stopping with a patience of 3 evaluations. The training results of the three models are as follows: The results presented in Table 3 show that the model consistently improved in performance. The training loss decreased from 0.5271 in the first epoch to 0.0080 in the fifth, while the validation loss dropped from 0.4502 to 0.3224. Accuracy, recall, and F1 also progressively improved, reaching 94.67%, 94.57%, and 0.9464, respectively, by the end of training. These metrics reflect a good model fit, with a reduction in both training and validation loss, indicating an improvement in the model’s ability to generalize.
Clickbait words Has questions Has numbers Number of words Has clickbait Clickbait ratio Uppercase ratio Has exclamations Additionally, Table 4 shows that the most decisive features for distinguishing clickbait content include both the frequency of specialized vocabulary words, with an average of 1.8 per text compared to 1.65 in normal texts. Regarding emotional and formatting patterns, there is a higher usage of exclamation marks (81.8% vs 61.3%) and uppercase letters (4.7% vs 5.4%). In contrast, non-clickbait texts tend to include more figures (36.2% vs 25.3%), suggesting a more informative and factual approach. According to the data in Table 5, the training of Model 2 showed consistent improvement in accuracy, F1score, and recall, with a notable decrease in training loss (from 0.2207 to 0.0001), indicating memorization, and validation loss (from 0.3293 to 0.6235). The model stopped at step 600 (epoch 8/10). The key metrics remained stable and high, indicating that the model generalizes well to validation data and has achieved a good balance between the classes.
Finally, according to the data in Table 6, Model 3 showed rapid convergence, reducing the training loss Training Loss 0.341900 0.346200 0.094400 0.010500 0.021400 0.000400 0.000000 to nearly zero. The validation loss reached its minimum during the second and third epochs before showing a slight increase, which may indicate potential overfitting. Nevertheless, the final validation metrics F1-score: 94.73%, recall: 96.43%, along with corresponding accuracy and precision demonstrate solid performance and good generalization ability on the validation set.
Once the training was completed, the trained model was evaluated using the test set, corresponding to 20% of the data initially reserved, resulting in the metrics shown in Table 7.
Model 1 Model 2 Model 3 The performance analysis on the test set reveals that Model 3 (Llama 3.2 – 3B) achieved the best overall results, with an F1 score of 0.9504, outperforming Models 1 and 2, which achieved values of 0.9464 and 0.9141, respectively. Model 1 (RoBERTuito with Hybrid Features), although not reaching the performance of Model 3, demonstrated competitive performance (F1 = 0.9464), suggesting that the combination of RoBERTuito model embeddings provides significant value in detecting clickbait patterns. This hybrid approach enhances the model’s ability to capture subtle signals that are not easily representable in the embedding space without the strategy.
On the other hand, Model 2 (RoBERTuito with Class Weighting), which used class weighting to mitigate the dataset imbalance, showed the lowest performance (F1 = 0.9141). Although the weighting technique aims to favor the minority class during training, in this case, it was not as efective as the previous strategies, possibly due to an underrepresentation of relevant patterns that were not suficiently captured by the adjusted weights.
The results revealed that the strategies implemented by our team, VerbNex, identified as "gsdeyson," ranked 6th among the participating teams in the TA1C challenge at IberLEF 2025 Subtask 1. The model’s performance metrics on the evaluation dataset provided by the competition organizers are shown in Table 8. The Table 9 presents the oficial ranking of participants in Subtask 1 of the challenge. The results confirm the trend observed previously in the validation sets. Model 3 (Llama 3.2 – 3B) achieved the best performance, with a score of 0.8011, establishing it as the most efective architecture for the task in our implementations. It was followed by Model 2 (RoBERTuito + Class Weighting) with a score of 0.7878, showing an improvement compared to its previous performance, suggesting that the class weighting adjustment may have had a more positive efect in this setting. Finally, Model 1 (RoBERTuito with Hybrid Features) reached a result of 0.7666, the lowest among the three, indicating that while its hybrid approach was competitive in the test environment, it failed to maintain the same level of efectiveness against a more diverse competitive set. These findings reinforce the robustness of Model 3 and its adaptability for clickbait classification.
According to the data in Table 9, the diference with the fourth and fifth positions was marginal—less than 0.003 points—which highlights the competitiveness of the proposed system. This result supports the efectiveness of Model 3 (Llama 3.2 – 3B) and its consistency compared to systems developed by other research groups, consolidating a solid foundation on which incremental improvements can be applied to climb rankings in future editions of the competition.
The work developed for the binary classification of clickbait on Twitter has shown interesting results in terms of the performance of the evaluated models. Although the technique employed to handle class imbalance, specifically through undersampling and oversampling, achieved an acceptable balance in the dataset, it was observed that the implementation of these methods could be improved to maximize their efectiveness.
Model 1, which combines the RoBERTuito model with manually defined features based on a predefined vocabulary, achieved acceptable performance during internal testing. However, its performance in the competitive evaluation was lower compared to other models, reflecting limitations in its generalization capability. The reliance on the predefined keyword vocabulary turned out to be a restrictive factor, as not all relevant expressions in the competition dataset were covered. Despite these drawbacks, the model shows high improvement potential through the dynamic incorporation of new words and the integration of strategies such as embeddings or contextual synonyms, which would allow better adaptation to new domains and greater robustness against linguistic variations in clickbait. Model 2 showed good overall performance, with stable accuracy, particularly in the weighted F1 score, which remained close to 91%. However, the model exhibited clear overfitting, reflected in the increase of validation loss while the training loss decreased. This phenomenon suggests that the model may have memorized specific patterns from the training set, which afects its ability to generalize. Furthermore, it showed a bias towards the majority class, with suboptimal performance in detecting the minority class (clickbait), where recall only reached 83.77%. To improve, it would be crucial to implement additional techniques such as threshold optimization and the use of SMOTE to increase the diversity of examples. Additionally, performing a detailed error analysis will help identify patterns in incorrect predictions, which could fine-tune the detection ability for complex cases.
Model 3, based on the Llama 3.2 model and trained with the QLoRA strategy, demonstrated excellent performance in clickbait classification, achieving a near-perfect balance between precision and recall for both classes. The use of an efective data balancing strategy significantly improved the model’s ability to handle imbalanced classes. Although the model only completed 7 of the 8 planned epochs due to early stopping activation, no performance degradation was observed.
The authors would like to acknowledge the support provided by the master’s degree scholarship program in engineering at the Universidad Tecnologica de Bolivar (UTB) in Cartagena, Colombia.
During the preparation of this work, the author(s) used GPT-4 for grammar, spelling, and translation assistance. After using this tool, the author(s) reviewed and edited the content as needed and take full responsibility for the publication’s content. The GitHub repository containing the implementation and resources of this work is available via: • GitHub