This paper shows the Arandanito team proposal for the Rest-Mex 2023 and examines the role of verbs, nouns, and adjectives in analyzing tourist opinions in Spanish. We employ sentiment analysis techniques on a large corpus of tourist opinions collected from online platforms. Our findings reveal that nouns play a significant role in classifying polarity, type, and country in tourist opinions. Surprisingly, verbs do not have the expected importance, while adjectives prove to be more influential. These insights contribute to our understanding of sentiment analysis in the tourism domain and have implications for related research.
The tourism industry plays a significant role in the economy of many countries around the
world [
In recent years, sentiment analysis has emerged as a popular technique for analyzing the
opinions expressed in social media and online reviews [
One aspect of sentiment analysis that has received less attention in the tourism literature is
the role of diferent parts of speech in shaping tourist opinions. Verbs, nouns, and adjectives
are the fundamental building blocks of language, and they play a crucial role in shaping the
meaning and sentiment of the text [
In this paper, the main aim is to examine the role of verbs, nouns, and adjectives in shaping
tourist opinions in Spanish. Our study is based on the Rest-Mex 2023 corpus [
We start by presenting an overview of the literature on sentiment analysis in tourism and the role of diferent parts of speech in shaping opinions. We then describe the data and methodology used in our study, including the corpus of reviews, the NLP tools used for analysis, and the statistical techniques employed. We present our results, which include an analysis of the most frequent verbs, nouns, and adjectives used in the reviews, as well as the sentiment associated with these words. Finally, we discuss the implications of our findings for tourism marketing and management and identify areas for future research.
Sentiment analysis has been widely used in the tourism industry to understand tourist opinions
and preferences [
Verbs, in particular, are essential in expressing opinions and attitudes toward specific actions
or events. In tourism, they are commonly used to describe the experiences and activities of
tourists, as well as the performance of tourism-related services [
Nouns, on the other hand, are essential in describing the objects and entities that are the focus of tourist experiences. In tourism, nouns are often used to describe the features of tourist destinations, such as landmarks, natural attractions, and cultural heritage sites. Nouns can also be used to describe the services and amenities ofered by tourism providers, such as hotels, restaurants, and transportation. For example, nouns such as ”beach,” ”museum,” ”hotel,” and ”restaurant” can be used to convey positive or negative sentiments towards specific aspects of a tourist experience [14].
Adjectives are also critical in expressing opinions and attitudes towards specific features or attributes of tourist destinations and experiences. Adjectives are commonly used to describe the physical features and qualities of tourist destinations, as well as the quality of tourism-related services. For example, adjectives such as ”beautiful,” ”clean,” ”luxurious,” and ”afordable” can be used to convey positive or negative sentiments towards specific aspects of a tourist experience [15].
While previous studies have primarily focused on the analysis of sentiment expressed through adjectives in tourism, there is increasing evidence that other parts of speech, including verbs and nouns, play a significant role in shaping tourist opinions. The relative importance of diferent parts of speech may vary depending on the type of tourist experience, highlighting the need for further research in this area.
The Rest-Mex 2023 organizers have compiled a training collection consisting of 251,702 opinions from TripAdvisor, categorized into three labels: 1. Polarity 2. Type 3. Country
The polarity classification includes five classes, where class 1 denotes the most negative polarity, and class 5 denotes the most positive polarity. The distribution of these classes is shown in Table 1, which reveals a clear imbalance.
There are three classes to classify the type of place: Attractive, Hotel, and Restaurant. The distribution of this trait is illustrated in Table 2. Although there is no marked imbalance as seen for polarity, the table shows that there is still some imbalance.
The classification of the country of origin of the visited place is based on three classes: Mexico, Cuba, and Colombia. Table 3 shows the distribution of this trait.
The methodology proposed in this study comprises three crucial steps. Firstly, a data subsampling approach is applied. Secondly, the text is transformed by filtering the Part of Speech tags for the experiments. Lastly, the data is classified. Each of these three steps is elaborated below.
In [16], the dificulty of working with unbalanced data is mentioned. In the case of the Rest-Mex corpus, there is a clear imbalance that could afect the results. To attack this problem we are going to make a selection of instances to try to balance the classes with polarity 3, 4, and 5 with respect to those with more negative polarity.
For this, the percentage was taken for each class > 3 , where
In this way, for each class , in the training corpus, percentage of the total instances is randomly chosen.
defined.
In order to be able to observe the performance of each part of speech, the first thing to do is obtain the parts of speech of each opinion. Then, the parts of the sentence of interest must be
Thus, it is possible to extract from each text the words with the POS label of interest. In order to generalize the texts, it is proposed to obtain their lemma from these words of interest.
In summary, the following function is proposed: = ( ( , )) Where is a text from the collection, is a list with the POS labels of interest, function that returns the words of T whose POS label coincides with one in the list of Finally, is a function that returns its lemmas from a list of words. is a .
For this work, the following values of
are proposed: • • • • = [ ] = [ ] = [ ] = [ , , ]
To perform sentiment analysis, we utilize a classifier based on BERT, specifically employing the Beto-cased model. BERT (Bidirectional Encoder Representations from Transformers) is a highly capable pre-trained language model known for its outstanding performance across various natural language processing tasks.
Model: Our choice is the Beto-cased model, a variant of BERT that is trained specifically on Spanish text. This model captures detailed information and retains word casing, which enhances its contextual understanding capabilities.
Max Length: In order to handle input sequences eficiently, we set a maximum sequence length of 32 tokens. If an input exceeds this limit, it is either truncated or segmented into smaller parts following BERT’s tokenization scheme.
Optimizer: For training the deep neural networks, we employ the Adam optimizer, a popular choice known for combining adaptive learning rates with momentum. This optimizer enables eficient optimization and convergence during the training process.
Learning Rate: The learning rate is set to 5 × 10−5, a commonly used value for fine-tuning BERT models. This value strikes a balance between achieving convergence at an optimal pace and fine-grained optimization.
Steps: The step size, also referred to as epsilon ( ), is set to 1 × 10−8. This parameter controls the level of noise introduced during the learning rate update, ensuring stability throughout the training process.
Epochs: Our classifier undergoes training for 2 epochs, where each epoch represents a complete iteration over the entire training dataset. This decision balances the model’s learning capacity with the available computational resources.
By leveraging BERT-based models with these specific configurations, our objective is to harness the contextual representation capabilities of BERT for precise sentiment analysis of Spanish text. The chosen settings establish a robust foundation for training and optimizing the classifier.
To test the models on the train partition, it is proposed to make a separation of 70% for training the models, while the remaining 30% will be used for evaluation.
Table 4 shows the F-measure results for Polarity, Type, and Country. [ ] [ ] [ ]
When considering individual part-of-speech tags, the best performance is observed for the Type trait when using [ ] as the value, with an 1 score of 0.9293. This suggests that nouns play a significant role in determining the type of place in the sentiment analysis.
On the other hand, using [ ] as the value yields the lowest performance across all three traits. This indicates that verbs might not provide strong discriminative features for sentiment analysis and classification of type and country. When using a combination of all three part-of-speech tags ([ , , ] ), the best performance is observed for Polarity and Country, with 1 scores of 0.4791 and 0.7134, respectively. This implies that a combination of diferent parts of speech can improve the classification results, particularly for polarity and country identification. Overall, these results suggest that considering multiple part-of-speech tags, including adjectives, nouns, and verbs, can enhance the performance of sentiment analysis in terms of polarity, type, and country classification.
For this edition, the organizers of Rest-Mex propose evaluation metrics that give greater weight to the correct classification of negative polarity classes.
To assess the efectiveness of the polarity classifier, the organizers propose Equation 1. This metric assigns an additive inverse of importance based on the percentage of instances in a class in the test collection.
() = () + () + ()
3 () = ∑|=| 1 ((1 − ) ∗ ())
∑|=| 1 1 −
To evaluate the Type and Country traits, they propose Equations 2 and 3. These metrics represent the macro F-measures of each trait.
() = () + () + () (3) 3
Finally, to obtain a single value per participant, they propose a combination of the results as indicated by Equation 4. It is important to note that polarity result is given more weight than the other two traits.
() = 2 × () + () +
() 4 Table 5 shows the test F-measure results for Polarity, Type and Country.
The () result, which combines the performance of all three traits, is highest for the [ , , ] combination, with a value of 0.6392. This indicates that considering multiple part-of-speech tags leads to improved overall performance in sentiment analysis. When considering individual part-of-speech tags, the [ ] combination achieves the highest 1 score for Type, with a value of 0.9280. This suggests that nouns play a crucial role in determining (1) (2) (4) the type of the place in sentiment analysis. On the other hand, [ ] yields the lowest performance across all three traits, indicating that verbs might not provide strong discriminative features for sentiment analysis and classification of type and country. Comparing the final result with the individual 1 scores, it is evident that the final result gives more weight to Type, as it has the highest impact on the combined metric. This suggests that accurately classifying the type of the place is crucial for achieving a high overall performance in sentiment analysis. Overall, these results demonstrate that considering multiple part-of-speech tags, including adjectives, nouns, and verbs, leads to improved performance in sentiment analysis, particularly in the classification of type and country. The combination of these tags results in a more comprehensive understanding of the sentiment expressed in the text.
In conclusion, this methodology provides evidence that nouns play a significantly more important role in classifying polarity, type, and country in sentiment analysis.
The prominence of nouns in classification can be attributed to the fact that both countries and types of places are directly mentioned in the opinions expressed by tourists. Common nouns such as ”hotels,” ”restaurants,” ”Mexico,” ”Cuba,” ”monuments,” and ”museums” are likely to appear frequently in the texts. Therefore, nouns carry valuable information for accurately determining the sentiment, type, and country associated with a given text.
Interestingly, nouns also exhibit interesting results in the classification of polarity. While it might be expected that verbs, representing actions, would have a greater impact, the findings show that verbs do not have the expected importance. Adjectives, on the other hand, appear to be more influential than verbs in sentiment classification, which aligns with the notion that descriptive terms hold significant sentiment-related information.
It is worth noting that with this methodology, the Arandanito team achieved 14th place in the Rest-Mex 2023 competition, indicating its efectiveness and competitiveness in sentiment analysis tasks improving the baselines.
In summary, this approach highlights the crucial role of nouns in accurately classifying polarity, type, and country. The unexpected lower importance of verbs and the relatively higher significance of adjectives contribute to a deeper understanding of the sentiment expressed in tourist texts. Science (2022). [12] M. Á. Álvarez-Carmona, E. Villatoro-Tello, L. Villaseñor-Pineda, M. Montes-y Gómez, Classifying the social media author profile through a multimodal representation, in: Intelligent Technologies: Concepts, Applications, and Future Directions, Springer, 2022, pp. 57–81. [13] C. Nicholls, F. Song, Improving sentiment analysis with part-of-speech weighting, in: 2009 International Conference on Machine Learning and Cybernetics, volume 3, IEEE, 2009, pp. 1592–1597. [14] E. Kouloumpis, T. Wilson, J. Moore, Twitter sentiment analysis: The good the bad and the omg!, in: Proceedings of the international AAAI conference on web and social media, volume 5, 2011, pp. 538–541. [15] W.-H. Khong, L.-K. Soon, H.-N. Goh, S.-C. Haw, Leveraging part-of-speech tagging for sentiment analysis in short texts and regular texts, in: Semantic Technology: 8th Joint International Conference, JIST 2018, Awaji, Japan, November 26–28, 2018, Proceedings 8, Springer, 2018, pp. 182–197. [16] M. Á. Álvarez-Carmona, R. Aranda, A. Y. Rodríguez-González, L. Pellegrin, H. Carlos, Classifying the mexican epidemiological semaphore colour from the covid-19 text spanish news, Journal of Information Science (2022). doi:https://doi.org/10.1177/ 01655515221100952.