Tools and Techniques for Opinion Mining Arminda Guerra Lopes 1,2, Luís Fonseca 1 1 Polytechnic Institute of Castelo Branco, Address, Castelo Branco, Portugal 2 ITI, LARSyS, Universidade da Madeira, Campus Universitário da Penteada, 9020-105, Funchal, Portugal Abstract The use of forums and social networks facilitates the exchange of cultural, social, and scientific comments. In the analysis of large loads of text, opinion mining enables numerous alternatives by reducing labour requirements. Opinion mining and tools facilitate the identification of the sentiment expressed in each message without requiring human intervention. There are platforms that provide sentiment analysis. The purpose of this paper is to provide an overview of the state of the art on this subject, the most relevant analysis techniques, and the results obtained. SentiStrength, IMDb, and SERENE were chosen as tools. The results of each tool application will be compared at the end of the analysis. Keywords 1 Opinion Mining, Sentiment Analysis, Emotions, Natural Language Processing, IMDb; SentiStrength, SERENE, Artificial Intelligence. 1. Introduction Emotions are essential to successful and effective human-to-human communication. Adoption of new technology depends on user acceptance. An opinion mining tool can help companies gain insight into how their customers feel about a brand, whether they are positive, negative, or neutral. One of the most important ways to keep customers engaged is to monitor their brands, including sentiment analysis. Emotion AI (also known as Artificial Intelligence) or Opinion Mining is a method of extracting feelings and emotions from a text. This is the methodology for systematically identifying, extracting, quantifying, and studying affective states by utilizing natural language processing, text analysis, computational linguistics, and biometrics. The purpose of opinion mining is to identify the emotional tone behind a body of text. In organizations, this is a popular method for determining and categorizing opinions about products, services, or ideas. The authors of this paper use the terms interchangeably. Sentiment Analysis provides insights into social media sentiment, brand experience, patient experience, customer satisfaction, multilingual insights, news trend analysis, and real-time sentiment insights. The process of opinion mining can be approached in a variety of ways. In large-scale sentiment analysis, traditional machine learning methods such as Naïve Bayes, Logistic Regression, and Support Vector Machines (SVM) are widely used because of their scalability. There are four main steps in opinion mining: Data collection - the data that will be analyzed; Text cleaning - the data can be processed and prepared for analysis through text cleaning tools; Analyzing the data; Understanding the results. The purpose of this paper is to present current knowledge about opinion mining tools, based on a literature review, and to identify how their use can affect the environment where they are used. They are also briefly discussed in terms of their advantages. IS-EUD 2023: 9th International Symposium on End-User Development, 6-8 June 2023, Cagliari, Italy EMAIL: aglopes@ipcb.pt (Arminda Guerra Lopes); luisffonseca94@gmail.com (Luís Fonseca) ORCID: 0000-0002-1969-9937 (Arminda Guerra Lopes) © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2. Conceptual Context In using information technologies to seek out and understand others' opinions, new opportunities and challenges arise. Natural Language Processing combines computational linguistics, machine learning, and deep learning models to model human language rules. In combination with these technologies, computers can analyze human language, whether it is text or audio data, and can comprehend the speaker's or writer's intent and mood. The process of understanding, interpreting, and changing human language is known as natural language processing, or NLP. One of the most common applications of NLP is opinion mining. The method identifies and extracts views based on spoken or written language. The term sentiment analysis is also used to describe it. 2.1 An overview of opinion mining applications Opinion mining is a branch of natural language processing that aims to recognize and extract opinions from a given text, which includes blogs, reviews, social media, forums, and news to get feedback from customers. A new product is also evaluated after it has been sold to discover what the public thinks of it. When a product attribute (interface, user experience, functionality) is searched for using opinion mining models, the exact information that is needed can be obtained. An audience's perception of a product can help determine its success. Is there anything that needs to be improved about the product? In addition to customer service firms, sentiment analysis is frequently used to automatically categorize incoming calls as "urgent" and "not urgent." Using machine learning to automate the customer service industry, it has become more important than ever to understand consumers' feelings. Because of this, businesses are increasingly using chatbots based on natural language processing. 2.2 A new approach to natural language processing using opinion mining In opinion mining, unstructured text can be turned into structured data using natural language processing and open-source tools. The Twitter platform, for instance, is a rich source of feelings, with individuals sharing their thoughts and opinions. People often attribute the most importance to social media comments based on moods (positive or negative). Customer emotions give a more comprehensive picture of what influences their decision- making process and, in some circumstances, even dictates it. Therefore, sentiment analysis based on emotion is a valuable use of Natural Language Processing. A sophisticated social media monitoring strategy paired with NLP for speech analysis can help companies analyze consumer emotions and respond appropriately. Among the examples of NLP for sentiment analytics are search results - search engines use natural language processing (NLP) to surface relevant results according to similar search patterns or user intent, making it easy for anyone to find what they're looking for; Language translations - Many languages do not allow direct translation and have different sentence structure orderings. In the early days of NLP online, email filters were used to filter emails and write grammatically correct replies. Spam filters began by looking for certain terms or phrases that indicated spam. Email classification in Gmail is one of the most frequent, newer uses of NLP. The Sentiment Analytics method allows one to determine the feelings and emotions experienced by a viewer. The experience of emotions is not included. A broad response is identified and analyzed by contrasting positive and negative experiences. The use of both Emotion Analysis and Sentiment Analysis can transform the way people react to new products. Since emotion and sentiment analysis have similar advantages and functions, people tend to use them interchangeably. Despite this, there are significant differences between these two systems. In contrast to sentiment analytics, which categorizes customer opinions into negative, positive, or neutral, emotion analysis provides insight into the motivations and emotional blocks of customers. Emotion analysis is beyond the scope of this paper. 3. A Study In this section, we present the analysis of papers relevant to our research. The focus for this analysis is the data or the results obtained through the usage of opinion mining tools, the databases where the research was conducted, and the results found. The focus for the research was opinion mining with data gathering methods and several algorithms for an all-encompassing state of the art review, as well as the measurements used for sentiment analysis in wherever a sentimental value could be obtained. The authors’ choice for research papers was in two different databases: ACM Digital Library (ACM Digital Library, n.d.)[1] and IEEEXplore (IEEE Xplore, n.d.), [2]. It was found a huge number of results, the most relevant were selected considering not only the title but also the paper’s date. Authors chose the recent ones. After the selection of the papers found in the initial search, the titles and abstracts were analyzed to choose the most relevant. For this selection the papers had to include some sort of opinion mining/sentiment analysis (text or other) and include the description of the algorithms used to obtain the narrated results described in each paper. A total of 10 papers was selected through this method, mostly related to comments in web platforms with most of them being about movie review platforms as it seems to be the most common use case according to our search. There were also two other papers added to this list, one from 2013 and the other from 2022 that use a tool (SERENE) relevant to the search. 4. Results The papers are organized in two tables by publication date, where each row represents one paper, with its title and citation, followed by the authors' names and a brief description of its contents (table 1 and 2). Table 1 Data Source 1 Title Authors Description Improving sentiment rating of movie The authors of this paper propose a mathematical approach to Jenq-Haur Wang, Ting- review comments for recommendation sentiment analysis by using not only the sentiment for Wei Liu (Wang & Liu, 2017) comments but also the ratings and the number of likes. This project aims to improve the rating system of a Chinese Comprehensive Rating Model of Douban movie platform (Douban) that at the time used only the average Movie based on Sentiment Analysis (Gu & Yi-Ran Gu, Yu-Yu Su user rating, the authors propose using sentiment analysis on the Su, 2017) platform’s comments to improve the rating by considering the general opinion. Prediction of Movie Sentiment Based on This work focuses on the Rotten Tomatoes website and uses Suhariyanto, Ari Reviews and Score on Rotten Tomatoes SentiWordNet, a sentiment analysis tool, attempting to improve Firmanto, Riyanarto Using SentiWordnet (Suhariyanto et al., what the platform’s present system predicts for a given movie’s Sarno 2018) acceptance. Swastika Pandey, A Technique to Handle Negation in This paper is focused on reducing the impact negative words Santwana Sagnika, Sentiment Analysis on Movie Reviews have on sentiment analysis, which according to the authors Bhabani Shankar (Pandey et al., 2018) constantly results in bad sentiment analysis. Prasad Mishra Using machine learning classification algorithms for sentiment A Sentiment analysis approach through Tanushree Dholpuria, analysis on comments, this paper compares the obtained results deep learning for a movie review Yuvraj Krishna Rana, with analysis made using neural networks. (Dholpuria et al., 2018) Chetan Agrawal A Word Vector based Review Vector This paper proposes a model where words are added to Yin Fulian, Wang method for Sentiment Analysis of Movie different dimensions, the algorithm then predicts to which Yanyan, Pan Xingyi, Reviews Exploring the Applicability of the dimension an analyzed word belongs to, the sentiment analysis Su Pei Movie Reviews (Yin et al., 2018) is done afterwards, using machine learning algorithms. Table 2 Data Source 2 Title Authors Description Predicting Success of a Movie from Heena Timani, Parag Shah, Mansi This paper aims to develop a system capable of Youtube Trailer Comments using Joshi predicting a movie’s success according to the sentiment Sentiment Analysis (Timani et al., analysis scores on comments for that same movie. 2019) Movie Recommendation System Nimish Kapoor, Saurav Vishal, The work in this paper considers most users don’t have Using NLP Tools (Kapoor et al., Krishnaveni K. S. time to read other users’ reviews and comments, 2020) therefore the authors suggest a system capable of analyzing sentiment to improve the accuracy of the overall ratings on movies. Sentiment Analysis for Predicting A.M.Rajeswari, M.Mahalakshmi, This project aims to use sentiment analysis on various Customer Reviews using a Hybrid R.Nithyashree, G.Nalini comments including movie reviews, product reviews and Approach (Rajeswari et al., 2020) tweets using SentiWordNet and other machine learning algorithms. A Combination of Machine Sayed Akram Saadat Neshan, Reza For this paper, the authors aim to improve the results of Learning and Lexicon Based Akbari existing sentiment analysis tools like SentiStrength and Techniques for Sentiment Analysis SentiWordNet using machine learning. (Neshan & Akbari, 2020) Integrating Multimodal Information Wasifur Rahman, Md. Kamrul The work in this paper intends to use parameters apart in Large Pretrained Transformers Hasan, Sangwu Lee, Amir Zadeh, from textual values for sentiment analysis, for this (Rahman et al., 2020) Chengfeng Mao, Louis-Philippe purpose the authors use two state-of-the-art datasets and Morency, Ehsan Hoque develop a method capable of detecting sentiment more accurately. Interplay between AI and HCI for Giuseppe Desolda, Andrea Esposito, The authors use SERENE, a web sentiment analysis tool UX evaluation: the SERENE case Rosa Lanzilotti, Maria Francesca to better the user experience with data gathered from the study (Desolda et al., 2022) Costabile actions taken by users on platforms. Context-Aware Multimodal Emotion Aaishwarya Khalane, Talal Shaikh For this paper the authors develop a multimodal system Recognition (Khalane & Shaikh, that uses textual, acoustic, and visual information on the 2022) CMU-MOSEI dataset, according to the authors, the method developed has a higher accuracy than the models in the dataset. As shown in the tables, each of the papers in the previous subsection was analyzed thoroughly and the data collection, algorithms, techniques, or software used for either data collection or sentiment analysis are presented in the same order. The authors use data from Chinese platforms Douban and iQiYi to improve sentiment ratings of movie review comments for recommendation purposes [3]. The authors do not provide a description of their data collection method; however, they use sentiment analysis techniques on the comments from these platforms. As well as heuristics that change the approach to sentiment analysis, the authors present an appropriate lexicon for platforms' language, along with words to describe cast parts instead of the film, as well as popular words like “binge watch”. By analyzing the top 10 movies on the platform, the authors compare five different methods. Overall accuracy was not high, but one of the methods had around 80% accuracy. In a 2017 study, crawling tools to gather information from the Chinese movie platform Douban was used to develop a comprehensive rating model of Douban movies based on sentiment analysis [4]. The paper proposes a rating model that includes comment sentiment as well as, ratings from 1 to 5. To improve this model, the authors suggest removing some comments based on the metadata, such as comments before the premiere, since they can be biased by social media. Another suggestion would be to highlight the comments that have been most liked. In comparison to the platform's method, the work developed for this paper shows a more accurate rating on all tests. Suhariyanto et al. predict movie sentiment using Rotten Tomatoes reviews and scores by using SentiWordnet [5] crawl tools directly from Rotten Tomatoes [6]. The authors propose further processing after storing the gathered data to improve sentiment analysis by removing stop words (and, or ...). To try to approach the result to the actual movie score, they combined SentiWordNet with expert reviews to determine whether comments were overwhelmingly positive or negative. Based on the expert review scores, the method developed was much more accurate than SentiWordNet. An online movie review sentiment analysis based on 100 random comments using a public database of movie reviews uses a technique to handle negative comments [7]. SentiWordNet is used in this work, but negation prefixes are treated to improve results. According to the authors, prefixes such as "anti" can change the meaning of a word and could negatively affect the result if not identified by the algorithm. According to the authors, despite not using the same test data, their algorithm has a notably higher accuracy than similar algorithms without prefixes. Using deep learning for sentiment analysis, authors analyzed movie comments on a public dataset which, according to the authors, had 3000 comments [8]. The purpose of this paper is to classify each comment according to its sentiment (good, bad, great, decent). A deep learning algorithm and a machine learning algorithm are then trained using these data sets. As a result of using this technique, the lowest accuracy achieved was 98%. A Word Vector Based Review Vector Method for Sentiment Analysis of Movie Reviews Exploring the Applicability of Movie Reviews [9] uses comments from IMDb, with no information about how the data was obtained, citing only the size of the dataset (25000 comments). They aim to use machine learning to classify words within a variable size vector that is generated by an algorithm generating dimensions. According to the article, the method presented in the article displays good results when compared to traditional sentiment analysis tools. Despite having an 87% accuracy, the authors consider their method to be an improvement, given that it helps in a few languages, despite not having the best accuracy. It was shown that the authors could predict the success of a movie based on the opinions expressed in comments on Youtube trailers using sentiment analysis [10]. This work makes use of an API that allows them to collect data, which are comments on video trailers. A random sample of 20 trailers from 2016 was chosen. For sentiment analysis, various Python libraries are used to find positives and negatives in the comments. A linear regression equation is then used to predict how successful the movies were in terms of box office revenue. The results of this paper demonstrate that the trailer comments can be used to predict the success of a movie. Using NLP tools, for movies based on the title, genre, rating, cast, revenue, and comments of the public dataset TMDb (The Movie Database) was proposed [11]. Using only common sentiment analysis words, such as good or bad, this work performs a comparative sentiment analysis. The authors report that the algorithm has an accuracy rate of 66.7%, which is then combined with the rest of the data for a final accuracy of 70%, which indicates the importance of full text analysis when it comes to sentiment analysis. Using a hybrid approach [12] examines three different datasets: tweets, movie reviews, and product reviews. Several datasets have been obtained from Kaggle, including the tweets and movie comments, and the product comments from an obsolete repository. They trained a machine learning model using four different techniques based on the obtained sentiment value from SentiWordNet. A comparison is then made between the classifications in the different platforms and the method developed by the authors. According to the authors, their solution uses sentiment analysis values to better classify neutral reviews. In a Combination of Machine Learning and Lexicon Based Techniques for Sentiment Analysis, [13] use an IMDb dataset containing movie reviews as well as an Amazon dataset containing product reviews. Both datasets contain human-classified reviews as either positive or negative. SentiStrength and SentiWordNet are used by the authors for generating numeric values for sentiment used in training machine-learning algorithms. The results of these algorithms are then combined into a meta-classifier to improve their individual performance. According to the authors, using the meta-classifier does not result in a significant improvement in accuracy. In Rahman et al. [14] the authors differ from the majority of sentiment analysis research by incorporating nonverbal information into their classification methods, according to the authors, who tested their developed methodology on state-of-the-art datasets (CMU-MOSI and CMU-MOSEI) using acoustic and visual parameters as well as text transcribed from the videos. Based on multimodal parameters, the proposed technique is effective in fine-tuning sentiment. The SERENE case study [15] interplay between AI and HCI for UX evaluation: SERENE (User Experience Detector) is a tool that, when implemented in web platforms, can detect user actions, and classify them based on their emotion, resulting in a heatmap displaying where users express certain emotions. The project aims to improve user experience using this data. This study shows sentiment analysis goes beyond text analysis, despite focusing on human-computer interaction. Khalan & Shaikh refers to the development of an algorithm suitable for recognizing emotions, in which textual, visual, and acoustic data is incorporated [16]. This is like the methodology used in another study reviewed in this article, but with the addition of analyzing the data that surrounds the analyzed data timewise, we obtained better results than those obtained from the models developed for the dataset (CMU-MOSEI). 5. Discussion In our first demonstration of sentiment analysis, we used movie comments gathered from IMDb to show how it works on text. The whole process is described using SentiStrength data collection and sentiment analysis. In this demonstration, Python was used, which allows for the addition of external libraries. A second demonstration of SERENE's functionality is displayed using publicly available papers in the context of user experience. SentiStrength Most sentiment analysis projects use data from publicly available datasets and data science projects. IMDb has an ongoing user interaction, so based on current data analysis, for this paper, the data are taken from the IMDb API (API for IMDb, TMDb, Wikipedia and More - IMDb API, n.d.). The API allows access to public movie data such as titles, ratings, and comments [17]. Figure 1 shows some of the endpoints available for data collection on the IMDb API page. For access to this API, users must register and obtain a key to use when requesting data. Only two endpoints were used in this demonstration, one returning the movie ID (unique identifier) from the title and one returning movie reviews given the movie ID. The reviews contain both the comments and ratings each user gave the movie, but the rest has been removed as it isn't necessary for this demonstration. As soon as the data has been collected, SentiStrength (SentiStrength - Sentiment Strength Detection in Short Texts - Sentiment Analysis, Opinion Mining, n.d.) will be used for sentiment analysis on the comments. Using this tool, a given sentence is split into words and analyzed for sentiment. By comparing each word to a set of words associated with a particular sentiment, this analysis is performed. After asking the user for a title, the program gets the movie's ID from the API, which is then used to retrieve the reviews. The sentiment value of each comment is then calculated using SentiStrength, which ranges between -5 and 5. Figure 1: IMDb API page (endpoints). The process of removing unnecessary data and organizing the data that is needed is a list of all the comments, the rating given by the user, and the SentiStrength sentiment value. In addition, the sentiments of all the comments were tested for accuracy. The program returns useful information, such as the sentiment value calculated by SentiStrength paired with the comment's rating. Any movie title, such as Jaws, can be used in this way. According to the result, sentiment values are very different from actual ratings given by users. Although there are a variety of sentiment analysis tools, with differing results, similar steps must be taken, from data gathering to tool use, to achieve the same results. SERENE To display sentiment analysis in fields other than text, SERENE uses which is an online tool for user experience sentiment analysis. During the data collection process, this tool gathers data itself if it is implemented on the platform. Whenever an action is taken by a user on a page, it is recorded to be analyzed later [15]. With the SERENE dashboard, a heatmap can be created based on the data gathered through user interaction. Figure 2: SERENE dashboard (example from Esposito et al., 2022) Using SERENE on a shopping page, figure 2 shows a heatmap, with colors indicating how strong a different sentiment (in this case Joy) is analyzed. The figure shows a high level of Joy around the shopping cart icon derived from users using SERENE. The results of this analysis can then be used by a user experience expert to suggest changes. 5.1 AI Support in Everyday Life – a Snapshot Using machine learning (ML) and other techniques in the background, Artificial Intelligence (AI) is being used in many areas of daily life. It is everywhere in our lives that artificial intelligence can be found, whether it is reading our emails, receiving driving directions, or suggesting music or movies. As society moves forward with the digital revolution, software and devices powered by artificial intelligence and machine learning (AI and ML) emulate human thought processes. A system powered by artificial intelligence recognizes its surroundings, handles what it sees, resolves issues, and assists with chores to improve the quality of daily life. Most people regularly check their social media accounts, including Facebook, Twitter, Instagram, and others. Besides customizing feeds, artificial intelligence also detects and eliminates false news. As an example, Deep Learning enables Facebook to extract value from a growing number of its unstructured data sets. Although the autonomous vehicle market is still in its early stages, there are already enough prototypes and pilot projects to suggest that such vehicles will become more common as artificial intelligence and IoT (Internet of Things) technologies become more advanced. The use of virtual assistants such as Siri, Cortana, Google Assistant, and others has simplified people’s lives. Throughout the process, they have become a friend to people, reminding them to pick up a package and telling them jokes. This software is capable of recognizing speech patterns and provides natural language processing capabilities. By monitoring working hours, screen time, and other relevant data, it is also possible to obtain information about the user. Artificial intelligence enables it to learn and listen as if it were a human. Streaming services such as music and video are also excellent examples of artificial intelligence. Based on people’s preferences, these platforms provide suggestions. Due to artificial intelligence (AI) technologies such as machine learning, online shopping is becoming more personalized and streamlined for consumers. Commercial enterprises can improve their logistics management with the help of AI-powered automated warehouses and supply chain management systems. Furthermore, sentiment analysis allows them to better understand and respond to their consumers' needs and behavior. Uber, for instance, provide people with a vehicle nearly every time they require one, making them extremely useful. With the aid of deep learning technology, these apps can identify people routine behavior. A great deal of fun can be had with the email communication system. The unwelcome emails are immediately filtered out and categorized as spam or non-urgent. During the creation of a new email, the software suggests possible replies. Additionally, some email systems provide users with a notification when it is time to submit their messages. For all these useful features to be available, artificial intelligence is required. By increasing our productivity and helping us focus on actual problems, artificial intelligence is already transforming our lives. The actuation field also encompasses games, medical applications, autocorrect texts, recipes and cooking, smart homes, etc. Moreover, artificial intelligence technology will continue to grow, expand, and become increasingly important to all industries and virtually every aspect of our everyday lives. This paper presented the state of the art for opinion mining, an application of AI in daily life. 6. Conclusions Integration with Systems Modeling Language (SysML) will enable teams to collaborate more efficiently by providing a common language and process to distribute models. In systems engineering, the Human Systems Integration component can represent behaviors, constraints, states, and goals throughout the entire lifecycle of the system. Although sentiment analysis measures sentiment, an otherwise impossible task for a machine, these techniques can be applied to a range of applications, from comment analysis to the analysis of webpages. With the improvement of these tools, not only will systems be able to have automated ratings, but developers will also be able to receive user feedback on what they should improve. Since comment-based platforms have grown in popularity, averaging user ratings might not be the most efficient method of rating. The IMDb movie database rates movies out of 10 and allows comments as well. Using sentiment analysis on comments might improve the overall rating of movies in movie databases or products in online stores, allowing users to better understand the quality of the product they are seeking information about. SERENE, for example, can measure sentiment in other fields besides comment analysis, such as user input, and improve the interface based on this sentiment for a better user experience. With tools that take data directly from user input, sentiment analysis can be a powerful tool to improve user experience, as well as text analysis, as comments on web platforms with many unread comments are common. A better user experience will also result from using the sentiment value of every comment to improve the rating accuracy. As shown in the usage of the SentiStrength tool, sentiment analysis can be very valuable for the field of user experience, but the results may not be very accurate. The accuracy issue could be addressed by using various tools to bring the results closer to reality. Overall, opinion mining focuses on extracting, classifying, understanding, and assessing opinions expressed in news reports, social media comments, and user-generated content. Often, online text is subjected to sentiment analysis to identify sentiment, affect, subjectivity, and other emotions. Moreover, with the Systems Modeling Language (SysML) extension, referred to as the "Human- Agent Teaming Modeling Language," and a companion method, known as the "Human-Agent Teaming Design Method," both of which are useful extensions of SysML, humans and artificial agents can work together in teams. Acknowledgements This research was funded by LARSyS (Projeto - UIDB/50009/2020). 7. References [1] IEEE Xplore. (n.d.). Retrieved March 23, 2022, from https://ieeexplore.ieee.org/Xplore/home.jsp [2] ACM Digital Library, https://www.acm.org › publications [3] Wang, J.-H., & Liu, T.-W. (2017). Improving sentiment rating of movie review comments for recommendation. 2017 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW), 433–434. https://doi.org/10.1109/ICCE-China.2017.7991181 [4] Gu, Y.-R., & Su, Y.-Y. (2017). Comprehensive Rating Model of Douban Movie Based on Sentiment Analysis. 2017 International Conference on Network and Information Systems for Computers (ICNISC), 111–116. https://doi.org/10.1109/ICNISC.2017.00032 [5] SentiWordNet, in Baccianella, S., Esuli, A., & Sebastiani, F. SENTIWORDNET 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. [6] Suhariyanto, Firmanto, A., & Sarno, R. (2018). Prediction of Movie Sentiment Based on Reviews and Score on Rotten Tomatoes Using SentiWordnet. 2018 International Seminar on Application for Technology of Information and Communication, 202–206. https://doi.org/10.1109/ISEMANTIC.2018.8549704 [7] Pandey, S., Sagnika, S., & Mishra, B. S. P. (2018). A Technique to Handle Negation in Sentiment Analysis on Movie Reviews. 2018 International Conference on Communication and Signal Processing (ICCSP), 737–743. https://doi.org/10.1109/ICCSP.2018.8524421 [8] Tan, K. L., Lee, C. P., & Lim, K. M. (2023). A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research. Applied Sciences, 13(7), 4550. [9] Yin, F., Wang, Y., Pan, X., & Su, P. (2018). A Word Vector Based Review Vector Method for Sentiment Analysis of Movie Reviews Exploring the Applicability of the Movie Reviews. 2018 3rd International Conference on Computational Intelligence and Applications (ICCIA), 112–117. https://doi.org/10.1109/ICCIA.2018.00028 [10] Timani, H., Shah, P., & Joshi, M. (2019). Predicting Success of a Movie from Youtube Trailer Comments using Sentiment Analysis. 2019 6th International Conference on Computing for Sustainable Global Development (INDIACom), 584–586. [11] Kapoor, N., Vishal, S., & K. S., K. (2020). Movie Recommendation System Using NLP Tools. 2020 5th International Conference on Communication and Electronics Systems (ICCES), 883–888. https://doi.org/10.1109/ICCES48766.2020.9137993 [12] Rajeswari, A. M., Mahalakshmi, M., Nithyashree, R., & Nalini, G. (2020). Sentiment Analysis for Predicting Customer Reviews using a Hybrid Approach. 2020 Advanced Computing and Communication Technologies for High Performance Applications (ACCTHPA), 200–205. https://doi.org/10.1109/ACCTHPA49271.2020.9213236 [13] Neshan, S. A. S., & Akbari, R. (2020). A Combination of Machine Learning and Lexicon Based Techniques for Sentiment Analysis. 2020 6th International Conference on Web Research, ICWR 2020, 8–14. https://doi.org/10.1109/ICWR49608.2020.9122298 [14] Rahman, W., Hasan, M. K., Lee, S., Bagher Zadeh, A., Mao, C., Morency, L.-P., & Hoque, E. (2020). Integrating Multimodal Information in Large Pretrained Transformers. Proceedings of the Conference. Association for Computational Linguistics. Meeting, 2020, 2359. https://doi.org/10.18653/V1/2020.ACL-MAIN.214 [15] Esposito, A., Desolda, G., Lanzilotti, R., & Costabile, M. F. (2022). SERENE: a Web platform for the UX semi-automatic evaluation of website. ACM International Conference Proceeding Series. https://doi.org/10.1145/3531073.3534464. [16] Khalane, A., & Shaikh, T. (2022). Context-Aware Multimodal Emotion Recognition. Lecture Notes in Networks and Systems, 350, 51–61. https://doi.org/10.1007/978-981-16-7618-5_5/COVER [17] SentiStrength - sentiment strength detection in short texts - sentiment analysis, opinion mining. (n.d.). Retrieved April 8, 2022, from http://sentistrength.wlv.ac.uk/