Sentiment analysis is a key technology for companies and institutions to gauge public opinion on products, services or events. However, for large-scale sentiment analysis to be accessible to entities with modest computational resources, it needs to be performed in a resource-eficient way. While some eficient sentiment analysis systems exist, they tend to apply shallow heuristics, which do not take into account syntactic phenomena that can radically change sentiment. Conversely, alternatives that take syntax into account are computationally expensive. The SALSA project, funded by the European Research Council under a Proof-of-Concept Grant, aims to leverage recently-developed fast syntactic parsing techniques to build sentiment analysis systems that are lightweight and eficient, while still providing accuracy and explainability through the explicit use of syntax. We intend our approaches to be the backbone of a working product of interest for SMEs to use in production.
focusing on the scientific part of the project. Section 4 describes the planning and the project team, with a more global focus that includes the transfer parts as well. it’s not at all good” and “This phone is good, and it’s not Our proposal is, thus, to use this new parsing paradigm to at all expensive”: both sentences actually have opposite create eficient syntax-guided sentiment analysis systems sentiment polarities, but the heuristics cannot correctly that can operate at a large scale with modest computacapture the scope of negation and its influence on how tional resources. This will democratize accurate largethe sentiment should be interpreted. scale sentiment analysis technology, putting it within
An alternative is to use approaches that take syntac- reach of smaller institutions and companies that cannot
tic stucture into account. Syntax can be injected into aford to deploy slow parsers or language models.
a sentiment analysis model either explicitly, by using a
parser [
a breakthrough achieved by the previous ERC-funded FASTPARSE project [7] to build sentiment analysis systems that are eficient and, at the same time, use explicit syntax to improve accuracy and provide explainability. The FASTPARSE project had the goal to substantially improve the speed of syntactic parsing, and was a resounding success, advancing the state of the art in both parsing speed and accuracy several times, and achieving accurate parsers that improved speed by an order or magnitude over previous approaches.
In particular, within the FASTPARSE project, we deifned a new syntactic parsing paradigm that casts parsing as a sequence labeling task by defining encodings that can represent syntax trees by means of discrete labels associated with words. This was shown to work for the two main families of grammatical formalisms, with encodings that proved viable for constituency parsing [9] and dependency parsing [10], and new encodings have kept being defined since then that further push parsing accuracy [11, 12].
For the purposes of this project, the approach to parsing as sequence labeling has two especially relevant advantages:
associated with words, a standard representation that is easy to feed to downstream tasks, be it as features [13] or by using multitask learning [14].
sentences per second on a standard consumer GPU [9, 10, 15]. (b) Fine-grained sentiment analysis, often called aspect-based sentiment analysis [16]: in this task, the goal is not to obtain a single value for the whole text, but individual opinions about the different targets that may appear in the message (for example, the text “I really like this phone’s camera, but the battery life is not acceptable” contains both a highly positive opinion about the camera and a negative one about the battery).
For this purpose, we will explore three approaches to
integrate fast syntactic parsing into solving these two
sentiment analysis tasks:
• Rule-based pipeline approach: we start by using
a fast dependency parser to obtain the syntactic
tree for an input sentence. We then use
syntaxbased rules and a polarity lexicon to navigate
the tree, assigning polarity values to its
syntactic components (addressing subtask (b) above).
These values can then be integrated by the rules
to obtain an overall polarity for the whole
sentence (addressing subtask (a)). The syntax-based
rules are specifically designed to address syntactic
phenomena that influence opinion polarity, such
as negation (for instance, in the example above,
"not acceptable" indicates a negative sentiment
because the negation alters the positive meaning
of "acceptable"), intensification (the phrase "really
like" signifies a stronger preference than merely
"like"), and adversatives (where "but" juxtaposes
a positive statement against a negative one). It
is worth noting that this kind of approach have
been applied in the past, particularly for subtask
(a) (see [
Each of the described approaches will be evaluated on freely-available sentiment analysis corpora. We will take a multilingual focus, ensuring that our systems work on a diverse range of languages (for the dependency parsing part, this is ensured by the use of Universal Dependencies [17]). For the initial value proposition, we will focus on Spanish and English, since our main target is the local Spanish market. However, the core technology should be available to easily adapt to more languages when needed.
In particular, a good starting point when it comes to datasets are the SemEval 2022 Task 10 corpora [18], as they cover five diferent languages and their annotation includes individual polarity targets, thus supporting both our subtasks (a) and (b). Given our focus on the Spanish market, we are also working with a corpus of TripAdvisor reviews in Spanish, from Rest-Mex 2023 [19]. This also allows us to evaluate performance in a noisy context. Some preliminary results of the rfist (rule-based pipeline) approach on the latter corpus can be found in [20].
The first one addresses the scientific research part of the project, and hence it is the one we have addressed in Sections 2 and 3. The other two correspond to the innovation and technology transfer parts of the project. In particular, the work packages are as follows: • WP1: Technical and Performance Validation. This involves three tasks: – Validation of fast parsers for polarity classification: the main objective is to adapt the parsers developed in the FASTPARSE project to work on polarity classification tasks. To achieve this, we have created linguistic rules that, when integrated with a parser’s output, allow us to determine the polarity of sentences by analyzing the words and syntactic structures in the text [20]. Additionally, we are exploring a purely machine learning-based method that does not rely on these rules. This method involves training a parser capable of sequence labeling in conjunction with a sequence model that uses discrete tags to represent sentence polarity, employing a multitask learning framework. The success of this task is assessed through the validation of various models’ ability to classify polarity, as evidenced by performance metrics. – Validation of fast parsers for fine-grained sentiment analysis: in this task, we aim to go beyond polarity classification by employing syntactic parsers for more detailed sentiment analysis. The goal is to analyze sentiments at a more granular level, capturing specific opinions within a sentence.
This involves recognizing how a single sentence might simultaneously provide positive feedback on one aspect of a product while criticizing another. To achieve this, we will adapt the approaches based on linguistic rules and on multitask learning to process fine-grained information: the rules will be used to infer sentiment information for smaller linguistic units apart from the whole sentence, and a sequence labeling encoding will be used to represent finegrained sentiment information for the multitask setup. In addition, we will try a third setup where the fine-grained sentiment information will be represented as a tree, so that syntactic parsers can learn to output it directly. – Cross-domain generalization. The performance of sentiment analysis models trained on a given domain (e.g. restaurant reviews) often degrades when they are used in a diferent domain (e.g. movie reviews). Since our project is targeted primarily towards making the technology ac– Market Analysis. We have performed a study to identify current sentiment analysis industry trends, market need and market size at the European level. This study, which was subcontracted to an external consultancy company, includes the potential market niches for our sentiment analysis models, identification of comparative technologies and information on potential user needs. – Validation of value proposition. This involves conducting interviews, as well as Design Thinking workshops, to identify target user needs and validate our proposition. – Commercial viability: product-market fit.
This task consists in involving potential users (identified in the previous tasks) in iterative testing and validation of the developed models. • WP3: Pre-Technology Transfer. This includes two tasks: – Creating SALSA business model canvas.
We will refine our business model canvas through a process of continuous improvement. This will involve defining, enhancing, and validating our value proposition, primary revenue streams, and cost structures, alongside identifying the crucial partners, customers, channels, and business relationships that form the core components of an open source business model. We will assess the viability of establishing a spin-of to ofer service agreements related to SALSA. – Building a business environment. The aim of this task is to establish contact with po
AAAI/article/view/6446. doi:https://doi.org/ How to best use syntax in semantic role labelling, 10.1609/aaai.v34i05.6446. in: A. Korhonen, D. Traum, L. Màrquez (Eds.), Pro[6] E. Strubell, A. Ganesh, A. McCallum, Energy and ceedings of the 57th Annual Meeting of the Associapolicy considerations for deep learning in NLP, in: tion for Computational Linguistics, Association for A. Korhonen, D. Traum, L. Màrquez (Eds.), Proceed- Computational Linguistics, Florence, Italy, 2019, pp. ings of the 57th Annual Meeting of the Associa- 5338–5343. URL: https://aclanthology.org/P19-1529. tion for Computational Linguistics, Association for doi:10.18653/v1/P19-1529.
Computational Linguistics, Florence, Italy, 2019, pp. [14] M. Strzyz, D. Vilares, C. Gómez-Rodríguez, Se3645–3650. URL: https://aclanthology.org/P19-1355. quence labeling parsing by learning across repredoi:10.18653/v1/P19-1355. sentations, in: Proceedings of the 57th Annual [7] C. Gómez-Rodríguez, Towards fast natural lan- Meeting of the Association for Computational Linguage parsing: FASTPARSE ERC Starting Grant, guistics, Association for Computational Linguistics, Procesamiento del Lenguaje Natural 59 (2017) 121– Florence, Italy, 2019, pp. 5350–5357. URL: https: 124. //www.aclweb.org/anthology/P19-1531. [8] C. Gómez-Rodríguez, I. Alonso-Alonso, D. Vi- [15] M. Anderson, C. Gómez-Rodríguez, A modest lares, How important is syntactic parsing Pareto optimisation analysis of dependency parsers accuracy? An empirical evaluation on rule- in 2021, in: S. Oepen, K. Sagae, R. Tsarfaty, based sentiment analysis, Artificial Intelli- G. Bouma, D. Seddah, D. Zeman (Eds.), Proceedgence Review 52 (2019) 2081–2097. URL: https:// ings of the 17th International Conference on Parsdoi.org/10.1007/s10462-017-9584-0. doi:10.1007/ ing Technologies and the IWPT 2021 Shared Task s10462-017-9584-0. on Parsing into Enhanced Universal Dependen[9] C. Gómez-Rodríguez, D. Vilares, Constituent pars- cies (IWPT 2021), Association for Computational ing as sequence labeling, in: Proceedings of the Linguistics, Online, 2021, pp. 119–130. URL: https: 2018 Conference on Empirical Methods in Nat- //aclanthology.org/2021.iwpt-1.12. doi:10.18653/ ural Language Processing, Association for Com- v1/2021.iwpt-1.12. putational Linguistics, 2018, pp. 1314–1324. URL: [16] G. Brauwers, F. Frasincar, A survey on aspecthttp://aclweb.org/anthology/D18-1162. based sentiment classification, ACM Comput. Surv. [10] M. Strzyz, D. Vilares, C. Gómez-Rodríguez, Vi- 55 (2022). URL: https://doi.org/10.1145/3503044. able dependency parsing as sequence labeling, in: doi:10.1145/3503044.
Proceedings of the 2019 Conference of the North [17] D. Zeman, et al., Universal Dependencies 2.12, American Chapter of the Association for Compu- 2023. URL: http://hdl.handle.net/11234/1-5150, tational Linguistics: Human Language Technolo- LINDAT/CLARIAH-CZ digital library at the gies, Volume 1 (Long and Short Papers), Associa- Institute of Formal and Applied Linguistics (ÚFAL), tion for Computational Linguistics, Minneapolis, Faculty of Mathematics and Physics, Charles Minnesota, 2019, pp. 717–723. URL: https://www. University.
aclweb.org/anthology/N19-1077. [18] J. Barnes, L. Oberlaender, E. Troiano, A. Kutu[11] A. Amini, T. Liu, R. Cotterell, Hexatagging: Projec- zov, J. Buchmann, R. Agerri, L. Øvrelid, E. Velltive dependency parsing as tagging, in: A. Rogers, dal, SemEval 2022 task 10: Structured sentiJ. Boyd-Graber, N. Okazaki (Eds.), Proceedings ment analysis, in: G. Emerson, N. Schluter, of the 61st Annual Meeting of the Association G. Stanovsky, R. Kumar, A. Palmer, N. Schneider, for Computational Linguistics (Volume 2: Short S. Singh, S. Ratan (Eds.), Proceedings of the 16th Papers), Association for Computational Linguis- International Workshop on Semantic Evaluation tics, Toronto, Canada, 2023, pp. 1453–1464. URL: (SemEval-2022), Association for Computational Linhttps://aclanthology.org/2023.acl-short.124. doi:10. guistics, Seattle, United States, 2022, pp. 1280–1295. 18653/v1/2023.acl-short.124. URL: https://aclanthology.org/2022.semeval-1.180. [12] C. Gómez-Rodríguez, D. Roca, D. Vilares, 4 and doi:10.18653/v1/2022.semeval-1.180. 7-bit labeling for projective and non-projective [19] M. Ángel Álvarez-Carmona y Ángel Díaz-Pacheco dependency trees, in: H. Bouamor, J. Pino, y Ramón Aranda y Ansel Yoan Rodríguez-González K. Bali (Eds.), Proceedings of the 2023 Conference y Victor Muñiz-Sánchez y Adrián Pastor Lópezon Empirical Methods in Natural Language Pro- Monroy y Fernando Sánchez-Vega y Lázaro Bustiocessing, Association for Computational Linguis- Martínez, Overview of rest-mex at iberlef 2023: tics, Singapore, 2023, pp. 6375–6384. URL: https: Research on sentiment analysis task for mexican //aclanthology.org/2023.emnlp-main.393. doi:10. tourist texts, Procesamiento del Lenguaje Natural 18653/v1/2023.emnlp-main.393. 71 (2023) 425–436. URL: http://journal.sepln.org/ [13] Y. Wang, M. Johnson, S. Wan, Y. Sun, W. Wang, sepln/ojs/ojs/index.php/pln/article/view/6572. [20] O. Kellert, M. U. Zaman, N. H. Matlis, C. Gómez
Rodríguez, Experimenting with ud adaptation of an unsupervised rule-based approach for sentiment analysis of Mexican tourist texts, in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2023), volume 3946 of CEUR Workshop Proceedings, Jaén, Spain, 2023. URL: https: //ceur-ws.org/Vol-3496/restmex-paper15.pdf.