Communicative intentions are one of the linguistic elements that usually determine the content of any message we want to express. However, regardless of the high precision Natural Language Processing (NLP) systems are acquiring these days, thanks to the revolution derived from the explosion of the latest large language models (LLMs), these architectures still show a lack of appropriate training in order to detect the intention of a message correctly. For the purpose of improving these systems, the present research project aims to create a communicative intention annotation scheme based on the taxonomy presented in the Speech Act Theory. Such resource could help NLP architectures to consider communicative intentions as a starting point to classify any message depending first on the intention it reflects. With this aim, the scheme will be created with the help of an already annotated corpus in Spanish. Subsequently, we will test the scheme within a classification system so that we can verify the accuracy of the intention patterns detected. In this way, it will be possible to check if NLP systems are capable of identifying Spanish communicative intentions or even generate messages that reflect a given intention, therefore enriching the linguistic information these architectures can infer automatically.
Natural Language Processing (NLP) systems, and more concretely Natural Language Generation systems (NLG), are nowadays at their peak due to the evolution that the large language models have shown these last years. Therefore, we currently have at our disposal more and more precise classification and generation systems when it comes to identifying the linguistic patterns demanded in the text to be processed or generated. This is the case of the task of abstractive summary generation in the NLG research branch, or the detection of ofensive or humorous messages if we focus on current NLP tasks. Both represent a few examples of how automatic learning systems are starting to correctly detect and generate more concrete and ambiguous linguistic features each time.
Nevertheless, despite the excellent results that these systems show when detecting linguistic patterns belonging to levels of analysis such as morphology, syntax, and semantics, there is still a long way to refine such architectures so that they can finally capture the natural aspect of a message. One of the elements that mainly helps to define the structure and, generally, the sense of a message is its communicative intention. In fact, two of the main scopes of research that are receiving a lot of attention nowadays in the context of Artificial Intelligence are the tasks of intention detection in the scope of NLP, and conscious text generation with the inclusion of external knowledge in the NLG branch of research.
These tasks are usually oriented normally to creating conversational agents where a human tries to keep a conversation with a robot. Consequently, researchers need to study the linguistic parameters that denote a given communicative intention and the best approach to integrate them into automatically generated responses so that they are created according to the contextual and linguistic requirements of the conversation. However, regardless of the diverse works we also find on intention detection in written textual genres, the scientific output is still reserved for a small percentage of languages, with English at the head.
Consequently, our goal with this research proposal is to create a communicative intention annotation scheme in Spanish to incorporate more semantic-pragmatic information in a NLG system thanks to the linguistic indicators of each intention that we have gathered in our guidelines for the annotation task. Moreover, this initial corpus will serve us as a base for testing several semi-supervised classicfiation techniques to augment the final weight of the corpus. In this way, we will provide the NLP research community with a valuable linguistic resource for identifying more linguistic patterns automatically in a language other than English.
The remainder of this article is organised as follows: Section 2 focuses on the diferent approaches made in NLP in order to tackle the automatic classification of semantic-pragmatic elements of language, then Section 3 shows the main hypotheses and objectives planned for this research. Subsequently, we explain the methodology proposed for fulfilling each project task in Section 4, and Section 5 sets out the diferent research issues we may need to face throughout the experimentation. Finally, the bibliography used for this study is included at the end of the paper.
Regardless of the numerous innovative techniques available nowadays to do research on diferent
linguistic tasks within the scope of NLP, there are well-known traditional linguistic theories
that still shed light on how to approach some of the most dificult NLP tasks to resolve. This is
the case of the Speech Act Theory (SAT) founded by Austin [
Later on, Searle [
Directives the speaker uses this type to make the lis- ask for, dare, invite, command, chaltener do something lenge, etc.
Commissives they commit the speaker to do an action swear, promise, commit, intend, etc.
in the future Expressives they express the psychological state of the thank, forgive, excuse, congratulate, speaker with respect to a topic specified in etc.
the message Declaratives when uttering them we get the content of declare, designate, resign, marry, the message to coincide with reality, that etc. is, the action is performed, or in Searle’s own words: ‘saying makes it so’ known as direct speech acts because the relation between the meaning and the intention of the message is straightforward, and other type of illocutionary acts called indirect speech acts. In the latter, the relation between the message and the intention requires other inferential processes (i.e., cultural references, social context, etc.) to successfully interpret the intention of the message, as in those texts containing irony, sarcasm or rhetorical questions, among others.
Despite the dificulties that the inclusion of pragmatic elements inside NLP and NLG systems
entailed, several studies focused on this linguistic level to make progress in these domains of
computational linguistics [
A very prolific area of research is that devoted to the study of computer-mediated
communication (CMC) [
Nevertheless, it is clear that most research on this subject handles English documents so
far. However, we can still find a few examples of works where Spanish speech acts are used to
either improve task-oriented dialogues as in Martínez-Hinarejos et al. [
Because of the numerous NLP scenarios in which we can make use of the SAT nowadays, our research proposal is based on the creation of a communicative intention annotation scheme in Spanish to use it as a resource to solve some of the most currently studied tasks in our field. In this way, we will try to improve the ability of current language models when identifying more semantic-pragmatic aspects of language to successfully reproduce them. More concretely, the main research questions that support this project are: • RQ 1) Which linguistic features can help us detect the intention of a given message in the
Spanish language? • RQ 2) Is it possible to identify those linguistic features in a CMC corpus, given its colloquial style and lexical variety? • RQ 3) Can a language model learn to diferentiate between diferent types of intentions with a training dataset, regardless of the ambiguities inherent to language? • RQ 4) Can both sentiment analysis and emotion detection help to identify the communicative intention of a message with better precision? • RQ 5) Does the automatic annotation of communicative intentions benefit NLP applications such as an automatic text generator?
To integrate communicative intentions in some NLP current tasks to enrich systems with further semantic-pragmatic information, we will focus on Searle’s classification of direct speech acts as explained in Section 2 and other linguistic features that also reflect the intention of the message straightforwardly. To create the corresponding annotation scheme, and to test its validity in an NLP application, several linguistic resources and computing tools were used to complete each of the experimentations that shape our research project: 1. Corpus creation with the Shared Task on Hope Speech Detection for Equality, Diversity and Inclusion [19] and UMUCorpusClassifier [20] The lack of suficient datasets in languages other than English forces researchers to either modify those existing corpora to accomplish the objective of their research work or create their own resources so that they can analyse language concentrating on particular linguistic phenomena. For our research, we first examined the corpus compiled for the Shared Task on Hope Speech Detection for Equality, Diversity, and Inclusion [19], but as it was focused on the task of hope speech detection, we didn’t find enough results to create the first version of our corpus. Consequently, we completed the selection of tweets we found with intention indicators in this corpus with a compilation of tweets extracted through the Twitter API thanks to the extracting tool UMUCorpusClassifier [20]. By combining both resources, we were able to compile a corpus of Spanish tweets about the LGTBIQ+ community. The final amount of tweets is 454, which gives us a corpus of 996 instances to analyse, as we decided to tag the intention of each of the utterances comprised in the same tweet, not the tweet as a whole. We made this decision after noticing that diferent utterances of the same tweet can show linguistic patterns linked to diferent intentions, so we preferred to separate the tweets in their utterances to not confuse the recognition of a given intention.
Parallel to the corpus creation, we compiled linguistic patterns linked to a particular intention according to the SAT classification we explained previously in Section 2. To this end, several resources were also of help to gather the best representation -within our means- of the Spanish linguistic structures that reflect an intention when used appropriately. On the one hand, we translated the verb lexicon comprised in [21] to get a Spanish equivalent of the verbs that, according to Austin and Searle, reflect a particular intention. This book provides in-detailed semantic descriptions of around 200 of the most frequent speech act verbs used in English. In this way, by studying the semantic particularities of each English verb, we could look for the equivalent verbs in Spanish that kept each semantic nuance so that the speech act verb classification would not difer from one language to another. On the other hand, we revisited two of the most extendedly used grammar references of the Spanish language to study their approach to detect speech acts through grammatical features in Spanish [22, 23, 24, 25].
The linguistic tool used to annotate our corpus of tweets was the platform created for
semantic annotation with intelligent assistance INCEpTION [26]. Thanks to its intuitive
interface, this platform allows users to individually manage, curate and modify annotation
projects in the same environment by assigning each project to the corresponding
annotators. In our case, two experts in the field of Spanish linguistics served as the
annotators for our corpus together with the author. As previously mentioned, the tweets
were uploaded in the platform and annotated utterance by utterance, as shown in Figure
1, so we could identify as many intention indicators as possible even within the same
tweet. Once the annotation task was completed, several metrics were calculated in
order to check the inter-annotator agreement achieved between the three annotators.
INCEpTION also includes a section that calculates both Fleiss’ Kappa and Krippendorf’s
Kappa automatically in the annotation project, so we checked for the results and
confirmed that our annotation scheme could be validated, thanks to achieving a 0.77 of
agreement with both measures.
4. Proposed experiment A): enriching our corpus while training the classification
system through active learning
Once we validated our annotation scheme, another task we currently study is exploiting
the "active learning" functionality that the annotation tool INCEpTION includes within
its platform. This machine learning method consists in training the classification system
with the instances we have manually annotated. Then, once we add new instances to be
automatically annotated by it, we check which examples the recommender can annotate
correctly or not. Those more dificult examples would be the ones to annotate manually
so that the recommender system keeps learning on those more ambiguous examples until
it finally classifies well those dificult tags without manual help. Consequently, with this
technique we will both boost the classifier performance and augment the final weight of
our corpus.
5. Proposed experiment B): combining SAT with sentiments and emotions
The second experiment to be fulfilled during our research is combining the identification
of communicative intentions in tweets with the tasks of sentiment analysis and emotion
detection. Following [
Given the suggestions and comments received in the previous editions of the Doctoral Symposium, we solved some of the research issues we came up with through the development of our study. However, as an inherent part of this project, new research questions arise that need to be discussed to ensure a good research quality that provides new knowledge within our research area in Spanish: • Should we add indirect speech act examples to our corpus to check if the classification system is capable of correctly detecting the semantic diferences between direct and indirect speech acts? • Do we have enough intention indicators so that the automatic classification system is capable of diferentiating between the diferent types of intentions we included in our scheme? • Would it be possible to apply our annotation scheme to other textual typologies outside
CMC? • What if we try to test LLMs ability to generate sentences with a given intention, to check whether there are inconsistencies regarding their intention classification, or if they could be of help to find even more intent linguistic patterns in Spanish?
This research work is part of the R&D project "CORTEX: Conscious Natural Text Generation" (PID2021-123956OB-I00), funded by MCIN/AEI/10.13039/501100011033/ and by “ERDF A way of making Europe”. [15] M. Caballero, L. Díaz, M. Taulé, Guía de anotación del corpus FerroviELE, 2014. [16] B. Gallardo Paúls, M. Fernández Urquiza, Etiquetado pragmático de datos clínicos, e-AESLA (2015) 1–12. [17] D. Pascual, Speech acts in travel blogs: Users’corpus-driven pragmatic intentions and discursive realisations, ELIA: Estudios de Lingüística Inglesa Aplicada (2021) 85–123. [18] S. Ridao Rodrigo, Actos de habla en redes sociales: perfiles privados versus perfiles públicos,
Literatura y lingüística (2021) 429–446. [19] B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, S. C. Navaneethakrishnan, J. P.
McCrae, M. Á. García-Cumbreras, S. M. Jiménez-Zafra, R. Valencia-García, Shared task on hope speech detection for equality, diversity, and inclusion - ACL, 2022. URL: https: //competitions.codalab.org/competitions/36393#learn_the_details-organizers. [20] J. A. García-Díaz, Á. Almela, G. Alcaraz-Mármol, R. Valencia-García, UMUCorpusClassifier: Compilation and evaluation of linguistic corpus for natural language processing tasks, Procesamiento del Lenguaje Natural 65 (2020) 139–142. [21] A. Wierzbicka, English Speech Act Verbs: A Semantic Dictionary, Academic Press, 1987. [22] R. A. Española, et al., Nueva gramática de la lengua española, volume 2, Espasa Madrid, 2009. [23] V. Demonte, Gramática descriptiva de la lengua española: Sintaxis básica de las clases de palabras, volume 1, Espasa, 1999. [24] V. Demonte, Gramática descriptiva de la lengua española: Las construcciones sintácticas fundamentales, volume 2, Espasa, 1999. [25] I. Bosque, Gramática descriptiva de la lengua española: Entre la oración y el discurso.
Morfología, volume 3, Espasa, 1999. [26] J.-C. Klie, M. Bugert, B. Boullosa, R. E. de Castilho, I. Gurevych, The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation, in: Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, Association for Computational Linguistics, 2018, pp. 5–9. URL: http: //tubiblio.ulb.tu-darmstadt.de/106270/.