T) Communicative Intentions Annotation Scheme for Natural Language Generation María Miró Maestre 0 Department of Software and Computing Systems, University of Alicante , 03690 Alicante , Spain 2021 1 9 20

Communicative intentions are one of the linguistic elements that usually determine the content of any text or message we want to express in our communicative interactions. With the purpose of contributing to the improvement of natural language generation systems, so that they can take the communicative intention as one of the starting points that will determine the structure and content of the message generated, the aim of this project is to create a communicative intentions annotation scheme based on the taxonomy presented in the Speech Act Theory. To do so, the scheme will be created with the help of a linguistic corpus and subsequently tested within a natural language generation system. In this way, it will be possible to check up to which point communicative intentions improve the planning stage of the text to be generated automatically, guiding the rest of decisions to be made by the system in order to create automatic messages with more similar results to any manually created text.

 eol>communicative intention annotation scheme speech acts natural language generation pragmatics 1. Introduction and Motivation

One of the main purposes of Natural Language Processing and Generation (NLP and NLG) is to process the elements belonging to each linguistic level in order to create systems that can tag linguistic features automatically, as well as subsequently generate new text with the condition of being natural. However, among the diferent linguistic levels that are processed to generate such messages (phonology, morphology, syntax, semantics, etc.), pragmatics is usually set aside. This is due to the clear preference of processing systems for a progression from the lowest level of linguistic analysis depending on the available resources in each research project, given their easier implementation in the system [ 1 ]. Nevertheless, pragmatics is considered as the linguistic level that studies the meaning of messages always taking into account context [ 2 ], and without which it would be much more dificult to obtain that «natural» condition of a message. This is due to the wide range of (para)linguistic elements that pragmatics considers, such as speakers intentions or their previously shared knowledge, or the sociocultural context in which the message is generated, among other aspects.

Consequently, the study of pragmatics from a computational perspective has become a need that, in spite of the progress made by the research community, with the creation of areas of research such as Computational Pragmatics [ 3 ] or Pragmalinguistics [ 4 ], there is still a long way to go. Arguably, this situation derives from the relatively recent consideration of this level as a discipline [ 5 ] and from the varied definitions we can find of pragmatics depending on the area of research in which we want to focus our study [ 6 ], showing diferent application fields such as clinical pragmatics, neuropragmatics, cultural pragmatics, variational pragmatics, among many others [ 7 ]. Moreover, the diversity of research areas in which pragmatics is starting to be considered nowadays has also fostered the inclusion of pragmatic aspects of language inside some of the most popular tasks in natural language processing, such as sentiment analysis [ 8 ], document summarisation [ 9 ] or rumour detection [ 10 ].

Pragmatics are also present in systems developed with the aim of generating natural text automatically. Indeed, inside the document planning stage, the system takes into account the type of information that needs to be included in the subsequent created text depending on factors such as the target audience or the communicative intention. However, the approaches to this type of systems are usually enclosed in quite specific domains of application [ 11 ]. In fact, a significant proportion of NLG systems are focused in human-robot interaction, so that the system can automatically understand speaker’s intentions [ 12 ], but without teaching the system how to recognise the most appropriate structure of the automatic text depending on the intention we want to determine for it.

Consequently, our motivation for the development of the present study arises from the need of such NLG systems of including pragmatic aspects such as communicative intentions inside the tasks that focus on the structure of the generation system. In this manner, we pretend to test the added value that the inclusion of this linguistic element in the structure of an NLG system could have, generating more linguistically rich text and therefore approximating it to any manually created natural text. To accomplish so, the main tasks to tackle within the present research are the following: • Create an annotation scheme on communicative intentions according to the taxonomy presented in the Speech Act Theory • Apply that annotation scheme to a linguistic corpus belonging to a particular genre to test its performance • Once the annotation scheme is validated, integrate both annotation scheme and the corpus in a NLG system to check up to which point we can enclose the pragmatic content of an automatically generated message by means of determining its communicative intention The remainder of this article is organised as follows: Section 2 focuses on the diferent approaches made in NLP and NLG in order to study pragmatic elements of language, then Section 3 shows the main hypotheses and objectives planned for this research. Subsequently, we explain the methodology proposed for fulfilling each project task in Section 4, and Section 5 sets out the diferent research issues that we may need to face throughout the development of the project. Finally, the bibliographical references used for this study are included at the end of the paper.

 2. Related Work

In spite of the dificulties that the inclusion of pragmatic elements inside NLP and NLG systems entailed, several researchers focused their study on this linguistic level in order to make progress in these research fields of computational linguistics [ 13, 14, 15 ]. Therefore, there is currently a great number of studies enriching their systems with pragmatic knowledge to improve their eficiency, making a very clear division between two diferent tasks. On the one hand we ifnd research devoted to the creation of intelligent interfaces between users and robots with functional conversational aptitudes. The continuous improvement of such systems make these dialogue programmes more complex each time, and therefore need more diferent actions to be fulfilled. Consequently, pragmatic information such as communicative intentions become fundamental in order to not complicate the information exchange [ 15 ].

Some of the researchers who focused on pragmatics applied to this branch of Artificial Intelligence are Trott et al. [ 16 ], who studied how to make a system recognise the intention of the uttered message, as well as superficially analyse human-machine bidirectional communication. As for Budzynska et al. [ 17 ], they based on dialogue argument mining in order to identify illocutionary forces denoting agreement, disagreement and argument with machine learning methods to classify them automatically. Finally, Griol and Callejas [ 18 ] opted for a neuronal network approach to face ambiguity in conversational agents dialogue systems by predicting users intentions.

On the other hand, a very prolific area of research is that devoted to the study of computermediated communication [ 19 ], which comprises all the media included in the Web 2.0, thanks to the communicative interactions it promotes with very varied tools such as Facebook or blog comments, retweets, likes on YouTube and many more. Some of the tasks studied in these types of media are analysing users feelings, just as Tian et al. [ 20 ] did on Facebook based on the idea that emoticons reflect the intention of the message [ 21 ] in order to study the meaning relationship between ‘emojis’ and the text. Inside the area of digital newspapers, Chen et al. [ 22 ] focused on the identification of clickbait cues, which they consider as ‘false news’, by means of several methods of analysis including the syntactic and pragmatic levels. As for Twitter, Saha et al. [ 23 ] and Zhang et al. [ 24 ] made use of the Speech Act Theory (SAT) founded by Austin [ 25 ] and extended by Searle [ 26, 27 ] to identify users’ intentions in their tweets, modifying the intention classification with several linguistic features to apply machine learning algorithms to test their classification accuracy.

Focusing on the previously introduced Speech Act Theory, its founder Austin [ 25 ] defended that language can serve as a means to perform actions depending on the uttered message, investigating verbs to identify which ones are able to denote actions on their own (called performative verbs) and those that only describe reality (descriptive verbs). However, given the main ambiguities of verbs meanings, Austin then focused his research on the elements that comprise the act of saying something, therefore creating the Speech Act Theory. According to this approach, any message includes three dimensions or acts: • locutionary act: the simple act of saying something; • perlocutionary act: the efect of the uttered message in the receiver; • illocutionary act: the actual intention of the message.

Subsequent to this first pragmatic division of every uttered message, Austin moved on to focus on the main role of the illocutionary act in communication, creating a classification divided in 5 types of illocutionary acts depending on the intention of the expressed message. However, many of the linguistic researchers that studied this theory later on took as a basis the taxonomy proposed by Searle [ 26 ], which is a more exhaustive and well delimited modification of Austin’s division according to the principles that lead language and the way we communicate. According to Searle, communicative intentions can be classified in the following five categories: • Assertives: by uttering them, we commit to the veracity of the message expressed. E.g.: declare, manifest, conclude, explain, etc.; • Directives: the speaker uses this type to make the listener do something. E.g.: ask for, dare, invite, command, challenge, etc.; • Commissives: they commit the speaker to do an action in the future. E.g.: swear, promise, commit, intend, etc.; • Expressives: they express the psychological state of the speaker with respect to a topic specified in the message. E.g.: thank, forgive, excuse, congratulate, etc.; • Declarations: when uttering them we get the content of the message to coincide with reality, that is, by using them, the action is performed, or in Searle’s own words: ‘saying makes it so’. E.g.: declare, designate, resign, marry, etc.

Later on, Searle [ 27 ] also made a distinction between the aforementioned types of intentions, which are known as direct speech acts because the relation between the message meaning and the intention is clear, and other type of illocutionary acts called indirect speech acts. In this last type of act, the relation between the message and the intention is not so clearly represented, and some other inferential processes need to be analysed in order to successfully interpret the intention of the message —as in the case of messages containing irony, sarcasm or rhetorical questions, among others—. However, the study of this second type of illocutionary acts was quite set aside by the time it emerged, mainly due to the linguistic and cognitive dificulties implied in the process of correctly identifying the intention of those more subjective messages. On the contrary, the previously exposed classification did attract the interest of the research community in linguistics and many other fields, giving rise to diferent versions of the taxonomy. More concretely, in the NLG field, this classification meant a great starting point for studying the best approach to develop systems that could automatically identify text intentions [ 28, 29 ].

Nowadays, many other research groups have also shown their interest for the SAT taxonomy, focusing here on the annotation and classification of speech acts in diferent fields of study. This is the case of Martínez-Hinarejos et al. [ 30 ] from the Pattern Recognition and Human Language Technology Research Centre at the Universitat Politècnica de València, who used several statistical annotation models such as the N-Gram transducer model to tag dialogue acts in the DIHANA corpus of oral dialogues focused on information related to long-distance trains in Spain.

Also, Caballero et al. [ 31 ], as part of the Centre de Llenguatge i Computació of the Universitat Autónoma de Barcelona, created a pragmatic-functional annotation scheme of the FerroviELE corpus, created with transcribed conversation between the customer service of the Spanish railway company and its customers. This linguistic tool includes an in-depth explanation of the several linguistic tags used in their annotation scheme to annotate 41 diferent communicative functions inspired by the MCER and based on the Vantage Level and EAQUALS Core Inventory descriptors.

Focusing now on clinical pragmatics, Gallardo Paúls and Fernández Urquiza [ 32 ] applied the three main aspects of every uttered message according to the SAT and the classification of illocutionary acts to pragmatically annotate the PerLa corpus, which is focused on clinical oral data to analyse pathological language. To achieve so, the authors extracted several examples of the oral corpora they transcribed in order to tag them with the diferent types of illocutionary acts by delimiting each intention with a particular annotation code easily readable for annotators.

It is in these last examples of pragmatic research that we base our current study, because in spite of the obsolescence that SAT could show nowadays, the recognition of communicative intentions still attracts many research groups and diferent areas inside NLP and NLG. Moreover, these publications show that our research field is leading to the analysis of pragmatic and contextual aspects that also condition language generation. This is given by the general aim of processing a greater number of linguistic nuances to obtain programmes that are able to identify those linguistic particularities to generate texts that can entirely consider the pragmatic nature of language.

 3. Main hypotheses and objectives

The purpose of the present research project is the creation of a communicative intention annotation scheme that could serve as a model for the linguistic annotation of several textual typologies and in diferent languages. By establishing an annotation manual adaptable to diferent research purposes, NLP and NLG systems could be further trained with pragmatic information in order to understand and generate new text depending on the particular intention we want to reflect in such text. Consequently, the improvement of those computational systems with more heterogeneous information will foster the creation of more natural automatically generated text and therefore achieve one of the multiple purposes of these areas of research. In order to carry out the annotation scheme, we need to take into consideration the following research hypotheses: • Up to which point is it possible to unambiguously annotate the main intention of a message according to the taxonomy described by Searle? Will it be suficient to classify intentions in 5 types only? • Can we annotate the 5 main types of intentions in a particular textual genre in a balanced way? Which textual typology could be more adequate for implementing the annotation scheme with such classification? • Given that the language in which we will first validate this annotation scheme is Spanish, could we possibly adapt the annotation scheme to other languages such as English, considering the verbs that denote each intention according to their corresponding equivalents? • Will it be possible to adapt this annotation scheme to an NLG system that creates automatic text on the basis of a particular intention?

 4. Methodology and proposed experiment

The proposed research will be based on the application of the SAT in an annotation scheme that could serve as a model for future NLG systems in order to recognise and select the desired communicative intention we want to implement in the automatically generated message. Consequently, for the purpose of our project we will focus on the illocutionary act of any message and Searle’s classification of direct speech acts as explained in Section 2, as they reflect the intention of the message by means of explicit elements of grammar such as verbs. In this manner, we will set aside indirect speech acts, where the intention depends on more inferential processes recognised thanks to the shared knowledge of the speakers and their context.

After consolidating the theoretical foundations of the SAT, the next methodological steps of the research project will be selecting an appropriate lexicon that contains a considerable representation of the entirety of verbs comprised in the Spanish language to then classify them in the annotation scheme according to the communicative intention that each of them mainly reflects. At the same time, a textual corpus of an adequate length will also be determined depending on the textual genre that we consider most convenient so as to represent a suitable sample of the five intentions we plan to annotate. In order to create a heterogeneous and exhaustive linguistic tool, the most appropriate option will be choosing a corpus previously annotated with linguistic information belonging to other levels of analysis, such as part of speech and syntactic and semantic relations. This will also give us the opportunity to collaborate with other researchers who have previously focused on other linguistic levels of annotation, and therefore widen the possible applications in which the annotation scheme could be employed.

Once that the linguistic tools and the communicative intention classification are established, we will focus our research in the creation of the annotation team, and we will determine the parameters and rules that will need to be taken into account for the intention annotation task of the verbs included in the lexicon. To ensure a clear and simple annotation performance, the annotation scheme will include usage examples of each of the intentions that are being classified, so that, wherever possible, we avoid the ambiguities expected in some of the cases that will be annotated.

Apart from the concrete examples that will be included in each of the intention types to explain and illustrate the usage of each communicative intention, the proposed annotation scheme will also include a final glossary with every annotated verb included in the scheme with its corresponding intention as a result of the previous annotating task. In this way, the annotation scheme will also become a linguistic resource for documentation and references in future studies that want to focus also on the intentions that diferent verbs have and their possible distribution. Moreover, the annotation scheme will also have a technical section devoted to the tags chosen for the annotation task of the research, so that this stage is performed in the most visual and mechanical way possible to create an efective annotation system that can be implemented in an NLG programme.

Finally, we will proceed to the experimentation stage of the intentions scheme in an NLG system once the annotation scheme is already completed. In this manner, we will be able to test to what extent the scheme is capable of correctly identifying the intention chosen for the automatically generated text. Such analysis will help us to verify if this type of pragmatic scheme could serve as a filter of the NLG system when establishing the information we want to represent in the generated text. Apart from this, the present study will include an evaluation of the performance of the language generation system with the intentions scheme to detect problems or modifications that should be made in future projects in order to improve it. Another lines of future research could be focused also on the application of our validated annotation guidelines into other language, as well as analysing which other pragmatic features could be included in the scheme.

 5. Research issues to discuss

As an inherent part of the present study, there are several research questions that will need to be discussed all throughout the development of the annotation scheme proposed: • What type of corpus would be more appropriate for the creation of the annotation scheme? – A generic corpus that could be used in further research projects on intention annotation (from resources such as Twitter or Reddit, for example). – A more delimited corpus from a technical field, in the same manner that other previously created annotation schemes (such as those focused on pathological language, medical reports, etc.). • Is it more appropriate to stick to Searle’s taxonomy of 5 types of intentions so that the annotation scheme could be adapted to diferent research purposes, or would it be more adequate to add other intention typologies so that the recognition system could identify each intention of the corpus more efectively? • What can we consider as a representative sample of the verbs that would be included in the lexicon? This could also depend on the nature of the chosen corpora, depending on the level of colloquial or technical language, where some types of verbs would prevail over others. • In which NLG tasks would it be more interesting to implement the annotation scheme for the experimentation task (generation of news summary, sport reports, narratives...)?

 Acknowledgments

This research work has been funded by the University of Alicante (Spain) and the Ministry of Economic Afairs and Digital Transformation of the Spanish Government through the project INTEGER (RTI2018-094649-B-100).

 [1] C. Cherpas , Natural language processing, pragmatics, and verbal behavior , The Analysis of verbal behavior 10 ( 1992 ) 135 - 147 . doi: 10 .1007/bf03392880. [2] R. Resende de Mendonça , D. Felix de Brito, F. de Franco Rosa, J. C. dos Reis , R. Bonacin , A framework for detecting intentions of criminal acts in social media: A case study on twitter , Information 11 ( 2020 ) 1 - 40 . doi: 10 .3390/info11030154. [3] D. Sayers , R. Sousa-Silva , S. Höhn , L. Ahmedi , K. Allkivi-Metsoja , D. Anastasiou , Š. Beňuš, L. Bowker , E. Bytyçi , A. Catala , et al., The Dawn of the Human-Machine Era: A forecast of new and emerging language technologies ., 2021 . URL: https://hal.archives-ouvertes.fr/ hal-03230287. [4] K. Esenova , F. Ismayilova , Major units in the notion of pragmalinguistics , European Journal of Natural History ( 2018 ) 45 - 52 . [5] W. Bublitz , N. R. Norrick , Foundations of pragmatics, volume 1 , Walter de Gruyter, 2011 . doi: 10 .1515/9783110214260. [6] A. H. Jucker , K. P. Schneider , W. Bublitz (Eds.), Methods in Pragmatics, De Gruyter Mouton, 2018 . doi: 10 .1515/9783110424928. [7] Y. Huang , The Oxford handbook of pragmatics , Oxford University Press, 2017 . [8] T. Mahler , W. Cheung , M. Elsner , D. King , M.-C. de Marnefe , C. Shain , S. Stevens-Guille , M. White , Breaking NLP : Using morphosyntax, semantics, pragmatics and world knowledge to fool sentiment analysis systems , in: Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems , 2017 , pp. 33 - 39 . [9] B. A. Mukhedkar , D. Sakhare , R. Kumar , Pragmatic analysis based document summarization , International Journal of Computer Science and Information Security 14 ( 2016 ) 145 - 149 . [10] A. Kumar , S. R. Sangwan , A. Nayyar , Rumour veracity detection on twitter using particle swarm optimized shallow classifiers , Multimedia Tools and Applications 78 ( 2019 ) 24083 - 24101 . doi: 10 .1007/s11042-019-7398-6. [11] A. Gatt , E. Krahmer, Survey of the state of the art in natural language generation: Core tasks, applications and evaluation , Journal of Artificial Intelligence Research 61 ( 2018 ) 65 - 170 . doi: 10 .1613/jair.5477. [12] K. Garoufi , Planning-based models of natural language generation , Language and Linguistics Compass 8 ( 2014 ) 1 - 10 . doi: 10 .1111/lnc3. 12053 . [13] W. C. Mann , Toward a Speech Act Theory for Natural Language Processing , Technical Report, University of Southern California Marina del Rey Information Science Inst , 1980 . [14] S. C. Herring , D. Stein , T. Virtanen, Introduction to the pragmatics of computer-mediated communication, in: Pragmatics of computer-mediated communication , De Gruyter Mouton, 2013 , pp. 3 - 32 . doi: 10 .1515/9783110214468. [15] C. Bonial , L. Donatelli , M. Abrams , S. Lukin , S. Tratz , M. Marge , R. Artstein , D. Traum , C. Voss , Dialogue-amr: abstract meaning representation for dialogue , in: Proceedings of the 12th Language Resources and Evaluation Conference , 2020 , pp. 684 - 695 . [16] S. Trott , M. Eppe , J. Feldman , Recognizing intention from natural language: clarification dialog and construction grammar , in: Workshop on Communicating Intentions in HumanRobot Interaction , 2016 , pp. 1 - 8 . [17] K. Budzynska , M. Janier , J. Kang , C. Reed , P. Saint-Dizier , M. Stede , O. Yaskorska , Towards argument mining from dialogue , in: Computational Models of Argument , IOS Press, 2014 , pp. 185 - 196 . doi: 10 .3233/978-1- 61499 -436-7-185. [18] D. Griol , Z. Callejas , A neural network approach to intention modeling for user-adapted conversational agents , Computational intelligence and neuroscience 2016 ( 2016 ) 1 - 11 . doi: 10 .1155/ 2016 /8402127. [19] A. Georgakopoulou , Computer-mediated communication , in: J. Verschueren , J.-O. Östman , J. Blommaert , C. Bulcaen (Eds.), Pragmatics in practice, volume 9 , John Benjamins Publishing Co, 2011 , pp. 93 - 110 . [20] Y. Tian , T. Galery , G. Dulcinati , E. Molimpakis , C. Sun , Facebook sentiment: Reactions and emojis , in: Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, Association for Computational Linguistics , 2017 , pp. 11 - 16 . doi: 10 .18653/v1/ W17 -1102. [21] E. Dresner , S. C. Herring , Functions of the nonverbal in CMC: Emoticons and illocutionary force , Communication theory 20 ( 2010 ) 249 - 268 . doi: 10 .1111/j.1468- 2885 . 2010 . 01362 .x. [22] Y. Chen , N. J. Conroy , V. L. Rubin , Misleading online content: recognizing clickbait as "false news" , in: Proceedings of the 2015 ACM on workshop on multimodal deception detection , Association for Computing Machinery , 2015 , pp. 15 - 19 . doi: 10 .1145/2823465. 2823467. [23] T. Saha , S. Saha , P. Bhattacharyya , Tweet act classification: A deep learning based classifier for recognizing speech acts in twitter , in: 2019 International Joint Conference on Neural Networks (IJCNN) , IEEE, 2019 , pp. 1 - 8 . doi: 10 .1109/IJCNN. 2019 . 8851805 . [24] R. Zhang , D. Gao , W. Li , What are tweeters doing: Recognizing speech acts in twitter , in: Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence , 2011 , pp. 86 - 91 . URL: https://www.aaai.org/ocs/index.php/WS/AAAIW11/paper/view/3803. [25] J. L. Austin , How to do things with words , Oxford at the Clarendon Press, 1962 . [26] J. R. Searle , Speech acts: An essay in the philosophy of language , volume 626 , Cambridge University Press, 1969 . [27] J. R. Searle , Expression and meaning: Studies in the theory of speech acts , Cambridge University Press, 1985 . [28] J. F. Allen , C. R. Perrault , Analyzing intention in utterances, Artificial intelligence 15 ( 1980 ) 143 - 178 . doi: 10 .1016/ 0004 - 3702 ( 80 ) 90042 - 9 . [29] G. Briggs , M. Scheutz , A hybrid architectural approach to understanding and appropriately generating indirect speech acts , in: Proceedings of the AAAI Conference on Artificial Intelligence , volume 27 , 2013 , pp. 1213 - 1219 . URL: https://ojs.aaai.org/index.php/AAAI/ article/view/8471. [30] C. D. Martínez-Hinarejos, J. M. Benedí , V. Tamarit , Unsegmented dialogue act annotation and decoding with n-gram transducers , IEEE/ACM Transactions on Audio, Speech, and Language Processing 23 ( 2014 ) 198 - 211 . doi: 10 .1109/TASLP. 2014 . 2377595 . [31] M. Caballero , L. Díaz , M. Taulé , Guía de anotación del corpus FerroviELE , 2014 . [32] B. Gallardo Paúls , M. Fernández Urquiza , Etiquetado pragmático de datos clínicos, e-AESLA ( 2015 ) 1 - 12 .