Despite the central role of nutrition in our lives, taste has been often classified as an inferior sense in the Western philosophical tradition. This downplayed role is reflected in the vocabulary used to describe the gustatory experience, which, together with smell, is characterized by a scarcity of domain-specific terms [1]. The difficulty in capturing the semantics of taste could help explain why there are few works in the fields of Natural Language Processing (NLP) and Digital Humanities (DH) that deal with this sense and, in particular, the language used to describe its experience. While there has been renewed interest in the automatic extraction of nutrients and ingredients from texts for health and medicinal purpose [2], less attention has been devoted to the development of tools and models focused on capturing the semantics of sensory experiences, especially in a diachronic fashion.
In this paper, we present an English benchmark for the study of gustatory language and a supervised system for the automatic extraction of taste-related events in English, which we trained using this benchmark. The benchmark was built to be a counterpart to the olfactory one presented in [3], with the idea of making the study of the language of these two senses comparable. The system is designed as a means to study the language used to describe the experience of tasting from both synchronic and diachronic perspectives. The selected formal representation for the semantics of taste is based on Frame Semantics [4], and the system is trained to identify the lexical units and the possible semantic roles contributing to the construction of a gustatory event. We present the results of the experiments and an exploration of the benchmark data, aiming to demonstrate the potential of frame-based analysis for sensory studies.
In recent years, there has been a growing interest within the NLP community in developing resources designed to capture the sensory content of language [5]. In particular, in the framework of the three-year European Project "Odeuropa" 1 aimed at preserving intangible cultural heritage, several works have focused on analyzing smell descriptions [6] and extracting olfactory information from texts. For instance, [3] created a manually annotated benchmark with smell events, which has been subsequently used to train a system for olfactory information extraction [7,8]. The benchmark focuses on the language used to describe olfactory experiences and covers a period of four centuries (1600-1900), making it useful for historical research. An extension in this direction is SENSE-LM, a system for extracting sensory information from texts, which shows that combining language models with lexical resource-based approaches yields better results in extracting sensory references from texts compared to systems that do not integrate these two components [9]. The authors were the first to combine sensorimotor representations with the textual features of language models for the task of sensory information extraction in text documents. Even if they propose the system for all the 5 senses, they only tested it on olfactory
The food items that are ingested Quality Any property used to describe the taste (usually adjectives) Taste_Carrier
Anything that can contain the taste source Taster
The person/animal who ingests the food Evoked_Taste
The taste that is evoked but it is not present (e.g., it tastes like onions) Location
The place in which the food is tasted Taste_Modifier An ingredient that can modify the perception of the taste of a taste source Circumstances
The condition or circumstance in which the taste event occurs Effect Any effect provoked by the tasting experience
List of Gustatory Frame Elements and auditory language, using respectively the benchmark of [3] and an artificial dataset they generated with GPT-4 [10]. Most existing work on food representation in the field of NLP focuses on health-related applications. A notable work with a linguistic focus is [2], where the authors concentrate on identifying noun-compound headnouns for developing conversational agents in the e-commerce domain. They propose a supervised approach based on a neural sequence-to-sequence model to identify the most informative token in Italian food compound-nouns, obtaining promising results despite the complexity of the task. Taste has been also addressed from a diachronic point of view in [11], in which the author reconstructs the evolution of food language focusing on the history of some dishes and ingredients across continents using computational linguistic tools. Several studies have developed named-entity recognition (NER) models to automatically extract food entities for medicinal purposes and food science applications [12,13], creating domainspecific corpora by sourcing data from culinary websites and online recipe books [14,15].
The training data we use for the models in this paper is a benchmark created according to the annotation guidelines presented in [16]. The formalization adopted to annotate the benchmark is inspired by Frame Semantics [4] and their implementation through the FrameNet annotation project [17]. In FrameNet, events and situations are constructed as frames, structures that represent the knowledge necessary to understand the meaning of words. Frames include two main components, namely lexical units, domain-specific words or expression that trigger the frame, and frame elements, domain-specific semantic roles usually attached as dependents to the lexical unit. In our case, taste events are captured through a so-called Gustatory frame, which is triggered in a document by Taste_Words (i.e., domain-specific lexical units). Each lexical unit is annotated in the bench-mark together with the frame elements associated with it, which the taste extraction system should then identify automatically. For instance, in the sentence "[Slimy milk]𝑇 𝑎𝑠𝑡𝑒_𝑆𝑜𝑢𝑟𝑐𝑒 has an [unpleasant] 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 taste", the system has to identify the Taste_Word ('taste'), and then the possible frame elements (in this case, Taste_Source and Quality). A list of the possible frame elements and their definition is provided in In Table 2 we report the statistics of the annotated benchmark (note that in [16] we presented only a preliminary version of the benchmark containing around 1,400 Taste_Words). The most frequent frame element is the Taste_Source, followed by Quality and Taste_Modifier, which represent the core frame elements, while the rest of the frame elements are much sparser. Even if the distribution of the frame elements is not balanced, the system is trained to extract the taste words and all the 9 frame elements. Two expert linguists, trained on [16]'s guidelines, annotated three documents from 1670, 1720, and 1920 to assess Inter Annotator Agreement (IAA). The Krippendorff's alpha score [18] at span level was 0.70, indicating a moderate agreement.
It has been observed that words used to describe olfactory and gustatory experiences tend to appear more frequently in emotionally charged contexts and carry a stronger evaluative content compared to words related to other senses [19]. By 'evaluative content', we refer in this paper to the concept of 'emotional valence', which is defined as "the pleasantness of a word in terms of positive and negative meaning" ([1], p. 201). We therefore conducted an exploration of the gustatory benchmark to investigate the positive and negative connotations of gustatory events across different text genres. We perform the same analysis for olfactory events, using the olfactory benchmark of [3] in order to compare the outcome for the two senses. To perform this analysis, we first divide Taste_Words and Smell_Words into positive and negative.
To this purpose, we use the categories proposed in the Historical Thesaurus of English of Savouriness and Unsavouriness for Taste and Fragrant/Fragrance and Stench for Smell 10 . This thesaurus contains almost every recorded word in English from medieval times to the present day, ordered into detailed hierarchies of meaning. In the Thesaurus, every category of the hierarchy is divided per part of speech (PoS). For our analysis, we manually selected all the nouns, adjectives and adverbs used in the period we cover with our documents, namely from 16 th century to 20 th century. We then assigned the words labeled as Taste_Words and Smell_Words in the documents to one of the two categories (positive or negative) and calculated the normalized frequency of each category across different text genres. As reported in Section 3, the genres represented in the gustatory bench- We display the output of this analyses in Fig. 1 (for taste words) and Fig. 2 (for smell words), aimed at showing which emotional valence prevails in each genre for the two senses. We observe that two genres exhibit opposite tendencies: medicine/botany shows a more negative orientation in the smell benchmark and a more positive one in the taste benchmark, whereas travel/ethnography is more positive concerning smell and more negative for taste (see Fig. 1 and Fig. 2, where the light blue refers to negative valencies and the dark blue to positive ones). We then analyzed the most frequent smell / taste sources in the two selected genres to motivate why they exhibit such difference in emotional valence. We notice that smell sources in medicine/botany tend to be common to hospital and disease-related domains having words such as 'urine' and 'fetid bronchitis', while taste sources more easily belong to the realm of common food, with words such as 'almonds' and 'apples'. For what concerns travel/ethnography instead, among the most frequently described taste sources there are exotic and rare foods such as 'coconut' and 'plantain', likely resulting unpleasant to the palates of foreign travelers. Smell sources tend to refer instead to plants, like 'flowers' or 'roots', hence usually pleasant or neutral to the noses of the writers. This analysis of categories and sources' distribution in the genres underlines the importance of a frame-base analysis for understanding and comparing sensory descriptions, in particular their emotional valence.
The benchmark introduced in the previous sections is used to train a classifier whose goal is to detect gustatory information in English texts. The system is based on multi-task learning (Section 5.1), and is then compared with a "single task" classifier, which we consider our baseline (Section 5.2).
To build our system for gustatory information extraction, we adopted a multitask learning approach [20,21], a configuration successfully tested for olfactory information extraction in [7,8]. This approach treats the classification of lexical units and each frame element as different tasks. Additionally, we explored a "single task" classification approach, where both lexical units and frame elements are classified within a multiclass token classification task. The results of these experiments served as a baseline for evaluating the effectiveness of the multitask approach. In both configurations, we employed a transformer-based model fine-tuned for a token classification task [22]. This methodology has proved effective across various NLP tasks, including olfactory information extraction [8] and the extraction of food-related ingredients [13]. We experiment the two configurations with monolingual (English) and multilingual versions of BERT and RoBERTa and with an English historical model, MacBERTh. The models we use are listed below:
-English BERT: bert-base-cased11 [23] -Multilingual BERT (mBERT): bert-base-multilingualcased 12 [23] -English historical model: MacBERTh13 [24] -English RoBERTa: roberta-base14 [25] -Multilingual RoBERTa (RoBERTa xlm): xlmroberta-large15 [26] We fine-tuned each model using the same data, maintaining identical training, validation, and test splits, and evaluated them using 5-fold cross-validation. Each fold contained 80% of the lexical units and their related frame elements for training, 10% for validation (dev), and 10% for testing. These splits were consistent across all configurations and not entirely random. This configuration ensured a balanced distribution of frame elements and comparability in every run. For labeling the data, we adopted the IOB (Inside-Outside-Beginning) labeling format, as used in [7,8]. This method facilitates a comprehensive analysis of sentences and lexical expressions by
Results (F1) of the classifiers on the lexical unit (T_Word) and 9 frame elements with single (italics) and multitask configurations.
The results are the average of the f1 results of each label across the 5 folds.
labeling each token with either Inside, Outside, or Beginning labels as appropriate. To fine-tune the models, we used MaChAmp [27], a specialized toolkit designed for multi-task fine-tuning scenarios. In this approach, each label classification is treated as a distinct task. This setup ensures that simpler tasks, such as recognizing lexical units, contribute as auxiliary tasks to more complex label classifications like "Circumstances" or "Effect" which include entire sentences rather than individual words. MaChAmp enables the choice of different parameters, such as loss weight, epochs and batch size, and we tested different configurations 16 . The results in Table 3 for the multitask approach share the configuration which yielded the best results. The configuration is the same for all the models and it is reported in Appendix A.
Similar to the system for smell information extraction presented in [8], we designed our baseline approach as a single-task multiclass classification, where the model assigns one of 21 possible labels to each token. These labels include 20 representing either "begin" or "inside" of each lexical unit and frame element, and 1 label representing "outside". As we did for the multitask approach, each model is fine-tuned with a token classification head on top 17 . During the training of each model, a hyperparameter search was conducted on the first fold of our data. The search space included learning rates 16,32], and training epochs up to 20, with warmup applied for 10% of the training steps. After determining the optimal hyperparameters for each model, it is fine-tuned 16 Loss weight with different combinations over the labels [1, 0.75], epochs [10,20,30], and batch size [16,32] 17 https://huggingface.co/docs/transformers/tasks/token_ classification five times, each time with a different data fold, and the average scores were computed. We present the results of for the single task approach of each model in italics in Table 3. We observe high performance variations across different frame elements, with the best results obtained for "Quality" and "Taste_Modifier". This is probably due to the fact that their syntactic realization tends to be consistent in the different documents, with "Quality" mainly expressed by adjectives and "Taste_Modifier" by prepositional phrases introduced by with. On the contrary, classification results for "Taste_Source" are quite low despite it being the most frequent FE in the training set, probably because they can be expressed by many different role fillers and syntactic constructions. Upon reviewing the test and prediction results, we find that most mistakes concerning Taste_Source are due to a wrong span extent, for instance the system predicts "the taste of [lollilop]" while the gold standard is "the taste [of lollipop]". This issue is also likely reflected in the inter-annotator agreement (IAA) of the benchmark. In the future, we will consider alternative ways to evaluate text spans beside exact match, for instance by computing the cosine similarity between gold instances and system predictions. Overall, MacBERTh is the best model for Taste_Word detection, but the different FEs are mostly detected with higher accuracy using RoBERTa xlm. For this reason, we plan to adopt this model for our future research on gustatory language.
In this paper, we presented a benchmark for gustatory events containing manually annotated taste-related information, built as a counterpart to the one proposed in [3].
The benchmark is constructed with the same approach adopting a frame-based methodological framework to analyze sensory language. We emphasized the importance of frame-based analysis to capture sensory events by exploring the characterization of positive and negative valence in the benchmarks through the analysis of taste and smell words and sources. The analysis based on frames seems to bring relevant insights into capturing sensory valence from different perspectives, likely supporting the suitability of this approach to deal with humanistic inquiries. We then presented a supervised system to automatically extract taste-related frames, trained on this benchmark. This preliminary exploration and the results obtained with our experiments seem promising for future exploration with automatically extracted data. Indeed, the limited data of the benchmark are not enough to draw relevant conclusions, and for this reason we plan to use our system to extract more data and conduct largescale analyses of the evolution of sensory information over time.
In Table 4, we display the list of lexical units or taste words presented in [16].
The hyperparameter setting for all our models is presented in Table 5. The setting is the default MaChAmp's hyperparameter values, with the addition of loss weights at 1, and 20 epochs of training.
Funded by the European Union under grant agreement 101088548 -TRIFECTA. Views and opinions expressed are however those of the author only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. The authors would also like to thank Marieke Van Erp, the head of the project, for her support.