Since the seminal work of John Austin [
In the last decades linguists and psycholinguists have faced the problem of how speakers recognize the kind of speech act (i.e. the illocutionary force) performed by a speaker uttering a sentence in a particular context, investigating the role played by the illocutionary force-indicating devices (IFIDs).
An IFID is a linguistic element that indicates or delimits the
illocutionary force of an utterance [
Research in linguistics and psycholinguistics has up to now produced a
rich literature focused strictly on the linguistic IFIDs. Nonetheless, it is
commonly recognized that the comprehension of a speech act depends by
nonverbal illocutionary force-indicators devices too [
The general purpose of this paper is thus to take a step in the direction of an experimental investigation of the non-verbal makers of illocutionary force.
In the field of multimodal communication, several psychological and linguistic studies have revealed the importance of the different functions played by non-verbal signals in verbal interactions, in particular, with a peculiar attention on the role played by facial expressions in language interpretation.
Face as a whole, besides the study of specific parts of the face, has been
considered mainly with respect to the expression of emotions, since the
seminal work of Ekman & Friesen [
In our experiments, we have focused our attention on the upper face AUs only.
In the present work we have focused our attention on three basic, prototypical types of direct speech acts, that correspond to three special syntactic types occurring in most of the world’s languages (most of the current languages reveal separate syntactic constructions to distinguish them): assertion, question and orders.
RQ1: what are the AUs or combinations of AUs jointly produced with the execution of speech acts conveying different illocutionary forces;
RQ2: which AUs or combinations of AUs are associated with different illocutionary forces in language comprehension;
RQ3: to what extent different AUs or combinations of AUs are recognized as compatible indicators of different kinds of illocutionary forces.
The experiment has been run in Italian with 18 participants (9 males, 9 females) each. We considered in total 27 possible sentences (15 target + 12 nonsensical sentences as fillers). Each sentence was presented to the parti cipants conveying one of three different illocutionary forces – i.e. assertion, question, or order. While different participants were presented each sentence with a different illocutionary force, its propositional content was fixed across the three conditions. The illocutionary force of each sentence was linguistically expressed by the use of an explicit performative verb. For example, the list expressing the proposition “Marco studia” (lit. Eng. tr. “Marco studies”) was:
(3) ITALIAN ENGLISH TRANSLATION Assertion: So che Marco studia Assertion: I know that Marco studies Question: Una domanda: Marco studia? Question: A question: does Marco study? Order: È un ordine: Marco studia!! Order: This is an order: “Marco study!”
The task consisted in reading and pronouncing each sentence. During the performance participants were video recorded. Our dependent variable was the realization of the Upper AUs in the three different conditions.
Results (Fig. 1) revealed that questions were significantly produced in combination with AU4, AU7, and in combination AU4 + AU7. Orders were associated with AU2. No AU or AU combination was specifically associated with assertions.
Fig 1 Forest plots showing odds ratio for the presence of each AU and combination of AU in combination with the illocutionary forces under test The experiment has been run with 86 Italian native-speakers (32 males, 53 females – Age M = 23,65, SD = 4.41) each. We have created in total 15 target lists of sentences (plus 5 filler lists), each one constituted by 5 sentences expressing the very same propositional content conveyed by three target illocutionary forces (i.e. assertion, question, order), plus 2 filler illocutions (see Fig. 2). We created pictures of the upper face with 9 different kinds of AUs and combinations of AUs (i.e. AU0 or Neutral, AU2, AU4, AU5, AU7, AU1+2, AU1+4, AU4+5, AU4+7), corresponding to different experimental conditions, using 5 different actors, so that in total we produced 45 pictures combining AUs and actors.
Fig 2 The sample of AUs and combinations of AUs used in the comprehension experiment The study was conducted online by Limesurvey 2.05+. Participants were presented a picture representing a certain AU along with 5 alternative sentences, each with the same propositional content but with different illocutionary forces. The task consisted in selecting among the list of 5 sentences the one that best matched the picture. Our dependent variable was thus the selection of the sentences conveying the illocutionary forces of assertion, question and orders in association with the 9 AUs or combination of AUs under test.
Results (Fig. 3) showed that assertions were primarily selected when the actor had a blank face (AU0). The presence of any AU, with the exception of AU7, significantly lowered the probability of choosing the assertion.
When some AU combinations (AU2, AU1+AU4, AU4 + AU7) were present, the probability of selecting question was higher than when some of the others were (blank face, AU7, AU4+5). AU4 and AU 1+2 did not significantly differ from the highly- and the lowly-associated AUs.
Orders appeared to be more likely to be selected when AU4 + AU5 was present, with a marginal probability over .50. When the individual AUs involved in the combination (AU4 and AU5) were present, the probability of choosing order was higher than some combinations (blank face, AU1+AU2, AU1+AU4), but not significantly higher than the others (AU2, AU7, AU4+AU7).
Assertions (see Fig. 4) appeared not to be associated with any facial expres sion, and with these trials the blank face was considered more compatible than some AU combinations (AU1 + AU4, AU4 + AU5, AU4 + AU7).
For questions, the only significant comparison was the one between AU4 + AU5 and either AU2 or AU1 + AU2. The former appeared to be considered less compatible to questions, while the latter ones more. Although the other comparisons were not significant, they roughly followed the pattern evidenced in Experiment 2.
For orders, two extreme effects emerged: first, AU4 + AU5 was considered highly compatible, and its compatibility judgement was significantly higher than all other action units, except its constituents AU4 and AU5; second, AU1 + AU4 was considered especially incompatible and had lower scores than all other expressions, including the blank face. Other pairwise comparisons that included AU4, AU5 and AU4+7 were significant, evidencing a higher compatibility with orders for these expressions.
Data collected in Experiment 1, 2 and 3 support one main result: in natural language processing there are peculiar AUs and combinations of AUs that are significantly combined with the production and comprehension of speech acts conveying the illocutionary forces of assertion, question and orders.
Data collected in Experiment 1 show that assertions were not significantly associated with any Upper AU. In Experiment 2, not only assertions were primarily associated with AU0 but the presence of any other Upper AU (with the exception of AU7) significantly lowered the probability of classifying a sentence as an assertion. Finally, in Experiment 3, assertions were not associated with any facial expression, and again the blank face was considered significantly more compatible than all other AU.
Taken together, the results of the three experiments seem to support a main conclusion: in language comprehension the illocutionary force of the assertion is not associated with any facial expression except for the default condition of the blank or neutral face: assertions constitute an illocutionary type that is not expected to be marked by non-verbal IFIDs indicating its occurrence.
Results about orders revealed that AU4+5 seems to clearly constitute a nonverbal IFID for such illocutionary force. There are three main results supporting this conclusion: first, in Experiment 2, sentences shown with AU4+5 were primarily interpreted as orders, and in Experiment 3 AU4+5 was rated as highly compatible with this illocutionary type; second, both in Experiment 2 and 3, not only AU4+5 but both the co-occurring action units, i.e., AU4 and AU5, were associated only with orders (although to a lower extent than AU4+5); third, AU4+5 was less associated to questions in both Experiment 2 and 3, which seems to suggest that AU4+5 is an Upper AU that discriminates questions from orders.
Finally, results about questions suggested that: first, AU2 and AU1+4 constitute non-verbal IFIDs for such illocutionary force. AU2 was in fact significantly associated with questions in Experiment 2 and rated as highly compatible with this illocutionary type in Experiment 3. AU1+4 not only increased the probability of interpreting a sentence as a question, as shown in Experiment 2, but it even seemed to constitute an AU that discriminated questions from orders, as in Experiment 3 it was considered as highly incompatible with this latter illocutionary type.
Second, AU4+7 represents a controversial case. AU4+7 was jointly produced with questions in Experiment 1 and it was associated with an increased probability of selecting questions in Experiment 2. This result, however, does not constitute strong enough evidence to identify AU4+7 as a question-specific non-verbal IFID. The reason is that not only in Experiment 2 AU4+7 was associated with orders, too, but in Experiment 3 it was rated as compatible with this other illocutionary type. The compatibility of AU4+7 with both questions and orders, to our view, is open to two possible interpretations.
First interpretation: AU4+7 is a non-verbal IFID compatible with both questions and orders.
Second interpretation: AU4+7 is a non-verbal IFID only for questions. A potential explanation for the co-occurrence of AU4+7 with both questions and orders is offered by the theory of indirect speech acts (Searle, 1975): when AU4+7 has been associated with orders, it has been interpreted as an IFID of a question indirectly conveying the act of an order, since a question constitutes the most appropriate and natural way to indirectly convey an order in the polite context of an interaction among peers.