This talk presents implications from Experimental Semiotics for dialogue systems. Experimental Semiotics investigates human communication by forcing participants to create new communication systems, or adapt existing systems in new ways to achieve joint goals. The key results in this field show the importance of interaction, indirect representation, iconicity, and systematicity in the representations of concepts. These in turn elucidate key expectations humans have for their interlocutors during dialogues.
Experimental Semiotics
A standard experiment in this field selects a particular medium or format and forces participants to use this medium or format to collaborate in solving a task that requires communication. For example, in the Pictionary task [Garrod et al., 2010], participants must communicate by drawing (but without writing) a series of objects in a given order to their partner who has the same list but in a randomised order. The participants succeed each time the correct item on the list is identified.
In another task, participants must convey a route on a map to their partner who has the same map, and knows the start and end points. The closer the path drawn by the matcher to the path given to the director, the better the result. The participants in this task communicate solely by a text-only chat interface.
There are some features of human communication which this paradigm has highlighted which may be instructive for AI dialogue systems which aim to be more human like. A number of experiments have compared what happens in interactive vs non-interactive communications. One key finding has been that in non-interactive circumstances, messages become progressively longer and more elaborate. In contrast, in interaction, messages become shorter and more concise.
The likely reason for this that as messagers routinise their representations, more cognitive resources become available for elaborating the message. Without indication that their partner has received the communication, messagers overcommunicate.
The implication for artificial dialogue systems is that identifying signals that a communication has been understood is crucial to efficient communication. Likewise, the dialogue system should provide such signals when it has been able to interpret its input successfully, and does not need further elaboration. 3
One of the dynamics of communication systems developed in laboratories is refinement. This is where representations become progressively simplified and conventionalised as their user base becomes more familiar with them. As the representations become simpler and more conventional, they usual also become less iconic and more symbolic.
This process is one of progressively indirect representational grounding. The initial iconic representations are grounded naturally because of their iconicity. Subsequent representations of the same concept are representations partly of the object, but partly of the previous representation itself. If previous representations are sufficiently distinctive, and suitably distinctive parts of them are reproduced, we end up with progressively simpler representations, which because of the indirection of representation, can become increasingly opaque for those not party to the evolution of the sign. Later representations, like earlier ones, are grounded iconically, but not in the reference itself, but in earlier representations. 4
Another result [Fay et al., 2008] in Experimental Semiotics is representations developed in communities have a number of properties not shared by those developed by single dyads. These are greater accessibility to participants who have not been party to the shared history. This accessibility can be accounted for [Fay and Ellison, 2013] as retained iconicity – it is iconicity which is retained despite the simplicity of the sign resulting from refinement.
The importance for dialogue systems development is the recognition that not all refinements to a referential representation are of equal value according to objective criteria, and that people recognise and choose between them based on these criteria [Tamariz et al., 2014]. Some options are intrinsically better than others, and selecting the more transparent representations leads over time to more acceptable representations. Failure to adopt a clearly superior representation would lead communicative partners to question the motives of a communicative partner. 5
Finally, yet-unpublished work indicates that communicators reuse representations for similar (but distinct) references. While some theories suggested that representational forms would only be reused in a structured way when the semantic space became crowded, our results show that even a single similar reference is likely to trigger reuse of representational form. In a dialogue, this means that a new salient object that is similar to an existing one is likely to be referenced by a similar representation (more than would be expected by the need for denotational accuracy), but with a clear differentiating additional sign component. In verbal discourse, this type of representation corresponds to using an utterance of the form another X but this time Y where X is a category shared with the previous item, and Y is a distinguishing feature. This construction is called systematicity as the communicator is constructing a system in which shifts form and meaning run parallel.
In conclusion, there are a number of lessons for artificial dialogue systems - whose end goal is communication - in experimental work exploring how humans communicate.