(Invited Talk) Experimental Semiotics and Representation by Dialogue Systems T. Mark Ellison College of Asia and the Pacific, Australian National University, Australia m.ellison@anu.edu.au Abstract The likely reason for this that as messagers routinise their representations, more cognitive resources become available This talk presents implications from Experimen- for elaborating the message. Without indication that their tal Semiotics for dialogue systems. Experimental partner has received the communication, messagers over- Semiotics investigates human communication by communicate. forcing participants to create new communication systems, or adapt existing systems in new ways to The implication for artificial dialogue systems is that iden- achieve joint goals. The key results in this field tifying signals that a communication has been understood is show the importance of interaction, indirect rep- crucial to efficient communication. Likewise, the dialogue resentation, iconicity, and systematicity in the rep- system should provide such signals when it has been able to resentations of concepts. These in turn elucidate interpret its input successfully, and does not need further elab- key expectations humans have for their interlocu- oration. tors during dialogues. 3 Indirect Representation and Symbol 1 Introduction Grounding Experimental Semiotics (see [Galantucci and Garrod, 2011]) One of the dynamics of communication systems developed in is a line of psychological research focussing on the experi- laboratories is refinement. This is where representations be- mental investigation of novel forms of human communica- come progressively simplified and conventionalised as their tion. user base becomes more familiar with them. As the repre- A standard experiment in this field selects a particular sentations become simpler and more conventional, they usual medium or format and forces participants to use this medium also become less iconic and more symbolic. or format to collaborate in solving a task that requires com- This process is one of progressively indirect represen- munication. For example, in the Pictionary task [Garrod et tational grounding. The initial iconic representations are al., 2010], participants must communicate by drawing (but grounded naturally because of their iconicity. Subsequent without writing) a series of objects in a given order to their representations of the same concept are representations partly partner who has the same list but in a randomised order. The of the object, but partly of the previous representation it- participants succeed each time the correct item on the list is self. If previous representations are sufficiently distinctive, identified. and suitably distinctive parts of them are reproduced, we end In another task, participants must convey a route on a map up with progressively simpler representations, which because to their partner who has the same map, and knows the start of the indirection of representation, can become increasingly and end points. The closer the path drawn by the matcher opaque for those not party to the evolution of the sign. Later to the path given to the director, the better the result. The representations, like earlier ones, are grounded iconically, but participants in this task communicate solely by a text-only not in the reference itself, but in earlier representations. chat interface. There are some features of human communication which this paradigm has highlighted which may be instructive for 4 Community AI dialogue systems which aim to be more human like. Another result [Fay et al., 2008] in Experimental Semiotics is representations developed in communities have a number 2 Interaction of properties not shared by those developed by single dyads. A number of experiments have compared what happens in in- These are greater accessibility to participants who have not teractive vs non-interactive communications. One key finding been party to the shared history. This accessibility can be has been that in non-interactive circumstances, messages be- accounted for [Fay and Ellison, 2013] as retained iconicity – come progressively longer and more elaborate. In contrast, in it is iconicity which is retained despite the simplicity of the interaction, messages become shorter and more concise. sign resulting from refinement. 1 The importance for dialogue systems development is the recognition that not all refinements to a referential representa- tion are of equal value according to objective criteria, and that people recognise and choose between them based on these criteria [Tamariz et al., 2014]. Some options are intrinsically better than others, and selecting the more transparent repre- sentations leads over time to more acceptable representations. Failure to adopt a clearly superior representation would lead communicative partners to question the motives of a commu- nicative partner. 5 Systematicity Finally, yet-unpublished work indicates that communicators reuse representations for similar (but distinct) references. While some theories suggested that representational forms would only be reused in a structured way when the seman- tic space became crowded, our results show that even a single similar reference is likely to trigger reuse of representational form. In a dialogue, this means that a new salient object that is similar to an existing one is likely to be referenced by a similar representation (more than would be expected by the need for denotational accuracy), but with a clear differenti- ating additional sign component. In verbal discourse, this type of representation corresponds to using an utterance of the form another X but this time Y where X is a category shared with the previous item, and Y is a distinguishing fea- ture. This construction is called systematicity as the com- municator is constructing a system in which shifts form and meaning run parallel. In conclusion, there are a number of lessons for artificial dialogue systems - whose end goal is communication - in ex- perimental work exploring how humans communicate. References [Fay and Ellison, 2013] Nicolas Fay and T Mark Ellison. The cultural evolution of human communication systems in different sized populations: usability trumps learnabil- ity. PloS one, 8(8):e71781, 2013. [Fay et al., 2008] Nicolas Fay, Simon Garrod, and Leo Roberts. The fitness and functionality of culturally evolved communication systems. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 363(1509):3553–3561, 2008. [Galantucci and Garrod, 2011] Bruno Galantucci and Simon Garrod. Experimental semiotics: a review. Frontiers in human neuroscience, 5, 2011. [Garrod et al., 2010] Simon Garrod, Nicolas Fay, Shane Rogers, Bradley Walker, and Nik Swoboda. Can iterated learning explain the emergence of graphical symbols? In- teraction Studies, 11(1):33–50, 2010. [Tamariz et al., 2014] Monica Tamariz, T Mark Ellison, Dale J Barr, and Nicolas Fay. Cultural selection drives the evolution of human communication systems. Proceedings of the Royal Society of London B: Biological Sciences, 281(1788):20140488, 2014. 2