Digital Proxemics and Interactive Narrative Design in Cultural Heritage Experiences

Abstract

Recent advances in information and communication technologies have enabled the design and implementation of novel applications that highlight the social role of disseminating Cultural Heritage. So far, diferent approaches have created a rich pallet of design options where users experience sociality whether they are spatially close to each other or apart. In this paper, we propose a novel synthesis of proxemic interaction, sensor-based technologies, and narrative design and discuss the case of building an interactive exhibition for traditional professions surrounding olive oil production.

1. Introduction

The widespread use of mobile applications, sensors and extended reality technologies has given rise to new cultural practices which can overturn, or challenge creatively the way visitors experience culture while communicating to each other. But do such new technologies bring people closer, or do they push them apart?1 Digital storytelling in Cultural Heritage is commonly designed as an individual experience rather than social. Usually, interpretation centers and institutions, such as museums, ofer visitors a custom-built device or the option to download an application on their own device which will guide them individually around a dedicated space. Nevertheless, the intrinsic social character of cultural visits was acknowledged early on in relevant studies [ 2, 3, 4 ] as people rarely decide to experience culture on their own [ 5 ]. Consequently, spaces of cultural heritage dissemination have been characterized as “social spaces” where visitors construct knowledge by sharing and exchanging views on aspects of the past [ 6 ].

More recently, scholars and field practitioners have attempted to introduce sociality in designing guided experiences through mobile digital storytelling as a way of enhancing communication, CEUR Workshop Proceedings htp:/ceur-ws.org ISN1613-073

CEUR Workshop Proceedings (CEUR-WS.org) 1Paraphrasing the title of the seminal paper by Rosenblat and Mobius [ 1 ] on communication technologies addressing the question of whether such technologies get people closer or make them drift apart. interaction and meaning making between visitors [ 7, 8, 9, 10 ]. Besides in mobile guide studies, social interaction has been highlighted as an important aspect of cultural visits [ 11, 12 ]. So far, diferent approaches have created a rich pallet of design options where users can enjoy social experiences in cultural heritage dissemination settings whether they are spatially close to each other [ 7, 10, 13, 14 ] or apart [ 15, 16 ].

In previous work, we explored blending individual digital storytelling with social interaction between visitors that remain close to each other during the experience [ 10 ]. In the context of that experimental design, we wished to facilitate common activities between individual storytelling sessions which promoted communication and exchange of views between participants. The results of the design and detailed evaluation verify the eficiency of such approaches in achieving a deeper level of understanding and engagement with the presented cultural content through social interaction [ 10 ]. In this paper, we present the conceptualization of a novel design approach for enabling social interaction in cultural heritage settings that utilizes the theoretical framework of proxemics combined with sensor technology and narrative design. While the majority of this in-progress work is still at a conceptual level, certain parts have been completed, prototyped and/or tested as indicated in the rest of the paper. The latter provides a brief introduction to proxemics in cultural heritage, presents and discusses the technological and narrative scenario of the application and finally, concludes with future work.

2. Digital proxemics in Cultural Heritage

The term “proxemics” was coined by cultural anthropologist Edward Hall in mid 1960s and refers to the study of the human perception of space and the culturally dependent ways in which individuals use distance to achieve certain communication goals [ 17 ]. It is also one of the non-verbal forms of communication alongside kinetics and haptics, while it has received a plethora of uses and meanings due to the important role it holds in human communication [ 18 ]. As physical and digital spaces blend in contemporary life, digital proxemics becomes a fertile ground for research and practice in digital cultural studies.

2.1. A brief overview of applied proxemics

So far, proxemics has been used in various research directions such as interaction design [ 19, 20 ], collocated interaction [ 21 ], games [ 22, 23 ], and location-based practices [ 24 ]. The most applied framework of proxemics theory is the concept of four distancing zones which describes the proximity between individuals depending on the type of their social interaction. Arguably, those zones are open to interpretation by individuals depending on their personal, cultural and cognitive baggage. Nevertheless, for practical reasons the four zones are categorized as below: intimate zone (<0.5m), personal (0.5m-1.2m), social (1.2-3.6m), and public (3.6-7.6m).

Another interesting aspect of proxemics theory describes configuration of spaces that are either inviting or discouraging individuals from social interaction and communication. More specifically, the configuration of space can be characterised as “sociopetal”, when it encourages social interaction, or in the opposite case as “sociofugal” [ 25, 26 ]. To the best of our knowledge this framework has never been utilized in cultural heritage dissemination settings, particularly alongside narrative design. However, the concept of proxemic distances has been previously used in designing interactions in museum scenarios [ 21 ] and as a spatial configuration approach for exhibits [27].

2.2. Digital proxemics interaction and system architecture

Despite the breadth of place-based and locative digital storytelling applications in the field of Cultural Heritage, there is not much in-depth work that examines the narrative design [28]. Do we construct narratives that fit specific locations, exhibits, content, communication, and socialization purposes; all too critical aspects of providing attractive experiences in the domain? If so, how do we go about it? How do we ensure that we create meaningful interactive stories throughout a designated space?

From the outset of this research, the main aim was to create a memorable storytelling experience around a collection of historic photographs regarding traditional professions around olive oil production in the N. Aegean, Greece. Our inquiry focused on proxemics to design visitor movement and interactions (between pairs of visitors and, between visitors and exhibits) and the type of narratives that would best serve the purposes of the experience. The objectives of this research project are as follows: a) to design the technical infrastructure to support the proxemic experience, and b) to design a prototype exhibition space and produce the storytelling that would fit with the envisaged social and sensory interactions. In the following paragraphs we describe the basic scenario of our application along with the respective technological and design decisions.

2.2.1. Basic application scenario

A collection of old photographs depicting traditional professions is placed in the exhibition space. The photographs are spatially arranged in a meaningful way, according to the main narrative scenario about the workflow of oil production and its by-products. Each photograph is placed in a frame on the wall. Each frame has LED strips which can light up diferent colors and provide non-verbal guidance throughout the experience.

Drawing from proxemics theory, both the design of the physical and the digital component of the exhibition followed the sociopetal and sociofugal scheme of interaction. Based on the scenario, visitors download the application which is designed to accommodate the social and narrative requirements of visitor pairs. According to the proxemics scheme, visitors are guided through the storytelling application and the sensory stimuli around the exhibition space.

2.2.2. System architecture and infrastructure

The technical implementation of this proxemic experience is based on technologies locating the position of visitors and exhibits. The user emits her position through the mobile device, whether in absolute value or relatively to the exhibits. This way, the system can locate the user in three instances: a) near exhibits, b) in front of the exhibits, 3) close to other users. The accuracy of tracking visitors’ position in scenarios where points of interest might be too close to each other is mostly dependent on the efectiveness of the respective algorithm. Bearing this in mind, we opted for a suitable approach that enables the envisaged social experience without stumbling upon technical dificulties.

The architectural schema of the system is presented in Figure 1. In this schema, we represent two users who participate in the experience, and each carries the following: 1. A mobile device which runs the application 2. An NFC reader integrated in the mobile phone 3. Beacon tags (preferably wearable) The exhibition infrastructure, apart from the actual curated collection included the following: 4. NFC tags, fixed on each photograph frame 5. Monitor display and sound speaker which are programmed by an Arduino microcontroller 6. Beacons, which are placed in certain locations of interest (i.e. between or next to photographs) 7. Smart Led strips placed as above

Finally, the infrastructure includes a server and a database, which support centrally the whole experience. So far, we have only prototyped items 4, 5, 6 and 7 from the above list.

3. Blending in narrative design and spatial configuration

For the second objective, the spatial configuration of the exhibition and the narrative design were led by the requirements of the proxemic experience (see schematic layout, Figure 2). More specifically, the experience starts with an introduction to the theme of olive oil production in the Aegean Island of Lesvos in area A. The audio narrative is common to both users who listen to it individually through their mobile apps. At the same time, visitors can walk around a cylindrical stand to observe a collection of old reprinted photographs showing instances of people working in olive groves. Although in this part of the experience, visitors listen to the narrative individually, both the configuration of space and the audio storytelling facilitate a common experience that leaves room for visitors to socialize freely, if they wish to do so. From a technical point of view, the mobile application is registered at the initiation of the experience to the control software, which ensures that both users start the experience simultaneously. Next, it notifies the mobile application to search for the localization markers. From here onwards, the distance of the mobile device from all the exhibits is calculated throughout the experience. Each time the device approaches a marker, the mobile application is informed of the proximity and sends updates to the system.

After area A, visitors are prompted to choose how they wish to proceed with the experience. This part of the narrative is also experienced individually and builds on perspective-taking in the story, so visitors can choose between four designated roles by choosing gender (i.e. man-woman) and type of profession (i.e. landowner-worker). Once the system registers their preferences, visitors are guided through respective visual cues (i.e. diferent coloured light coming from the photograph frames) towards separate exhibits in area B, where they listen to a first person story about the respective profession by a historic person depicted in the photograph in front of them. The space itself is purpose-designed with partitions to avoid any social interaction while each exhibit is placed at suficient distance to facilitate efective tracking.

In the next stage of the experience, visitors are once more visually guided in front of an exhibit in area C, where they share a common storytelling experience in the form of a dialogue taking place between characters that they’ve been introduced to in exhibit area B and are prompted to interact with one another. The content is activated only when both users approach at the suggested exhibit which is illuminated with the same colors, as the colors of their respective role in exhibit area B. Each combination of roles leads to a respective shared narrative which includes dialogue and reflective discussion sessions based on the perspective-taking approach ??. In this case, each exhibit stands in an open space welcoming social interaction. Finally, visitors are once again prompted by the application to freely walk and choose to pause and listen to separate stories about other professions linked to the olive oil production and its by-products in secluded spaces, in area D.

The design of the exhibition space is still at a conceptual level while the narrative design, the final scripts and the production of audio snippets have been completed, integrated in the Narralive authoring and storytelling platform [29], and Wizard of Oz tested in a temporary space with the respective curation of reprinted photographs.

4. Discussion and Future Work

In this paper, we describe a novel storytelling approach based on proxemic theory and the conceptual design of the system architecture supporting it. In this approach, both the placement of the exhibits and storytelling meet two basic conditions: a) alternating between individualsocial interaction, and b) bringing together and pushing apart visitors according to the proxemics “sociopetal and sociofugal” scheme. In the first case, the configuration of the space, the exhibits, the digital media, and storytelling create the right conditions for interpersonal and social interaction of visitors, while in the second case there is an intentional use of all the above to avoid communication and interaction between visitors. Also, we opted for designing a variation between communication and interaction scenarios spanning from intuitive exchange of views based on common stories, to intentional and structured social interactions.

As this is an on-going work, we cannot yet claim to have reached the set goals of this research project, but we firmly believe that proxemics provides an interesting framework to inform the design of interactions between people who participate in shared cultural experiences, and between individuals and exhibits. Therefore, we plan to fully implement the application scenario, to experiment and expand on the types of storytelling for social purposes, and finally to conduct a thorough evaluation of the design approach to validate our approach.

Acknowledgments

This research has been conducted in the frame of the project “Aria - Augmenting the Reception of music through Innovative solutions and Archives” with the code T2EDK-02084 supported by EU and National Greek funds under the General Secretariat for Research and Innovation funding programs. We are also indebted to the CuRep Lab at the University of the Aegean for providing us with the cultural content.

Edmonson, J. Fischer, D. Hymes, S. T. Kimball, et al., Proxemics [and comments and replies], Current anthropology 9 (1968) 83–108. [26] J. A. McArthur, Digital proxemics: How technology shapes the ways we move, Peter Lang

International Academic Publishers, 2016. [27] K. Wolf, Y. Abdelrahman, T. Kubitza, A. Schmidt, Intimate proxemic zones of exhibits and their manipulation using floor projection., 2016. [28] A. Chrysanthi, A. Katifori, M. Vayanou, A. Antoniou, Place-based digital storytelling. the interplay between narrative forms and the cultural heritage space, in: Emerging Technologies and the Digital Transformation of Museums and Heritage Sites, Springer International Publishing, 2021, pp. 127–138. doi:10.1007/978-3-030-83647-4\_9. [29] E. Vrettakis, V. Kourtis, A. Katifori, M. Karvounis, C. Lougiakis, Y. Ioannidis, Narralive – creating and experiencing mobile digital storytelling in cultural heritage, Digital Applications in Archaeology and Cultural Heritage 15 (2019) e00114. doi:10.1016/j.daach. 2019.e00114.

References

[1] T. S. Rosenblat , M. M. Mobius , Getting closer or drifting apart? , The Quarterly Journal of Economics 119 ( 2004 ) 971 - 1009 .

[2] P. McManus , Good companions: More on the social determination of learning-related behaviour in a science museum , International Journal of Museum Management and Curatorship 7 ( 1988 ) 37 - 44 . doi: 10 .1080/09647778809515102.

[3] G. Leinhardt , K. Knutson , Listening in on museum conversations, Rowman Altamira , 2004 .

[4] L. D. Dierking , J. H. Falk , Family behavior and learning in informal science settings: A review of the research , Science Education 78 ( 1994 ) 57 - 72 .

[5] J. H. Falk , Identity and the Museum Visitor Experience , Routledge, 2016 . doi: 10 .4324/ 9781315427058.

[6] J. H. Falk , L. D. Dierking , Learning from museums, Rowman & Littlefield , 2018 .

[7] C. Callaway , O. Stock , E. Dekoven, Experiments with mobile drama in an instrumented museum for inducing conversation in small groups , ACM Trans. Interact. Intell. Syst . 4 ( 2014 ). URL: https://doi.org/10.1145/2584250. doi: 10 .1145/2584250.

[8] S. Perry , M. Roussou , S. S. Mirashrafi , A. Katifori , S. McKinney , Shared digital experiences supporting collaborative meaning-making at heritage sites , in: The Routledge International Handbook of New Digital Practices in Galleries, Libraries, Archives, Museums and Heritage Sites , Routledge, 2019 , pp. 143 - 156 .

[9] A. Katifori , S. Perry , M. Vayanou , L. Pujol , A. Chrysan , V. Kourtis , Y. Ioannidis , Cultivating mobile-mediated social interaction in the museum: Towards group-based digital storytelling experiences ( 2016 ).

[10] A. Katifori , S. Perry , M. Vayanou , A. Antoniou , I.-P. Ioannidis , S. McKinney , A. Chrysanthi , Y. Ioannidis , “ let them talk!”: Exploring guided group interaction in digital storytelling experiences , J. Comput. Cult. Herit . 13 ( 2020 ). URL: https://doi.org/10.1145/3382773. doi: 10 . 1145/3382773.

[11] C. Emmanouilidis , R.-A. Koutsiamanis , A. Tasidou , Mobile guides: Taxonomy of architectures, context awareness, technologies and applications , Journal of Network and Computer Applications 36 ( 2013 ) 103 - 125 . doi: 10 .1016/j.jnca. 2012 . 04 .007.

[12] D. Raptis , N. Tselios , N. Avouris , Context-based design of mobile applications for museums: A survey of existing practices , in: Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services , MobileHCI '05, Association for Computing Machinery, New York, NY, USA, 2005 , p. 153 - 160 . URL: https://doi.org/10. 1145/1085777.1085803. doi: 10 .1145/1085777.1085803.

[13] L. Fosh , S. Benford , S. Reeves , B. Koleva , P. Brundell , See me, feel me, touch me, hear me: Trajectories and interpretation in a sculpture garden , in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13 , Association for Computing Machinery, New York, NY, USA, 2013 , p. 149 - 158 . URL: https://doi.org/10.1145/2470654. 2470675. doi: 10 .1145/2470654.2470675.

[14] S. Huws , A . John, J. Kidd, Evaluating the afective dimensions of traces-olion: a subtle mob at st fagans national museum of history, wales , in: 2018 3rd Digital Heritage International Congress (DigitalHERITAGE) held jointly with 2018 24th International Conference on Virtual Systems & Multimedia (VSMM 2018 ), IEEE, 2018 , pp. 1 - 8 . doi: 10 .1109/DigitalHeritage. 2018 . 8810117 .

[15] T. Kuflik , J. Sheidin , S. Jbara , D. Goren-Bar , P. Sofer , O. Stock , M. Zancanaro , Supporting small groups in the museum by context-aware communication services , in: Proceedings of the 12th International Conference on Intelligent User Interfaces , IUI '07 , Association for Computing Machinery, New York, NY, USA, 2007 , p. 305 - 308 . URL: https://doi.org/10. 1145/1216295.1216353. doi: 10 .1145/1216295.1216353.

[16] T. Kuflik , O. Stock , M. Zancanaro , A. Gorfinkel , S. Jbara , S. Kats , J. Sheidin , N. Kashtan , A visitor's guide in an active museum: Presentations, communications, and reflection , J. Comput. Cult. Herit . 3 ( 2011 ). URL: https://doi.org/10.1145/1921614.1921618. doi: 10 .1145/ 1921614.1921618.

[17] E. T. Hall , E. T. Hall, The hidden dimension , volume 609 , Anchor , 1966 .

[18] A. Mehrabian , Nonverbal communication, Routledge , 2017 .

[19] W. IJsselsteijn , W. Van Den Hoogen, C. Klimmt, Y. De Kort , C. Lindley , K. Mathiak , K. Poels , N. Ravaja , M. Turpeinen , P. Vorderer , Measuring the experience of digital game enjoyment , in: Proceedings of measuring behavior , volume 2008 , Noldus Maastricht , the Netherlands, 2008 , pp. 88 - 89 .

[20] N. Marquardt , S. Greenberg , Informing the design of proxemic interactions , IEEE Pervasive Computing 11 ( 2012 ) 14 - 23 . doi: 10 .1109/MPRV. 2012 . 15 .

[21] J. Lanir , T. Kuflik, Proxemic-based collocated interactions at the museum environment , in: Proxemic mobile collocated interaction workshop at CHI 2016 , 2016 .

[22] F. F. Mueller , D. Edge , F. Vetere , M. R. Gibbs , S. Agamanolis , B. Bongers , J. G. Sheridan , Designing sports: A framework for exertion games , in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '11 , Association for Computing Machinery, New York, NY, USA, 2011 , p. 2651 - 2660 . URL: https://doi.org/10.1145/1978942. 1979330. doi: 10 .1145/1978942.1979330.

[23] F. Mueller , S. Stellmach , S. Greenberg , A. Dippon , S. Boll , J. Garner , R. Khot , A. Naseem , D. Altimira , Proxemics play: Understanding proxemics for designing digital play experiences , in: Proceedings of the 2014 Conference on Designing Interactive Systems, DIS '14 , Association for Computing Machinery, New York, NY, USA, 2014 , p. 533 - 542 . URL: https://doi.org/10.1145/2598510.2598532. doi: 10 .1145/2598510.2598532.

[24] H. Berghel , Wireless infidelity i: War driving , Commun. ACM 47 ( 2004 ) 21 - 26 . URL: https://doi.org/10.1145/1015864.1015879. doi: 10 .1145/1015864.1015879.

[25] E. T. Hall , R. L. Birdwhistell , B. Bock , P. Bohannan , A. R. Diebold Jr, M. Durbin , M. S.