Public and situated displays are commonly available in shopping malls around the world. Typically, they are used to present location information, such as shop locations. In addition, they usually include directories listing shop and service categories. Recently, public displays in shopping malls have become more active. Static displays have been replaced with video displays and interactive touch displays. This allows users to browse and search content on a single display. Interactive displays can offer increasingly rich media content, such as photos, advertisement videos, animations, and even simple games. However, users are still reluctant to use these displays and the interactivity is not obvious. Users cannot easily personalize these displays for their own needs. Therefore, usage periods are usually short and the user experience and engagement are quite poor.
Many customers value the physical dimension of visiting the shopping mall, instead of online shopping. Current technology enables to introduce the same interactive elements that are already available on the web to the physical context. This paper focuses on new ways of providing personally relevant video content for users in public and situated displays in shopping malls. The questions we aim to answer in the future include:
How both customers and retailers goals and needs would be fulfilled with context-aware and social video service in a shopping mall? • How user created video content is perceived and consumed in shopping malls and public places?
What mechanisms should be used to increase contribution, i.e. physical or virtual rewards?
How users perceive tags and annotations as a method of personalization and content discovery in public settings?
To identify the current customer needs three workshops, utilizing participatory design methods, were organized. Based on the ideas and opinions of the participants we identified requirements and scenarios for new services for shopping malls. We present a service concept and preliminary design for social and location based video sharing application Ubeel. With Ubeel people can use their smart phones to record short video clips and share them within the service. Other users can access the videos through mobile devices, computers and interactive public displays. Wide variety of categorized tags in the content allows the creation and personalized presentation of context-aware public narratives, based on presence of people and devices. The information can be composed based on social connections and personal profiles.
This paper has been divided into seven parts. Section two analyzes related research. Section three describes participatory workshops. Section four presents our future scenario. In Section five we introduce our implementation. Finally, in Section six we discuss future directions and conclude in Section seven.
According to Wallace et al. [10] applications developed for shared, public displays, called Single-Display Groupware (SDG) are known to support the teamwork aspects of collaborative work, i.e., the activities required to work as group, such as coordination, communication, and awareness maintenance. However, prolonged interaction with large, touch-based systems can lead to fatigue and physical discomfort. On the other hand, Multi-Display Groupware (MDG) systems, which often consist of multiple connected personal and often public devices allow users to prevent fatigue when interacting with large display by enabling users to utilize more powerful and efficient input techniques on a personal display [2]. The Ubeel system, which we propose in this article, is an MDG system intended for digital story telling between shopping mall users.
Müller et al. [5] have studied why users ignore the content of public displays. Users expectations appear to depend on the perceived context of the display, especially, who they believe is the owner of the display. For certain owners (e.g., universities, public offices) users expect the content to be interesting, while for others (e.g., shops) they expect advertisements. Users would like to see personalized content, such as, local city information, local news, sports, and entertainment. Our motivation is to concentrate primarily to the shopping malls and user expectations in that context.
According to Numa et al. [7], common devices, such as mobile phones, can be turned into expression tools, which allow users to collect and share their stories. In their experiment, the users' stories are connected to each other using questions and answers made by the users themselves. Each story consists of 1) answer to the question from the previous participant, 2) short presentation about a physical item, which user is carrying, 3) connecting phrase to next presentation, and 4) a question to the next participant.
Peltonen et al. [8] have studied sharing of user-created mobile media related to large-scale events on public displays. They focused especially on sharing mobile photos. The public display called CityWall supports multi-touch interaction, and thus enables collaborative use of the display. The combined use of personal mobile devices and a large public display, used collaboratively with other users, creates a unique setting that extends the group's feeling of participation in the events. However, it does not work as a personal communication media. Our system is a bit similar to CityWall, but the focus is set especially on video content and personalization.
We organized a series of participatory design workshops [1], in order to gather ideas and needs for digital service innovations for shopping malls in co-operation with the customers. Customer surveys and focus groups tend to concentrate on the current use and rely on what people say [3,4]. Therefore, they often fail to uncover latent needs and sticky information and might result only minor improvements rather than innovative thinking [9].
Workshop participants were recruited purposively from Social Media Services. 20 participants attended the workshops. The participant group, aged from 25 to 65, included as much women and men. Most mentioned shopping as a hobby and visited shopping malls several times a week. The participants were rewarded with gift cards after each workshop.
In the first workshop, participants were divided into groups that walked within the shopping mall observing it with researchers. Semi-structured interview questions were used to find out participants' experiences and views about the mall as a shopping place as well as a place for spending time. The discussions were audio recorded and transcribed. Discussion themes were identified from the transcripts and used to form ten theme cards for the second workshop to back up discussion. In the third workshop, participants used video cameras as future "shopping tools". They were asked to create stories and visions about the shopping mall in 2014. In addition to workshops, a closed Facebook group was created to continue discussions about ideas and to bring up new topics.
All workshops were audio recorded and transcribed. Service ideas and related comments were written on post-it notes. Altogether approximately 450 ideas were found from the transcripts and Facebook discussions. However, the exact amount of distinct ideas is somewhat smaller. The ideas were organized in thematic groups based on applications with affinity diagram.
Preliminary analysis and implications of ideas and opinions discussed in the workshops is presented in this Section. Full results, validation of ideas and analysis of the method will be reported in following publications.
Participants considered interactive public displays as a commodity in shopping malls. According to them, the most important things include up to date information, quick access to personally relevant content and location information. People are accustomed to search engines, which creates expectations for searchability. They wished that public displays would provide an interface to relevant information as well, such as news and public transport schedules.
They identified some problems with current interactive public displays. Users do not often identify the availability of the touch screen, while advertisements are rolling while the display is idle. Sometimes information is not discovered, because users do not dare to touch the screen. Attention could be attracted with placing displays close to the entrance or in spaces where people hang out,.
Participants wished in-shop screens as well. On the other hand, some screens were considered too public, which results non-use. Empty space around the displays would decrease the barrier to interact.
Participants wished to obtain a bi-directional service community to the shopping mall ecosystem, including participation of all shops and services. The service could be provided via interactive online channel, facilitating direct communications with the customers. They envisioned a virtual feedback box inside the shopping mall premises and more personal experiences. Viral marketing was considered as a viable method for advertising. Customers could be promoters for the shopping mall and products. Rewards were mentioned as a way to encourage people to contribute to the common good and to the community services.
Participants felt that the mall could be the cultural center of the area, with emphasis on locality. They wished possibility for locals to promote their own artistic projects. One central topic was a local media display, to which users could send content they have created. Customers could curate the content collaboratively. The shopping experience could be better if customers would be able to sit down for a moment and enjoy for example video installations.
Interactive applications, such as quizzes, auctions and games could be used with mobile devices. Small prizes, such as coffee or lunch, were mentioned as good motivators for contribution. People expect something surprising from the user experience, even though it cannot be too distracting or obtrusive.
Motivated by some of the ideas presented above, we derived a scenario for social sharing of videos. The proposed service can simultaneously work as a viral marketing tool and as a public voice of the local communities and individuals.
Video sharing tool Ubeel is designed to eventually fulfill the scenario presented above. The core of the service is to embed presence data automatically into the videos and allow tagging and commenting. The server is designed to restore and share both automatic and human created video annotations. The advantage of the annotated and tagged video content is reduced effort in content discovery. Content can be displayed based on location of the user, either currently or when the video was recorded and based on social connections. This approach combines traditional automated recommendation systems and human computing. Privacy issues are not presented in this paper, but are considered as one relevant.
In this section we introduce the current status of the implementation and explore possibilities to enhance Ubeel prototype to be a part of public sharing platform and include it to multi touch displays in smart spaces. Proper balance between more private mobile interface and public use might create more balanced experience.
Mobile application can be set to launch video recording by touching an NFC tag on the location. Video recording sessions are supported by automatic ID collection and visualization, as presented in Figure 1. Mobile User Interface identifies a device. While video is being recorded the application augments available Bluetooth device names on the video display. Ubeel can be set to share tags automatically. Users with Bluetooth enabled devices can be visible on video recordings. The application uses automatic mechanism to inform users if they have been tagged in other videos, which are available for viewing, as presented in Figure 2.
In current implementation users can browse, replay, annotate and share videos on desktop computers and tablets. Desktop enables more functionality for interacting with the content and more efficient input techniques. The annotation function allows user to mark points of interest on the video by drawing a rectangular shape on live video feed. It is possible to add textual comments to these annotations and they can be also shared socially to other users of the system. Desktop tool will be extended to support public multi-touch displays.
The workshops conducted with the shopping mall users confirmed that the content offered by the public displays should be up to date and personally relevant. Bi-directional communication with the services community of the shopping mall was considered as way to increase personal relevance of the content. The service should also offer experiences and surprises, something unexpected, but not too obtrusive.
The experimental Ubeel service allows users to create content with their mobile phones and share it based on physical presence. NFC tags are used to automate video recording and content annotation with Bluetooth device-names. Context information can be used to automatically link to content created by different users. The prototype implemented so far shows that NFC technology, which is currently being introduced in mobile devices, enhances the user experience of the service. The Ubeel service is based on the MDG model. Mobile devices allow users to create the content themselves. The public display can be used to share the content and communicate with other customers and shopping mall services. Multi-touch can be utilized as an interaction method in both mobile devices and public displays. In addition, NFC and Bluetooth can be used to automatically identify the users near public displays.
Wide variety of categorized tags in the content allows the creation of public narratives, based on location of the display and on presence of people and devices. Automatically generated tags enable the linking of places, items, and persons as a part of the service content. The videos created by the individual users can be viewed from public displays. They can be presented and organized based on multiple variables, such as location, time, devices and tagged objects related to a place or a person and annotated comments. In addition, users can add annotations to the timeline of the video, which can be either video or textual input. The information can be presented to users based on social connections and personal profiles, by combining recommendation systems and human computing.
In the workshops, participants presented small prizes as persuasive motivational triggers for contributing to community services. Several successful examples exist, which reward active users for participation. Social games have revealed that people appreciate rewards especially when they come from a person versus a machine and represent approval [6]. Small physical prizes, such as free coffee or lunch, were considered sufficient to motivate users to contribute. The rewards could be granted based on activity, contribution or actions related to certain locations, shops or services.
Next step is to develop more robust and extensive version of the application with the enhancements described above and to test it in a public field trial in a shopping mall. Mobile application will be available for smart phones, which is a small limitation.
In shopping malls, people would like to receive personalized content and feel a sense of togetherness. Workshop participants perceived the shopping mall as a shared living room where they can enjoy themselves and share experiences with friends. We presented preliminary results from three participatory design workshops organized to identify new service ideas and opportunities. We introduced our current implementation, which enables context-aware mobile video creation, automatic tagging, annotation and social sharing. Public displays can be used to present situated interactive narratives, which engage the users and simultaneously offer relevant information related to the context. Our current findings suggest that such systems might benefit from gamification, integration of social sharing and automatic recommendation systems and integrated tags or addresses of devices and things. Future work includes further development of our implementation, especially the functionality for public displays and validating these issues in a field trial. This system will have a major impact on the way people discover, consume and share video content.
This work has been done in Smart 4D Project, funded by Multidiciplinary Institute of Digitalisation and Energy (MIDE). We want to thank Riikka Hänninen and Tomi Hokkanen, who participated in the work.