=Paper=
{{Paper
|id=None
|storemode=property
|title=Ubeel: Generating Local Narratives for Public Displays from Tagged and Annotated Video Content
|pdfUrl=https://ceur-ws.org/Vol-720/Ojanen.pdf
|volume=Vol-720
}}
==Ubeel: Generating Local Narratives for Public Displays from Tagged and Annotated Video Content==
https://ceur-ws.org/Vol-720/Ojanen.pdf
Ubeel: Generating Local Narratives for Public Displays
from Tagged and Annotated Video Content
Pia Ojanen, Petri Vuorimaa, Petri Saarikko and Sanna Uotinen
Aalto University School of Science
B.O. Box 15400, FI00076 Aalto
Finland
+358 50 307 7716
pia.ojanen@aalto.fi
ABSTRACT shopping mall, instead of online shopping. Current technology
In this paper, we present our vision and research directions for enables to introduce the same interactive elements that are already
personalized and context-aware sharing of videos. Our aim is to available on the web to the physical context. This paper focuses
explore new methods for providing situated video content. on new ways of providing personally relevant video content for
Utilizing multi-touch displays and encouraging users to users in public and situated displays in shopping malls. The
collaboratively create content in shopping malls and public questions we aim to answer in the future include:
spaces. We seek to understand what motivates people to
contribute to the local narratives or in viral marketing purposes. • How both customers and retailers goals and needs
One objective is to explore content personalization utilizing tags would be fulfilled with context-aware and social video
and context-aware information. We present relevant preliminary service in a shopping mall?
results from participatory design workshops, organized to involve • How user created video content is perceived and
the customers to elaborate ideas for new digital services in consumed in shopping malls and public places?
shopping malls. Preliminary implementation for a mobile location • What mechanisms should be used to increase
aware social video sharing service Ubeel is presented. We will contribution, i.e. physical or virtual rewards?
discuss our current vision of elements which are important and • How users perceive tags and annotations as a method of
should be considered in such systems in the future. personalization and content discovery in public settings?
To identify the current customer needs three workshops, utilizing
Categories and Subject Descriptors participatory design methods, were organized. Based on the ideas
H.5.3 [Information interfaces and presentation]: Group and
and opinions of the participants we identified requirements and
Organization Interfaces
scenarios for new services for shopping malls. We present a
service concept and preliminary design for social and location
General Terms based video sharing application Ubeel. With Ubeel people can use
Design, Human Factors their smart phones to record short video clips and share them
within the service. Other users can access the videos through
mobile devices, computers and interactive public displays. Wide
Keywords variety of categorized tags in the content allows the creation and
Public displays, mobile, video, location aware, collaborative
personalized presentation of context-aware public narratives,
based on presence of people and devices. The information can be
1. INTRODUCTION composed based on social connections and personal profiles.
Public and situated displays are commonly available in shopping
malls around the world. Typically, they are used to present This paper has been divided into seven parts. Section two
location information, such as shop locations. In addition, they analyzes related research. Section three describes participatory
usually include directories listing shop and service categories. workshops. Section four presents our future scenario. In Section
Recently, public displays in shopping malls have become more five we introduce our implementation. Finally, in Section six we
active. Static displays have been replaced with video displays and discuss future directions and conclude in Section seven.
interactive touch displays. This allows users to browse and search
content on a single display. Interactive displays can offer 2. RELATED WORK
increasingly rich media content, such as photos, advertisement
According to Wallace et al. [10] applications developed for
videos, animations, and even simple games. However, users are
shared, public displays, called Single-Display Groupware (SDG)
still reluctant to use these displays and the interactivity is not
are known to support the teamwork aspects of collaborative work,
obvious. Users cannot easily personalize these displays for their
i.e., the activities required to work as group, such as coordination,
own needs. Therefore, usage periods are usually short and the user
communication, and awareness maintenance. However, prolonged
experience and engagement are quite poor.
interaction with large, touch-based systems can lead to fatigue and
physical discomfort. On the other hand, Multi-Display Groupware
Many customers value the physical dimension of visiting the
(MDG) systems, which often consist of multiple connected
personal and often public devices allow users to prevent fatigue from the transcripts and used to form ten theme cards for the
when interacting with large display by enabling users to utilize second workshop to back up discussion. In the third workshop,
more powerful and efficient input techniques on a personal participants used video cameras as future “shopping tools”. They
display [2]. The Ubeel system, which we propose in this article, is were asked to create stories and visions about the shopping mall
an MDG system intended for digital story telling between in 2014. In addition to workshops, a closed Facebook group was
shopping mall users. created to continue discussions about ideas and to bring up new
topics.
Müller et al. [5] have studied why users ignore the content of
public displays. Users expectations appear to depend on the All workshops were audio recorded and transcribed. Service ideas
perceived context of the display, especially, who they believe is and related comments were written on post-it notes. Altogether
the owner of the display. For certain owners (e.g., universities, approximately 450 ideas were found from the transcripts and
public offices) users expect the content to be interesting, while for Facebook discussions. However, the exact amount of distinct
others (e.g., shops) they expect advertisements. Users would like ideas is somewhat smaller. The ideas were organized in thematic
groups based on applications with affinity diagram.
to see personalized content, such as, local city information, local
news, sports, and entertainment. Our motivation is to concentrate 3.2 Ideas and Opinions
primarily to the shopping malls and user expectations in that
Preliminary analysis and implications of ideas and opinions
context.
discussed in the workshops is presented in this Section. Full
results, validation of ideas and analysis of the method will be
According to Numa et al. [7], common devices, such as mobile
reported in following publications.
phones, can be turned into expression tools, which allow users to
collect and share their stories. In their experiment, the users' 3.2.1 Public Information Displays in Shopping malls
stories are connected to each other using questions and answers Participants considered interactive public displays as a commodity
made by the users themselves. Each story consists of 1) answer to in shopping malls. According to them, the most important things
the question from the previous participant, 2) short presentation include up to date information, quick access to personally relevant
about a physical item, which user is carrying, 3) connecting content and location information. People are accustomed to search
phrase to next presentation, and 4) a question to the next engines, which creates expectations for searchability. They
participant. wished that public displays would provide an interface to relevant
information as well, such as news and public transport schedules.
Peltonen et al. [8] have studied sharing of user-created mobile They identified some problems with current interactive public
media related to large-scale events on public displays. They displays. Users do not often identify the availability of the touch
focused especially on sharing mobile photos. The public display screen, while advertisements are rolling while the display is idle.
called CityWall supports multi-touch interaction, and thus enables Sometimes information is not discovered, because users do not
collaborative use of the display. The combined use of personal dare to touch the screen. Attention could be attracted with placing
mobile devices and a large public display, used collaboratively displays close to the entrance or in spaces where people hang out,.
with other users, creates a unique setting that extends the group's Participants wished in-shop screens as well. On the other hand,
feeling of participation in the events. However, it does not work some screens were considered too public, which results non-use.
as a personal communication media. Our system is a bit similar to Empty space around the displays would decrease the barrier to
CityWall, but the focus is set especially on video content and interact.
personalization.
3. PARTICIPATORY WORKSHOPS 3.2.2 Service community
We organized a series of participatory design workshops [1], in Participants wished to obtain a bi-directional service community
order to gather ideas and needs for digital service innovations for to the shopping mall ecosystem, including participation of all
shopping malls in co-operation with the customers. Customer shops and services. The service could be provided via interactive
surveys and focus groups tend to concentrate on the current use online channel, facilitating direct communications with the
and rely on what people say [3, 4]. Therefore, they often fail to customers. They envisioned a virtual feedback box inside the
uncover latent needs and sticky information and might result only shopping mall premises and more personal experiences. Viral
minor improvements rather than innovative thinking [9]. marketing was considered as a viable method for advertising.
Customers could be promoters for the shopping mall and
3.1 Method products. Rewards were mentioned as a way to encourage people
Workshop participants were recruited purposively from Social to contribute to the common good and to the community services.
Media Services. 20 participants attended the workshops. The
participant group, aged from 25 to 65, included as much women 3.2.3 Video Content and Ambience
and men. Most mentioned shopping as a hobby and visited Participants felt that the mall could be the cultural center of the
shopping malls several times a week. The participants were area, with emphasis on locality. They wished possibility for locals
rewarded with gift cards after each workshop. to promote their own artistic projects. One central topic was a
In the first workshop, participants were divided into groups that local media display, to which users could send content they have
walked within the shopping mall observing it with researchers. created. Customers could curate the content collaboratively. The
Semi-structured interview questions were used to find out shopping experience could be better if customers would be able to
participants’ experiences and views about the mall as a shopping sit down for a moment and enjoy for example video installations.
place as well as a place for spending time. The discussions were Interactive applications, such as quizzes, auctions and games
audio recorded and transcribed. Discussion themes were identified could be used with mobile devices. Small prizes, such as coffee or
lunch, were mentioned as good motivators for contribution.
People expect something surprising from the user experience,
even though it cannot be too distracting or obtrusive.
4. SCENARIO
Motivated by some of the ideas presented above, we derived a
scenario for social sharing of videos. The proposed service can
simultaneously work as a viral marketing tool and as a public
voice of the local communities and individuals.
Tina and her friends love shopping. Tina is spending time at the
mall and meets her friend Carol. They go to a clothing store to try
on new skirts. Carol checks herself in to the store with her phone
and starts shooting a video of Tina, wearing a blue skirt. The
application identifies active devices in the area, and tags Tina and Figure 1. Mobile User Interface identifies a device.
a shopping assistant to the video. Carol reads the QR code printed Ubeel can be set to share tags automatically. Users with Bluetooth
in the label of the skirt, which is also tagged to the video. Carol enabled devices can be visible on video recordings. The
uploads the video to the server. Tina tries on another skirt, and application uses automatic mechanism to inform users if they
Carol records again. Meanwhile their friend Marion checks the have been tagged in other videos, which are available for viewing,
display on the hallway. Marion is automatically recognized as a as presented in Figure 2.
friend of Tina, based on her social graph. Her profile indicates
interest in clothing. Video about Tina wearing the blue skirt
appears on the screen and after the video has ended Marion inserts
a comment inside the video: “Wow, looks great!”. Back in the
store Carol has uploaded the second video and sees the comment.
Tina is thrilled of the attention and decides to buy that skirt. Girls
leave the shop. A group of teenage girls stop by the display. They
have youth clothing listed in their interest profile. Therefore
video’s related to the new collection appear. They immediately
like what they see and decide to head towards the clothing store.
When Tina’s friend Michelle arrives to the store later, the in-store
display shows videos of Tina on the top of the list. She checks the
videos and decides to try on the red skirt Tina did not buy.
5. IMPLEMENTATION Figure 2. User Interface provides content based on tags
Video sharing tool Ubeel is designed to eventually fulfill the
scenario presented above. The core of the service is to embed 5.2 Desktop Application
presence data automatically into the videos and allow tagging and In current implementation users can browse, replay, annotate and
commenting. The server is designed to restore and share both share videos on desktop computers and tablets. Desktop enables
automatic and human created video annotations. The advantage of
more functionality for interacting with the content and more
the annotated and tagged video content is reduced effort in content efficient input techniques. The annotation function allows user to
discovery. Content can be displayed based on location of the user, mark points of interest on the video by drawing a rectangular
either currently or when the video was recorded and based on shape on live video feed. It is possible to add textual comments to
social connections. This approach combines traditional automated these annotations and they can be also shared socially to other
recommendation systems and human computing. Privacy issues users of the system. Desktop tool will be extended to support
are not presented in this paper, but are considered as one relevant. public multi-touch displays.
In this section we introduce the current status of the
implementation and explore possibilities to enhance Ubeel
6. DISCUSSION AND FUTURE WORK
The workshops conducted with the shopping mall users confirmed
prototype to be a part of public sharing platform and include it to
that the content offered by the public displays should be up to date
multi touch displays in smart spaces. Proper balance between
and personally relevant. Bi-directional communication with the
more private mobile interface and public use might create more
services community of the shopping mall was considered as way
balanced experience.
to increase personal relevance of the content. The service should
5.1 Mobile User Interface also offer experiences and surprises, something unexpected, but
Mobile application can be set to launch video recording by not too obtrusive.
touching an NFC tag on the location. Video recording sessions are The experimental Ubeel service allows users to create content
supported by automatic ID collection and visualization, as with their mobile phones and share it based on physical
presented in Figure 1. Mobile User Interface identifies a device. presence. NFC tags are used to automate video recording and
While video is being recorded the application augments available content annotation with Bluetooth device-names.
Bluetooth device names on the video display. Context information can be used to automatically link to content
created by different users. The prototype implemented so far recommendation systems and integrated tags or addresses of
shows that NFC technology, which is currently being introduced devices and things. Future work includes further development of
in mobile devices, enhances the user experience of the service. our implementation, especially the functionality for public
The Ubeel service is based on the MDG model. Mobile devices displays and validating these issues in a field trial. This system
allow users to create the content themselves. The public display will have a major impact on the way people discover, consume
can be used to share the content and communicate with other and share video content.
customers and shopping mall services. Multi-touch can be utilized
as an interaction method in both mobile devices and public 8. ACKNOWLEDGMENTS
displays. In addition, NFC and Bluetooth can be used to
This work has been done in Smart 4D Project, funded by
automatically identify the users near public displays.
Multidiciplinary Institute of Digitalisation and Energy (MIDE).
Wide variety of categorized tags in the content allows the creation We want to thank Riikka Hänninen and Tomi Hokkanen, who
of public narratives, based on location of the display and on participated in the work.
presence of people and devices. Automatically generated tags
enable the linking of places, items, and persons as a part of the 9. REFERENCES
service content. The videos created by the individual users can be [1] Buur, J., Matthews, B., (2008) Participatory innovation.
viewed from public displays. They can be presented and International Journal of Innovation Management 12(3), 255-
organized based on multiple variables, such as location, time, 273.
devices and tagged objects related to a place or a person and
annotated comments. In addition, users can add annotations to the [2] Gutwin C., Greenberg S., (1999) The effects of workspace
timeline of the video, which can be either video or textual input. awareness support on the usability of real-time distributed
The information can be presented to users based on social groupware. ACM Trans Comput Hum Interact 6(3):243–281
connections and personal profiles, by combining recommendation [3] Hanington, B. (2003) Methods in the Making: A Perspective
systems and human computing. on the State of Human Research in Design. Design Issues 19
In the workshops, participants presented small prizes as (4), 9-18.
persuasive motivational triggers for contributing to community [4] Matthing, J., Sandén, B., Edvardsson, B. (2004) New Service
services. Several successful examples exist, which reward active Development: Learning from and with Customers.
users for participation. Social games have revealed that people International Journal of Service Industry Management 15(5),
appreciate rewards especially when they come from a person 479-498.
versus a machine and represent approval [6]. Small physical [5] Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt,
prizes, such as free coffee or lunch, were considered sufficient to A., Jay, T., Krüger, A., (2009) Display Blindness: The Effect
motivate users to contribute. The rewards could be granted based of Expectations on Attention towards Digital Signage,
on activity, contribution or actions related to certain locations, Proceedings of Pervasive Computing, May 11-14, 2009
shops or services.
[6] McGonical, J., (2009) Reality Is Broken: Why Games Make
Next step is to develop more robust and extensive version of the Us Better and How They Can Change the World, The
application with the enhancements described above and to test it Penguin Press HC, January 20, 2011, ISBN-10: 1594202850
in a public field trial in a shopping mall. Mobile application will
be available for smart phones, which is a small limitation. [7] Numa, K.; Sugimoto, T.; Miyata, M.; Toriumi, K.; Abe, J.;
Tanaka, Y.; Niida, S.; Hori, K., (2009) ,"Using common
devices as collaborative tools for collecting and connecting
7. CONCLUSIONS people's stories," Innovations in Information Technology,
In shopping malls, people would like to receive personalized 2009. IIT '09.115-119, 15-17 Dec. 2009
content and feel a sense of togetherness. Workshop participants
[8] Peltonen, P. Salovaara, S. Jacucci, G. Ilmonen, T. Ardito, G.
perceived the shopping mall as a shared living room where they
Saarikko, P. and Batra, V., (2007), Extending large-scale
can enjoy themselves and share experiences with friends. We
event participation with user-created mobile media on a
presented preliminary results from three participatory design
public display. In Proceedings of MUM '07. ACM, New
workshops organized to identify new service ideas and York, NY, USA, 131-138.
opportunities. We introduced our current implementation, which
enables context-aware mobile video creation, automatic tagging, [9] von Hippel, E. (2005) Democratizing Innovation. The MIT
annotation and social sharing. Public displays can be used to Press, London, England.
present situated interactive narratives, which engage the users and [10] Wallace, J. Scott, S. Stutz, T. Enns, T. and Inkpen, K. (2009)
simultaneously offer relevant information related to the context. Investigating teamwork and taskwork in single- and multi-
Our current findings suggest that such systems might benefit from display groupware systems. Pers. Ubiquit. Comput., 13:569-
gamification, integration of social sharing and automatic 581