This paper describes several designs and innovations that comprise an online multimedia storytelling system facilitating user-driven story authoring conducted directly by loosely formed communities sharing a common social event. The genre or format of the story is not limited by the design of the system, though we use narratives of personal experience in social events as the reference scenario. Other applicable use scenarios include citizen journalism, selfpromotion, interactive exhibition, collective awareness, etc. The system allows us to study and experiment with the entire eco-system of online storytelling, from video capture and annotation, through to discovery, editing and delivery. The system also incorporates social context derived from the analysis of social networks to improve user experience in creative editing. The synergy between social context information and multimedia delivery applications is also exploited to improve media retrieval and delivery.
The eco-system is comprised of the following four components.
A mobile application to streamline the capturing, tagging, sharing and browsing of user content. Metadata is essential for social sharing, but manual entry can be cumbersome during the live event, when users prefer to focus on the creative process of capturing content. The application should also capture metadata such as the geographical location of the user device at the time of capturing. A "lightweight" multimedia story-authoring engine suitable for heterogeneous user devices and networks. The biggest challenge of online storytelling using multimedia content is the process and delivery of audio-visual content for interactive user operations. An online story-authoring system should aim to make online video editing as easy as editing a shared text document and minimize the processing and network load at user devices [1].
Integration of social context. Widely adopted social networks greatly influence user preferences and activities. The rich context information and social atmosphere embedded in social media is crucial in improving user experience within social applications such as online storytelling, particularly for live events. Figure 3 illustrates the changing attributes and intensity of social interactions during the life-cycle of a live event, and how integration of social context benefits storytelling at different stages of the event.
An experiment was arranged at Schladming, Austria during the Nightrace 2014 event to evaluate the storytelling system. This experiment examined how the design of multimedia systems can help to facilitate social interaction, and how the integration of social context improves user experiences within such a system. A number of test participants travelled from the UK and the Netherlands to the venues prior to the event as the main storytellers while a few others joined the experiment from various locations in the UK. Caching nodes were also installed in the UK, Italy, and the Netherlands to study the effectiveness of chunk caching for story playback. The consensus among the test participants is that storytelling of group experience of an event is "a very natural thing to do". Most participants found that using the storytelling system for capturing and sharing their own creations throughout the course of a live event made them feel that they were "telling a live story to their friends". They were mostly adding the narratives while recording by talking to the microphone. Sometimes a member of a group spontaneously acted like a reporter and let the other group members talk about what had just happened. Social-context integration is effective in improving video annotation and in enhancing the search function during the experiments. Using information from Twitter, an adaptive event profiler (AEP) provides a list of related keywords and a metric to quantify the relevance for user search requests. An example for the search of "Schladming" is given below, with lower values representing higher relevance. The list covers a range of items such as sport, location (planai mountain), and popular competitors.
Because of AEP's ability to recognize trending events, the integration of social context presents an ideal solution to the classical "cold start" problem in content recommendation. Given a user location, the storytelling system may suggest stories related to socially trending keywords nearby.
Collaboratively authoring a story by joining shared multimedia content with different perspectives is becoming a popular way to enable collective user experience in live events. Our online multimedia storytelling eco-system enables the capturing, sharing, and authoring of user stories online with a low footprint, while an event-based user experiment highlighted the contribution of social features to enhancing the user experience of loosely collaborating groups in content annotation, retrieval and media distribution as part of a storytelling eco-system. We witnessed the unique role of collaborative user creativity in the entire development cycle of engaging social stories, and we have shown that social context derived from social media, location-based services and emerging mobile technologies can also greatly improve the creative story capturing and authoring process.
The work presented is supported by the European Commission within the FP7 Project STEER (grant no 318343).