We present an event-based approach for users to explore, annotate and share media. We are constructing a web-based environment that allows users to explore and select events, including discovering meaningful, surprising or entertaining connections among them. We build a knowledge base of events from event directories that will be linked to the Linked Open Data (LOD) cloud, in conjunction with event and media ontologies. The approach is user-driven and, having carried out initial user inquiries, we are designing interfaces that support user-identi ed tasks while exploring the connections between users, multimedia content and events.
As with all developing technologies, it is di cult to identify novel user needs that
can be satis ed with emerging semantic web technologies. At the same time, it
is di cult to develop the technology in speci c directions without knowing what
users are likely to want to do with the technology. In previous work we have
identi ed comparison search tasks that can be supported using a combination
of thesaurus-based linked data search and a modular user interface design [
In the context of the Petamedia4 Network of Excellence, we are exploring a
similar method for designing an application that takes into account the \triple
synergy" of users and their social networks, user-created content and metadata
attached to this content in an application for supporting users in interacting
with events. Events are a natural way for referring to any observable
occurrence grouping persons, places, times and activities that can be described [
through di erent media (e.g. videos and photos). We explore this intrinsic connection between media and experiences so that people can search and browse through content using a familiar event perspective.
While wishing to support such functionality, we are aware that websites already exist that provide interfaces to such functionality, e.g. eventful.com, upcoming.org, last.fm/events, and facebook.com/events to name a few. These services have sometimes overlap in terms of coverage of upcoming events and provide social networks features to support users in sharing and deciding upon attending events. However, the information about the events, the social connections and the representative media are all spread and locked in amongst these services providing limited event coverage and no interoperability of the description. Our goal is to aggregate these heterogeneous sources of information using linked data, so that we can explore the information with the exibility and depth a orded by semantic web technologies. Furthermore, we will investigate the underlying connections between events to allow users to discover meaningful, entertaining or surprising relationships amongst them. We also use these connections as means of providing information and illustrations about future events, thus enhancing decision support.
The work reported here uses an explorative user-centered design approach, where users are asked about real-world tasks they would like to carry out, and then asked for their opinions on speci c technologies that they are familiar with and how these might be used to support the tasks. This approach ensures making design decisions that contribute towards an e cient, e ective and satisfying user experience. Section 2 describes the method and the results of this user study, and presents the requirements for an event-based system for discovering and sharing media. Section 3 describes the event and media ontologies developed to support the semantic description of events extracted from event directories. Section 4 explains the design rationale of the interfaces, and gives some interface mockups to illustrate the types of task support and expected functionality we will be providing in the coming months. Finally, we give our conclusions and outline future work in Section 5. 2
We follow a user-centered design process consisting of an assessment of user needs and insights, identi ed through interaction with potential users at di erent stages of development. Our research starts by identifying who the users are, their interests, their goals and which tasks need to be supported in order to achieve these goals. We collect this information to de ne a rst set of requirements and identify prospective scenarios that illustrate the environment task scope and a rst design concept. The steps that follow consist of iterative cycles of re-design and evaluation until a satisfactory design is reached. 2.1
The rst step of our research was done in order to collect potential end-user experiences, opinions and interests while discovering, attending and sharing events, and user insights about potential web-based technologies that support these activities. We collected this initial input through an exploratory study with 28 participants (11 females). Participants were mostly students and researchers with ages varying from 23 to 47 years old. The study was done through an on-line survey with 8 questions divided into 2 sections. The same topics were then presented in discussion sessions with two groups of master students totalizing 35 additional participants: One discussion was done with students (n=10) from an Interactive Multimedia Systems course and the other with students (n=25) from a Human-computer interaction for the Web course. The results from these discussions were used to validate the survey responses and to extend it with other collected insights.
The rst half of the survey aimed at identifying participants' personal experiences and behaviors. It invited them to recall memorable previously attended events (e.g. festivals, conferences, concerts, art galleries, exhibitions, gatherings) and to share their opinions and experiences regarding: (1) how events are discovered; (2) characteristics that support deciding rather or not to attend to an event; (3) how the event experiences are registered and shared; and (4) meaningful, surprising or entertaining relationships amongst events.
On the second part of the survey, participants' were asked to share opinions regarding existing web technologies in the context of the aforementioned activities. To better address the triple synergy paradigm (Section 1), we explored the concept of merging event directories, media directories and social networks. With that in mind, we asked participants to share their opinions regarding: (1) the perceived bene ts and drawbacks of event directories (e.g. Eventful); (2) enabled possibilities, bene ts and drawbacks of combining media sharing websites (e.g. YouTube and Flickr) with event directories; (3) enabled possibilities, bene ts and drawbacks of combining social networks (e.g. Facebook and Twitter) with event directories; and (4) suggestions regarding desired and useful features.
Answers obtained from the survey were analyzed through a nity diagramming. The process consists of iterative clustering cycles which allow organizing the collected ideas into common themes, thus allowing to identify the most common opinions for each raised subject. The results from this rst exploratory study are described in the following section. 2.2
In this section, we present a summary of the results of our user study. The summary contains main reported experiences, interests and opinions around event related activities.
Past experiences. Concerning participants' experiences when discovering events, the vast majority reported to nd out about events through invitations and recommendations from friends and colleagues. Traditional media such as posters, yers, news articles and television ads seem to play a major role when discovering events. Social networks were also reported to be used, with speci c reference to event posting and invitation features. More seldom participants use event directories (e.g. Livenation, local city event directories, Ticketmaster, last.fm) or participate in mailing lists, newsletters and forums to obtain updates. The use of search engines was reported, speci cally when they knew what to look for. Moreover, participants also rely on previously attended events or venues as reference for nding new events. During the group discussions, participants seemed to rely more heavily on social networks in comparison to the survey responses.
When deciding whether or not to attend to an event, participants seem to prioritize background information. Location was often referred to for orientation and because of distance constraints. Price was commonly mentioned to allow identifying cost-bene t ratios and due to budget constraints. Time of the event was a main decision factor, followed by information about who else would be joining the event. and more speci cally, which friends will attend. The content of the event itself (e.g. type, performer, topic) and subjective factors such as fun, relevance, interest, atmosphere, target audience and reputation were also mentioned. Students from the group discussions, preferred the event attendance (\who's joining?") and price constraints over all other characteristics.
Regarding how participants register the experience, they often take pictures for sharing after the event. Less commonly, participants record short videos. As for the how they share the information, they most commonly talk to others, describing their experience. Participants share the collected media directly (e.g.
le transferring, showing on the mobile) or use media directories and social networks such as Facebook, Flickr or YouTube.
Concerning relationships between events, the most referred to characteristics that motivate participants to look into related events were the event categories (e.g. type of event, topic, genre). Another important factor was the event attendees, to nd other events they would attend. This could refer to groups of people (i.e. target audience, users with similar interests), but most importantly, individuals in their social networks. Other main event characteristics also mentioned were: location, performers, organizers and time/duration. Lastly, future events from repeated events was also seen as a strong relationship. Existing technologies. Existing event directories (e.g. eventful) perceived bene t was clearly to be a single access point providing an overview of event information. Another reported bene t is that it supports opportunistic event discovery and facilitates exploration based on di erent contexts (e.g. location, popularity, categories). Other positive features include: social features (e.g. commenting, sharing events), noti cation of upcoming events, and shortcuts (e.g. ticket purchase). As for the drawbacks, the main reference was about the unreliability (i.e. unknown source) and incompleteness of information. In particular, low coverage of events and insu cient information for decision support (e.g. lack of location map, videos) have been mentioned. In contrast, the information overload was also seen as a potential drawback making it di cult to nd speci c events.
When presented the possibility of combining media and event directories, participants recognize bene ts due to information enrichment. They claim it would help illustrate events with videos and pictures of past related events (i.e. past performances), other people's experiences, promotional (marketing) material, and so on. The main recognized value would be to give a better idea about the event's environment/atmosphere and provide visual information to support decision making. Participants said it would also support remembering and sharing past experiences. Drawbacks from the merger concern information overload and privacy issues while sharing personal media.
Regarding the possibilities a orded by merging social networks and event directories, some participants think that the main bene ts are communication between users, and the sharing of more information (e.g. invitations, opinions, pictures). It was also said to facilitate viewing event attendance, identifying event popularity, and even provide an overview about friends' whereabouts. Live event information (e.g. real-time tweets and comments, live pictures/streams) updates were also seen as a positive a orded feature. Despite the bene ts, some participants think the amount of information could clutter the service. Others pointed out that services such as Facebook already provide enough event sharing features. Suggestions included making use of existing social network pro les and/or extending these services.
Other features that users would appreciate having when dealing with events were broadly described with little overlap. Some of these features are: recommendations (based on past attendance, preferences, and from people with similar interests); better visualizations for exploring and searching events (e.g. map integration); the potential to combine categories and attributes while browsing; obtain more information about events and users (e.g. opinions, price and availability).
Conclusions. The opinions gathered seem to support the development of an environment that merges event directories, social networks and media sharing platforms. Moreover, this information enrichment is thought to provide better means of supporting the decision making process. This assumption is based on the possibility of allowing users to better experience an event by viewing associated media. On the other hand, social information obtained implicitly (behaviors) and explicitly (comments, reviews and ratings) provide better judgments of events in terms of attendance, shared interests and reputation. A common concern about information overload suggests that the interface should avoid cluttering and provide only necessary information. Furthermore, there is a need to support di erent visualizations and better browsing possibilities depending on user interests and constraints. Lack of event coverage and information completeness is another important identi ed issue that can be addressed using and combining multiple information sources. These issues, along with other identied user interests were translated into a set of requirements in order to guide the following steps of the environment design and development. The following section describes these requirements in more detail. 2.3
Based on this user study, we de ne a rst set of requirements, translating user
needs into functionalities that the system should support (Table 2.3). It is
important to note that the requirements presented here are representative of users
who participated in the previous studies. They should be complemented with
other non-functional and functional requirements as described in existing design
patterns and interface guidelines [
Discovering
Provide a comprehensive coverage of past and upcoming events Allow searching events based on tags (e.g. performer name, genre, title) Allow opportunistic discovery by ltering and combining properties (e.g. categories, location, time, price) Inspecting
Show complete background information about events (e.g. title, location, description, venue, performers, time, category, genre, availability, size) Allow identifying subjective aspects of events (e.g. popularity, fun, atmosphere, reputation) Show media associated to events for reliving experiences and for decision support Show who is joining or joined the event (attendance)
Allow identifying related and repeated events Visualizing
Rely on traditional media information display (e.g. posters, yers, ads) Show only the necessary information in a simple way
Allow di erent visualizations and browsing contexts (e.g. time, location, people) Enriching
Allow creating events Allow associating pictures and videos with existing events
Allow associating comments and opinions with existing events Sharing
Make use of existing social networks (e.g. Facebook, Twitter)
Allow inviting and recommending events using existing services Recommending & Preferences
Allow receiving recommendation about events based on personal interests and
behaviors
Allow receiving recommendations based on other people's preferences and
behaviors (collaborative ltering)
Identify interests and preferences based on past event attendance
Scenarios are informal narrative descriptions that allow exploration and
discussion of context, user needs and requirements [
We provide below four di erent scenarios, each from one di erent persona. Scenario 1: Johnny was invited to a party by a friend and receives a link providing information about this event. He wants to know when and where this event will be and who else was invited. More importantly, he wants to know whether his closest friends con rmed to attend the event or not.
Scenario 2: Julie would like to go to a play on her favorite theater. She wants to see a comedy, hopefully playing the upcoming week. She has only been to a few comedies, but she remembers one she speci cally enjoyed. Julie would like to see if there is something similar playing and read what other people say about it. Scenario 3: Jack recorded a video with his mobile phone camera while he was attending the Haiti Relief concert from Radiohead given on 24 January 2010 in Los Angeles. He thinks it was a really nice experience and wants to share it on-line. He would also like to see what other pictures and videos were captured during the concert and see how other people experienced the show. Scenario 4: Jessica is going to Paris on her honeymoon and she would like to see what will be happening there during her stay. She wants to do many di erent things, but cannot decide yet, so she wants to put these things on a \maybe" list in order to decide later. If possible, she would like to see videos of these events to make sure it has a cozy and romantic atmosphere. 3
In this section, we present the ontologies used for representing events, media and users metadata. We use the scenario 3 described above to illustrate these models. 3.1
The LODE ontology5 is a minimal model that encapsulates the most useful
properties for describing events [
rhetorically produced will depend on one's epistemological stance. We exclude,
at this stage, properties for categorizing events or for relating them to other
events through parthood or causal relations. We will see in the next section how
these aspects, that belong to an interpretive dimension, can be handled through
the Descriptions and Situations approach of the Event-Model-F [
The Figure 1 depicts the metadata attached to the event identi ed by 1380633 on last.fm according to the LODE ontology. More precisely, it indicates that an event of type Concert has been given on the 24th of January 2010 at 20:00 PM in the Henry Fonda Theater featuring the Radiohead rock band.
LODE is not yet another \event" ontology per se. It has been designed as an interlingua model that solves an interoperability problem by providing a set of axioms expressing mappings between existing event ontologies. Therefore, an OWL-aware agent would infer that the resource identi ed by dbpedia:Radiohead is a dul:Agent as described in the Dolce Ultra Lite ontology. 3.2
The Ontology for Media Resource currently developed by W3C is a core vocabulary which covers basic metadata properties to describe media resources6. It also contains a formal set of axioms de ning mapping between di erent metadata formats for multimedia. In the Figure 1, we see that the video hosted on YouTube has for ma:creator the user aghorrorag.
The Ontology for Media Resource can then be used to attach di erent types of metadata to the media, such as the duration, the target audience, the copyright, the genre, the rating. Media Fragments can also be de ned in order to have a smaller granularity and attach keywords or formal annotations to parts of
the video. The link between the media and the event is realized through the illustrate property, while more information about the user could be attached to his URI using for example the FOAF ontology. 3.3
The pattern-based ontologies M3O [
We use the Last.fm API to convert the event description into the LODE ontology (Section 3.1) and the Flickr API to convert the media description into the Ontology for Media Resources (Section 3.2). The result of this operation is a minimal description of the events where all values are strings (literals). Therefore, we perform an additional step in order to update this description into a truly linked data one. We invoke semantic web lookup services such as the dbpedia one in order to transform these strings into URI identifying unambiguously resources in the web of data. Hence, the "Radiohead" string is transformed into a dbpedia URI7 which provides additional information about the band such as its complete discography. This URI is declared to be owl:sameAs another identi er from the New York Times8 which provides information about the 38 associated articles from this newspaper to this band. The venue has also been converted into a dbpedia URI9 but has been augmented with geo-coordinates thus increasing the amount of information available in the LOD cloud for the bene t of all semantic web applications.
The linked data journey can be rich and long. One of the challenges we want to address is how to visualize these enriched interconnected datasets while still supporting the user tasks identi ed in the Section 2.3. 4
In this section, we illustrate initial interface possibilities derived from the requirements and tasks presented in the Section 2.3. The interfaces are represented through low- delity prototypes. The prototypes allow exploring, re ning and validating prospective concepts along with interface and interaction aspects through small studies with potential end-users and usability experts. Unsurprisingly, the sketches below correspond to the basic properties de ned in the LODE ontology.
8 http://data.nytimes.com/N12964944623934882292
4.1
What - One prospective view is media centered and allows to quickly illustrate the event through associated media. In this view we display events through a representative images and convey di erent event characteristics (e.g. relevance, rating, popularity, etc) with one and/or more of the image properties, i.e., size and transparency. This approach has been used in other applications10 to represent clustered result sets or convey sorting by size on di erent contexts (Figure 3a).
When - Ordering can also be used to represent chronological event occurrence. In fact, the time centric view can be interpreted as the sorting of events chronologically (Figure 3b).
Where - A location centric view can be used to represent where the events occur geographically to orient the user and convey distance. The use of maps is commonly used to visualize such information (Figure 3c).
Who - Events are intrinsically bound to a social component. Users want to know who will be attending to an event when deciding to attend to it. In this context, a people centric view would be relevant to explore the relationships between users and events. Alternatively we can combine attendance information to other views such as location, allowing users to browse for friends on a map and identify their attended events. It could also be used to provide means of visualizing event popularity ,e.g. identify the cities hot-spots on a map, indicate visual cues of popularity according to number of attendees.
In order to allow users to relive experiences from events attended in the past,
follow future con rmed events, and keep track of authored events, it is necessary
to display events in the context of the users own attendance and ownership.
For this reason we will support a \my events" feature with overall browsing
possibilities. If several views are to be supported one challenge that can arise
concerns transitions between these di erent views. This is speci cally important
for facet browsing, due to sudden disappearance of items during navigation [
When discovering events we believe users will also rely on browsing, which allow
them to analyze large sets of event sets, and narrow them according to their
interests and constraints. Overall, we believe users will have di erent information
or browsing/search needs as follows:
{ Navigational - when the intention is to reach a particular known event;
{ Contextual Browsing - discover one or more events given a speci c context
(e.g. by location, performer, type, time);
{ Entertainment Browsing - serendipitous and opportunistic discovery of events;
10 See for example http://www.jinni.com or http://www.ted.com/talks.
Since it is often easier to recognize a word or name than it is to think up that
term, it is useful to prompt users with information related to their needs. Based
on that principle, we will explore the use of hierarchical faceted metadata which
will allow users to browse through multiple categories, each corresponding to
di erent dimensions of the collection [
A B
C
While trying to reach a speci c event, traditional keyword search can be done
through entry forms. Dynamic term suggestions or auto-completion can be used
to provide rapid and e ective user feedback by suggesting a list matching terms
as the user types the message. Semantic auto-completion extends this method
by providing means of clustering the terms according to di erent categories or
facets [
When representing an event instance, we show all information needed to support the decision making process (e.g. Figure 4). Since experiences are centered around media content, we wish to explore di erent media that better illustrate the event to end-users. Some information that can support decision making are the following.
{ background information (e.g. performers, topic, genre, price, attendance list, etc) { subjective or computed attributes (e.g. reputation, fun, atmosphere, audience) { user opinions, comments and ratings (strangers and friends) { representative media (ads, media from past related events, media from the audience, etc.) Apart from the inspection of the event instance, other conceptual classes (e.g. users, venues, performers, media) should also have accessible views, so that the user can obtain more information about these instances and explore events related to them. In future work we will also identify what are the relevant associated information and how to represent navigation from and to these nodes. 4.4
Regarding event content enrichment, interfaces that allow users to add/upload information and assign such information to events will be investigated and explored in future studies.
One of the required enrichment features refers to assigning user attendance
and keeping track of the users' previously attended events. This information can
be used so that the user can easily access past experiences. Moreover, attended
events may be used to identify user interests for recommendation and
personalization of the facet-pears during search [
Finally, since users are likely to revisit information they have viewed in the
past [
In this paper, we have described an event-based approach for users to explore, annotate and share media. We rst conducted a user study where users were asked about real-world tasks they would like to carry out. We have then extracted requirements and described some scenarios for an event-based system for discovering and sharing media. We advocate the use of linked data technologies for integrating information contained in event directories and we described how event and media ontologies can be used. Finally, we present some sketches of user interfaces that we will develop in the coming months.
In following studies, we intend to use the scenarios we have written to understand how end-users interpret and ful ll associated goals. This will allow us to identify patterns of interaction, information seeking strategies and information sources required to complete the described tasks. We will continuously increase our coverage of event directories by scraping more data sources and hence demonstrating how interoperability problems can be addressed using semantic web technologies. 6
The research leading to this paper was supported by the European Commission under contract FP7-216444, Petamedia Peer-to-peer Tagged Media, and contract FP7-215453, WeKnowIt. The authors would also like to thank Hyowon Lee from Dublin City University (DCU) for fruitful discussions on the design of the EventMedia interfaces.