The prevalence of Social Media in sharing day to day information regarding all aspects of our life is ever increasing. More so, with access to cheap Internet-enabled devices and proliferation of Social Media applications. Among the variety of information shared, the most relevant, in the context of this paper is how individuals assess their surroundings and how they or their loved ones are a↵ected by adverse events, disasters and crises. Traditional channels of communication often fall behind in providing timely information for emergency responders to formulate an accurate picture of the situation on the ground. The role of Social Media in complimenting such sources of information is thus invaluable and Social Media has been recognised as a key element of assessing evolving situations. Timely, accurate and ecient means to explore and query Social Media is essential for an e↵ective response during emergencies, and hence this gives rise to a Knowledge Management issue. Our paper presents our approach to analysing Real-Time Social Media data streams using Visual Analytic techniques. We discuss the highly visual and interactive approach we employ to provide emergency responders means to access data of interest, supporting di↵erent information seeking paradigms.
The challenge of gathering a good understanding of large volumes of Social
Media data streams is a significant one. The nature of Social Media itself, being
highly dynamic, multi-lingual, geographically distributed, and highly relevant
to the short-term zeitgeist poses enormous challenges. Additionally, the highly
repetitive and noisy nature of Social Media also adds significant challenges to
analysis e↵orts. In spite of these challenges, the potential of Social Media is
immense and has been recognised as significant in Emergency Response by various
organisations. Social Media empowers analysts with better means to understand
public perception of events and situations on the ground as well as facilitating
decision making processes for planning rescue operations12 [
The goal of our research is to facilitate exploration of large volumes of Social
Media information for gathering a very quick understanding of a situation at
hand. It is important to note that this paper discusses events and situations
from a broader perspective – events refer to anything that occurs3 that is of
interest to an analyst, while situations relate to the conditions and state of
a↵airs (including events that occur to shape situations). To this end, we have
developed a system, TUI (Tracking User Intelligence) [
This paper is structured as follows: the next section describes related work. Section 3 presents an overview of the TUI interface. Section 4 describes how various types of information needs are addressed by elements within the framework, and how they are relevant to Emergency Response. Section 5 briefly discusses several field studies we conducted in real world examples and we conclude the paper with lessons learned and a discussion of future work. 2
We present related work in two areas: Situational awareness and Visual
Analytics. Situational Awareness in emergencies is paramount to deliver a timely
and e↵ective response [
disaster-relief-20-future-information-sharing-humanitarian-emergencies
2 http://www.unocha.org/hina
3 http://www.complexevents.com/2011/08/23/event-processing-glossary-version-2-0/
and authorities [
Visual Analytic techniques have been proposed to represent and filter Social
Media at di↵erent levels of specificity [
We designed our solution centering to the di↵erent modes of Information Seeking
proposed by Bates [
TUI is a multi visualisation platform that consists of multiple views over
di↵erent facets of Social Media data. Users can interact with the system via
each visualisation widget or a set of generic query elements to define filters. The
user interface is separated from the backend processing, in order to deliver a
more responsive performance, and hence our solution consists of two independent
solutions. Several interaction paradigms govern how users can interact with the
system, and therefore, provides multiple ways of engaging with the data. While
being founded on the four information seeking models of Bates, TUI supports
Schneiderman’s well-known information seeking paradigm “overview first, zoom
and filter, and details on demand” [
Figure 1 shows the four main components of the TUI interface. The interface
is designed to enable users simultaneously access the most important information
that is relevant to their present session. We enable this by providing a set of
tabs (Section A: Contextual Information Visualisation), each presenting a piece
of contextual information depending on the user’s information need. The tabs
(most relevant ones are discussed in the paper) are as follows: Map View (‘Map’),
Timeline View (‘Timeline’), Graph view (‘Graph’), Chart View (‘Chart’), Trend
View (‘Trend’), Freq View (‘Popular’). Map view provides contextual spatial
information on a geographical map, while timeline provides visual summaries on
a temporal scale. Graph view provides topical information, presented as a graph
of topical relation among Social Media messages, where each topic is visually
encoded based on the number of occurrence, associated collective sentiment or
any other parameter that is decided as appropriate. The type of graph that is
deployed depends on the particular use case, and TUI can select from a set of
candidate graphs based on the number of topics. For example, a typical
nodelink graph can present a lower number of topics, but can provide an easier
way of communicating relational information between co-occurring topics. The
Context and Hierarchy chain [
The Social Media posts section (Section B: Content Presentation) shows the data instances, where each Social Media post is presented as a snippet, chronologically listed. The display of Social Media posts is designed to address the last part of Schneiderman’s ‘details-on-demand’ paradigm, where the session of user exploration dictates the relevant content to be displayed. The central part of Schneiderman’s paradigm, ‘zoom and filter’ is the final filtering section (Section C: Filters). The filtering interface provides two functions – visual communication and overviews, and filtering. This section presents tag clouds of several facets of the data such as authors, places, type of post (photo, video, audio, link etc.), keyword or hashtag. Additionally, other interaction mechanisms such as text entry boxes, sliders, calendar widgets and buttons are presented to users for entering specific filters (or queries). Overall, interactions within Section A and C result in defining the context of the user’s information needs and eventually retrieve the content for Section B. The Real-Time Updates section (Section E) provides a real-time reflection of the background data collection: this informs the user that in the time that has passed since the last time the visualisations have been refreshed, there has been a number of new messages that have been collected. The user has three choices: ignore the message, view only the new messages, or view all the data that has been collected. The final component presents the filters and selections that are active on the present exploration session of the user. These filters are generated by the user by interacting with visualisations and interface elements from the Sections A and C. Users can disable each filter by clicking on the (x) button if required. 4
Within the scope of Emergency Response, Searching involves directed and active information seeking activities, where the user has a very specific information need, and knows the information exists. For example, an emergency responder is investigating a situation at hand (such as a flood) and is looking for more information related to the state of rivers overflowing. Monitoring involves the user having a specific information need, but is unsure if the information exists and hence monitors for relevant information. For example, an emergency responder is aware that flooding is a regular occurrence in a city, and is monitoring the condition of the city with a higher than usual predicted rainfall. Browsing, on the other hand involves the user actively looking for any relevant information, but in an undirected manner. An example of browsing would be an emergency responder looking for any occurrence of floods within a large geographical area. Being aware involves the user being passively looking for relevant information in an undirected manner. For example, an emergency responder looking for anything which can indicate an incident occurring that might be of interest to be further investigated.
As explained previously, Schneiderman’s information seeking paradigm was central to the design of the interface. Bate’s information seeking model provided a basis for understanding how users may need to search/explore/monitor existing situations using the system. During initial design phases, Bate’s model along with interviews and focus groups with emergency responders and practitioners provided realistic scenarios that can explain what are the information needs of users at di↵erent stages of an emergency situation. This section presents four main scenarios that relate to the di↵erent elements of Bate’s model, and how TUI can be typically used to identify data of interest. It is to be noted that while all of the aspects of the model requires active searching, the manner in which the user can trigger the searches and the follow up tasks play a role in addressing di↵erent types of information needs.
The filters section (Section C) is key to this task, where the user can drilldown to a set of data instances of interest, based on the information need of the user. The selection can be done using di↵erent interaction techniques and via a variety of facets. The variety of facets can be observed on the accordion menu on the right – three levels of filters are provided to users. Figure 3 describes how these filters are organised - Content filters, Immediate Context filters and Wider Context filters. Information that can be easily extracted from the content of the Social Media posts are referred to Content Filters. The content filters are the features that are aligned the closest to the Social Media post itself. A minimal analysis of the Social Media posts can provide a few more features, which have been organised into Immediate Context. While some features like Language or Type can be easily retrieved from the data provider, they may require some basic analysis to interpret content and infer. The Wider context filters indicate the features that require a further level of analysis/interpretation/query. Features such as identifying locations may be easily provided by geolocated posts, but the process may be considerably more dicult if there is a need to analyse text to interpret locations. Information surrounding a post such as the relevant users (who posted, who replied or who were mentioned) can be retrived by further queries to the data providers and may require further processing.
A combination of filters can be used to identify an initial subset of the data, and then users can progressively apply further filters to reach the data instances that are of interest. For example, if the user is interested in the negative/positive messages that have had a high impact among the users, he/she can make a combination of selections of emotions/sentiments as well as selection of a high number of repeated messages. Observing the hashtags or keywords can then provide a rough summary of the topics of discussions and provide a greater overview of the situation. 4.1
The first, and possibly most relevant to most Semantic Web applications is the ability to search for existing information, when there is a highly specific information need. Within the context of Emergency Response, the need to search for information, plays a significant role once an incident has been identified. Another role searching plays is post-event analysis where all data is searched to understand how an event evolved from di↵erent perspectives. E↵ectively, searching provides answers to questions that are already known by analysts and to facilitate follow-on analyses. In order to perform searching tasks, users simply enter (or select) an appropriate set of filters and reach the data of interest. Section C (filtering section) is the most appropriate in this context, as it provides direct means for users to search. Interactions within Section A (visualisations) also generate filter queries but such features are more pertinent to browsing activities. The search terms that are involved in this process are highly dependent on the evolution of the event as well as the event itself. Hashtags and keywords vary on how they have propagated within Social Media, and therefore, it is dicult to predict which terms would best suit the scenario being investigated. This calls for the need to monitor situations based on an initial ‘guess’ to encapsulate possible terms, and then fine-tuning queries to capture and retrieve more relevant information. 4.2
Perhaps most relevant to Emergency Response is the notion of monitoring, where the analyst needs to monitor an evolving situation, and looks out for information that may be relevant. This is achievable by setting queries and filters, and during standard data exploration via visualisations (Section A) and reading Social Media content (Section B), users can be updated when new relevant content is made available after harvesting recent Social Media posts. Once new content is available that is relevant to the present search criteria, users are communicated by a clickable label which states ‘X new messages available’. The analyst is faced with one of three options in such situation: the first being, continue exploration and ignore the update. This would have no e↵ect on the present exploration, and the user can proceed with finishing his/her analysis. With more information being made available, the notification is updated. This is enabled/disabled via the dropdown (session type) selection option ‘Static’ (the other options are ‘dynamic’ and ‘batch-update’). The second option the user has is to proceed with analysing only the new content that has been retrieved. This notes the time when the previous analysis had started (the time when the last query was triggered). Selecting ‘batch-update’ from the dropdown enables this option and the user can then click on the button to view the relevant data instances that have been recently added. The third option the user has is to proceed analysing all of the data, with the new instances added to the analysis. This is performed by selecting ‘static’ and clicking on the click-able label.
The last option is a dynamic monitoring option, which regularly updates with new posts continually appearing during exploration sessions. This options is the least used, and hence is presently not encouraged to be used – more work is needed to understand how continuously evolving data can be presented to users without causing confusion and loss of analysis e↵ort.
Browsing involves users looking for information as and when their interests evolve. This is a mode which is also highly relevant to Emergency Response. Monitoring continuously evolving data, and searching existing data can indicate interesting events, keywords, or hashtags that might be relevant to improve the understanding of an evolving situation. In our approach, interactive Visualisations are key to this: while exploring a relevant dataset, users have the possibility to add new filtering terms by clicking elements from tag clouds, charts, timelines or maps. Clicking on sensitive areas of the visualisations immediately triggers queries, which result in a new filter being added and the system to drill-down into a more relevant subset of the data being explored. The user can remove the recently added filter by deselecting the filter from Section D (Current Selection). 4.4
The most complex of information seeking modes is the state of being aware. TUI facilitates this by easily combining some of the approaches that are employed in the other three modes. The state of being aware in Emergency Response implies that an analyst is aware of the wider context of his/her analytic activity, but unaware of what event/situation may arise. This is often perceived to be the precursor to some of the modes and is conducted as a part of an initial or continued survey of the broader area (geographic, temporal or topical). In a typical ER (Emergency Response) activity, it is expected that an instance of the interface is continuously dynamically updated to provide overviews, based on query terms that are most likely to retrieve interesting events – generic terms such as ‘flood’, ‘accident’, ‘911’ or ‘fire’ could form a good candidate set of keywords to look for4. Identifying any relevant interesting event would then initiate monitoring, browsing and searching activities on other instances of TUI. 5
The TUI interface is the second part of two systems - the first being a Social Media harvesting system which enables gathering of Social Media posts, and the second being a system which interfaces with the harvesting system and a local data store. While the first system (harvester) involves querying multiple sources of data continuously for new information and storing the results in local datastores following several iterations of backend processing, the second (TUI) is the interactive interface that users can use to access the data being stored/analysed/harvested. TUI is implemented as a standard web application, and written completely in HTML and Javascript. Several toolkits have been used 4 It is to be noted that several of such ‘generic’ terms can also occur in posts that are highly irrelevant such as song lyrics, quotes from speeches etc. The decision to select relevant terms is mostly left with the analysts, based on their interpretation of the content being currently monitored. within TUI to facilitate interactions such as jQuery5, and provide customised look and feel such as jQueryUI6, Less7 and BlockUI8. Visualisations are provided by D3.js9, Highcharts10, Google Maps11 and Javascript Infovis toolkit12 6
Over the summer of 2013, as a part of multiple projects, TUI was used to monitor several large events across the UK. Several Emergency Response organisations, Police, City Councils, event organisers and authorities were involved in the events. During the events the di↵erent types of information seeking modes were employed, and the system was used to monitor by several analysts at the same time. One of the key findings was the need for supporting multiple types of information seeking at the same time. Analysts need to explore, query, browse and be aware of situations at the same time, and hence, several instances of TUI are necessary to be active for improving Situation Awareness.
The TUI system is currently being redesigned to provide support for multiple tasks within one interface. Several layouts are presently being evaluated to understand which layout from a set of candidate layouts would be the most e↵ective. The system is also planned to be evaluated with emergency responders during planned and unplanned events. Several techniques such as focus group, contextual inquiry and shadowing are planned to be used in the evaluations to understand how the system compliments traditional techniques for Emergency Response. 5 http://jquery.com/ 6 http://jqueryui.com/ 7 http://lesscss.org/ 8 http://malsup.com/jquery/block/ 9 http://d3js.org/ 10 http://www.highcharts.com/ 11 https://developers.google.com/maps/ 12 http://philogb.github.io/jit/
Mazumdar et al.