=Paper=
{{Paper
|id=Vol-2009/fmt-proceedings-2017-paper14
|storemode=property
|title=Evaluation of the User Experience of Interactive Infographics in Online Newspapers
|pdfUrl=https://ceur-ws.org/Vol-2009/fmt-proceedings-2017-paper14.pdf
|volume=Vol-2009
|authors=Julia Langer,Michael Zeiller
|dblpUrl=https://dblp.org/rec/conf/fmt/LangerZ17
}}
==Evaluation of the User Experience of Interactive Infographics in Online Newspapers==
https://ceur-ws.org/Vol-2009/fmt-proceedings-2017-paper14.pdf
Evaluation of the User Experience of Interactive
Infographics in Online Newspapers
Julia Langer Michael Zeiller
Information Technology and Information Management Information Technology and Information Management
University of Applied Sciences Burgenland University of Applied Sciences Burgenland
Eisenstadt, Austria Eisenstadt, Austria
1410639004@fh-burgenland.at michael.zeiller@fh-burgenland.at
Abstract— Information graphics are a powerful tool to appropriate medium of communication for storytelling – apart
communicate complex information. Adding interactive elements from simply attracting attention due to beautiful graphics. In the
to infographics that are published in online media enables publishing phase (interactive) visualizations can play multiple
journalists to tell even more complex and exiting stories. However, roles: they help to illustrate new insights in a more compelling
the usability of such complex data presentations is crucial for their way, they can remove unnecessarily technical information from
acceptance among readers of online newspapers. The results of a prose, or they offer a new perspective since they can show
usability study of interactive infographics published in online changes over time, show connections, or compare values much
newspapers reveal weaknesses and success factors for designing more efficiently than text [6].
interactive infographics to ensure an improved user experience.
Stories published in online media can take advantage of
Keywords—interactive infographics; information graphics; narratives including complex graphics and especially interactive
information visualization; interaction; usability; user experience; infographics. Especially in online newspapers we find an
data-driven journalism; online journalism; online newspaper. emerging number of stories including interactive infographics.
Due to interactive elements readers can explore the data and can
I. INTRODUCTION control by themselves which and how much information shall be
Information visualization describes the use of visual displayed. Adding interactivity introduces an additional level of
representations of abstract data to amplify cognition [1] [2]. The required skills to users (i.e., data literacy) to control and navigate
visual representation of information enables users to effectively within the interactive graphics. Additionally, inadequate user
and efficiently perceive, recognize and interpret information. experience, flaws in the infographics’ usability and simple
Especially information graphics (short: infographics), that mistakes in the interactive presentation can lead to wrong
combine graphics, image and text, are an efficient means to conclusions and force readers to stop exploring the infographics
communicate complex information, data or knowledge [3]. [10].
Static representations can be enhanced with interaction (e.g., Although interactive infographics are increasingly used in
filtering, selection, input of data, navigation) to provide users online media, readers face the challenge of finding and getting
with different ways of controlling how and which kind of access to the interactive infographics because they are not
information is presented [4]. Since Shneiderman [5] proposed marked properly and not all control elements for interaction can
his Visual Information-Seeking Mantra: Overview first, zoom be identified [11]. Since previous studies (e.g., [11]) reveal a
and filter, then details-on-demand, interaction has been a key significant lack of convenience during the utilization of
principle for the success of information visualization. interactive infographics in online newspapers, this paper
Due to their capability to communicate complex data, focusses on the user experience of interactive infographics that
information and knowledge efficiently, infographics are often have been published in German-speaking online newspapers.
applied in data-driven journalism. In data-driven journalism The results of a usability test, that has been applied to several
large amounts of data are collected, evaluated, interpreted and interactive infographics, are presented. The usability test reveals
presented to readers [6] [7]. Lorenz [8] defines data-driven some weaknesses, but also success factors that can help
journalism as a workflow where data is the basis for analysis, journalists and designers of interactive infographics to improve
visualization, and storytelling. Based on large amounts of data, the user experience for readers of online news stories.
data journalists explain new insights and tell complex stories that Section II gives a short introduction to interactive
are enhanced by (interactive) visual representations [6] [9]. infographics which is followed by a brief overview on related
Information visualization supports data journalists in work on usability of (interactive) infographics in section III. In
multiple ways. In the reporting phase information visualization section IV we introduce the usability test method based on
helps them to identify themes and questions, to identify outliers, Thinking Aloud and a questionnaire. Sample results on selected
or to find typical examples [6]. When journalists publish stories infographics and selected ergonomic principles are presented in
based on their investigations, information visualization is an section V. We end up with some remarks on future work in
section VI and conclusions in section VII.
97
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
II. INTERACTIVE INFOGRAPHICS information. In contrast, a high level of interactivity enables the
user to fully explore the infographics. He/she can interact with
A. Information Graphics information by input of data, retrieving data, or filtering, thus
Information graphics are visual representations of changing the content.
information or data, e.g. as a chart or diagram, telling a specific The activity model identifies the way users can interact with
story [3] [12]. They combine graphics, image, text and numbers the infographics and distinguishes between:
to communicate information, data or knowledge efficiently [13]. Linear
Infographics can be used to communicate complex topics and Nonlinear
draw the attention of percipients to them. They provide the Linear-nonlinear
percipient with new insights and a quick overview on complex
facts on subjects like politics, science, technology, and nature The linear type restricts the user to move forward or
that are hard to understand just using text-based information. backward through a predetermined linear sequence [19]. The
However, despite obvious advantages there is an ongoing debate step-by-step course is predefined by the author, i.e., this is an
on visual embellishment. author-driven style of interaction [18]. The user can only follow
a strict path using navigation tools like Start, Stop, Forward,
While trying to create appealing infographics designers have Backward, or Next and cannot explore the visualization by
to prevent from adding unnecessary visual embellishment – himself. [16]
chart junk. They should adhere to a reduced approach using plain
and simple charts, e.g., by following the data-ink ratio for non- In contrast, a nonlinear visualization does not provide a
interactive infographics proposed by Tufte [14] to reduce chart prescribed ordering. This type offers the user many ways to
junk. explore and query the visualization, including free exploration
without predefined navigation paths. Thus, its narrative is
B. Infographics and Interaction reader-driven [18]. Navigation tools for nonlinear infographics
Most infographics published in books, newspapers, include input box, data query, filter, or brushing. [16]
magazines, on TV, or online media provide static The third type called linear-nonlinear is a combination of the
representations. However, an increasing number of infographics other approaches. This type enables the author to communicate
published in online media can be manipulated by the user his message using a predefined path, but additionally it allows
interactively. Interaction is the ability to change in reaction to the user a limit amount of selection, for example using
the user and enhances all types of static information interactive timelines, time controller, or an integrated menu for
visualization [4]. In the context of infographics there exist navigation. [16]
several methods of interaction to manipulate a visual
representation, like scrolling, overview plus detail, focus plus III. RELATED WORK
context, filtering, or data reordering [15].
Interactive infographics shall communicate complex topics
Weber and Wenzel [13] define interactive infographics as a fast, easy, in an easily understandable way to a broad audience.
visual representation of information that integrates several To achieve this goal they have to be user-friendly, i.e., the
modes (at least two) – e.g., image/video, spoken or written text, usability has to be well designed. However, designing and
audio, layout, etc. (the image mode is constitutive) – to a creating interactive infographics is a challenging task [10]: After
coherent ensemble that offers at least one option of control to the identifying and structuring the topic and deriving an appropriate
user. Interactive infographics can be controlled by, e.g., Start or type of representation the multimedia elements – written text,
Stop button, forward or backward button, menu item to select, spoken text (audio), images (photos, diagrams, graphics), videos
timeline or time controller, filter, data request or input box [13] (video, animation) – have to be combined in a meaningful way.
C. Types of Interactive Infographics Interactive elements and hypertext elements have to be defined
and embedded within a concept of navigation [10]. Since online
Following Weber [16] we can classify interactive newspapers are read by a broad audience and not by experts,
infographics according to five distinctive features that cover only a limited knowledge on the linguistic knowledge (e.g.,
interaction as well as narrative issues: degree of interactivity, technical terms), structural knowledge (structure of the
activity model, communicative intent, topic, and the classic information service), application knowledge (e.g., utilization of
questions What, Where, When, How, etc. [17]. Additionally, interactive elements like buttons of sliders), and functional
features like genre or visual narrative can be applied, too [18]. knowledge (e.g., filtering of data) can be assumed [20]. There
Most important for the usability of interactive infographics are are a few “standards” for designing static information graphics
the degree of interactivity and the activity model [11]. (for print and web), but for interactive applications in the web
The degree of interactivity of interactive infographics is no standards exist. Burmester, Wenzel and Tille [10] provide
made up of three levels [13]: Low interactivity, medium some recommendations for designing interactive infographics
interactivity, and high interactivity. While a low level of they derive from a user study on 23 interactive infographics.
interactivity allows a user to manipulate interactive infographics The utilization of interactive infographics has been analyzed
without changing the graphics itself (e.g., zooming, mouseover in some studies. Some authors take a general and global view on
effects for showing details, Next or Start buttons), on a medium interactive infographics [21] and some authors focus on the
level a user can manipulate the graphics (e.g., using a timeline utilization by journalists and publishing houses [9] [11] [22].
slider or menu items) by applying changes and comparing Only a few studies have been published that analyze the
98
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
utilization by readers and focus on usability issues, like Eyetracking would be a method of first choice on evaluating the
Schumacher [23], Burmester, Mast, Wenzel and Tille [24], and usability of an online services including infographics [31] [32].
Zwinger, Langer and Zeiller [25]. Since a study by Zwinger and However, most eyetracking tools cannot cope with the dynamic
Zeiller [11] revealed a significant lack of convenience and behavior of interactive infographics and do not track the changes
usability during the utilization of interactive infographics on the screen, e.g., caused by mouseover, interactive timelines,
published in online newspapers of Austria, Germany and time controllers, and integrated navigation. Thus, a combined
Switzerland, the usability of such infographics will be analyzed method based on a questionnaire and the well-known qualitative
in detail. method Thinking Aloud (Think Aloud) is applied. By combining
both methods the weaknesses in the presentation of infographics
IV. USABILITY OF INTERACTIVE INFOGRAPHICS and problems of the users while interacting with them can be
identified and the course of action can be reconstructed [27].
A. Research Question 1) Thinking Aloud
This study examines the usability of interactive information Thinking Aloud is a cheap, flexible, robust and easy to learn
graphics that have been published in online newspapers. It usability test [33]. Since it was first introduced to interface
focusses on the utilization of interactive infographics by readers design by Lewis [34] in 1982 it became one of the most popular
of those online news. We analyze how readers perceive, methods for usability testing. While testing the usability of an
interpret and interact with interactive infographics. We interactive infographics the test persons are encouraged to
investigate whether background knowledge (structural, continuously comment their actions and their thoughts. Ideally
application, functional knowledge) and previous experience is the test person describes all paths of action taken and all of
required to provide a sufficient user experience. The his/her impressions [35]. The verbalized thoughts of the test
requirements of users related to a user-friendly design (based on persons are recorded (audio and/or video recording). We used
the international standard ISO 9241 on the ergonomics of the recording feature of QuickTime Player to record the spoken
human-system interaction) of interactive infographics for online word of the test persons plus the movement of the mouse on the
news are identified due to an analysis of usage problems and screen and mouse clicks.
identifying weaknesses. Success factors and potential areas of
improvement will be shown. This method provides immediate response that enables the
test moderator to draw conclusions on the actions taken and
Therefore, we focus on the following question: Which emotions already during the evaluation. A few test persons are
success factors improve the usability of interactive information sufficient to derive qualitative feedback of good quality [28].
graphics in online journalism?
2) Questionnaire
B. Usability and User Experience After the test phase where the Thinking Aloud protocol had
been applied the evaluation was continued by a questionnaire the
Usability – in particular web usability – can be defined in test persons had to fill in. A usability evaluation can apply the
various ways. One of the pioneers Jakob Nielsen [26] defines questionnaire ISO 9241/110-S [28]. It follows the ergonomic
usability as “a quality attribute that assesses how easy user principles of the ISO standard 9241-110: suitability for the task,
interfaces are to use”. Consequently, he characterizes usability suitability for learning, suitability for individualization,
by five quality attributes: learnability, efficiency, memorability, conformity with user expectations, self-descriptiveness,
errors, and satisfaction [26]. The international standard ISO controllability, and error tolerance. Each of the seven principles
9241 on the ergonomics of human-system interaction defines in is evaluated by five items on a seven-point Likert scale (“very
part 11 usability as the extent to which a product can be used by negative” to “very positive”). Since this is a standard
specified users to achieve specified goals with effectiveness, questionnaire, some items may be inappropriate for a specific
efficiency and satisfaction in a specified context of use [27]. usability evaluation.
Information systems with high usability have to be user friendly,
easy to use, ease to learn, and its interface has to be To perform the usability test on interactive infographics the
ergonomically designed [28]. ISO 9241/110-S questionnaire was modified and adapted to the
special needs [36] of this specific test. The number of items in
The term “user experience” not only considers the actual use each section (i.e., ergonomic principle) had been adapted:
of an information system, but includes the anticipated usage conformity with user expectations used the five original items;
(before using the system) and the processing of the use situation suitability for the task, self-descriptiveness, and controllability
(identification or distancing; after using the system) [28]. were reduced to four items; suitability for individualization and
According to Norman and Nielson [29] user experience error tolerance were reduced to three items; and suitability for
encompasses “all aspects of the end-user's interaction with the learning was reduced to two items. The Likert scale for
company, its services, and its products”. ISO 9241-210 defines assessing each item was reduced from seven to four levels: ‘very
user experience as "a person's perceptions and responses that negative’ “- -“ | ‘negative’ “-“ | ‘positive’ “+” | ‘very positive’
result from the use or anticipated use of a product, system or “+ +” [36].
service" [30].
To illustrate which items had been included in the adapted
C. Method questionnaire the items for the most significant and meaningful
ergonomic principles will be listed in detail: suitability for the
The user experience and in particular the usability of an task, conformity with user expectations, self-descriptiveness,
interactive system can be measured in various ways [31]. and controllability.
99
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
Example 1: The principle "suitability for the task" consists infographics had been published in German-speaking
of the following items: The interactive infographics … newspapers from 2012 till 2016: two infographics had been
published by “Kurier” from Austria, two infographics by
is complicated and confusing / is straightforward and “Spiegel” from Germany, one had been published by “Berliner
clearly structured. Morgenpost” from Germany, and the sixth infographic had been
is boring and unimaginative / is exciting and creatively published by “20min” from Switzerland. The infographics have
designed (i.e., motivates to interact). been selected according to the activity model (section II.C). To
includes unnecessary elements for control and ensure a balanced usability test two infographics have been
interaction / includes an appropriate number of elements chosen for each type: linear, nonlinear, and linear-nonlinear.
for control and interaction.
offers too much and unnecessary information to keep Although this is a quite small sample, we included examples
me informed / offers exactly fitting and necessary from different kinds of media in all three countries, different
information to keep me informed. topics, all three types of the activity model, and consequently
different levels of complexity. Obviously the small number of
Example 2: The principle "conformity with user infographics tested cannot represent all characteristics of
expectations" is made up of the following items: The interactive possible occurrences. Furthermore, the “quality” of the specific
infographics … implementation (e.g., design, depth of content, adherence to
complicates orientation and use due to a non-uniform usability guidelines) has a great influence on the result –
design of interaction elements / makes orientation and including a risk of introducing a bias to the results. Thus, the
use easy due to a uniform design of interaction results are only partially representative – see also section VI.
elements.
contains text that is hard to read / contains easily TABLE I. SELECTED INTERACTIVE INFOGRAPHICS
readable text.
Type Publisher Title
complicates orientation and use due to bad color-coding Vegan auch bei Fleischessern
/ makes orientation and use easy due to good color- Linear Kurier
beliebt [37]
coding. Linear Spiegel So sank die „Titanic“ [38]
reacts slowly and with unpredictable turnaround times
Berliner
and reaction times / reacts fast and with predictable Nonlinear
Morgenpost
WM 2014 [39]
turnaround times and reaction times. Interaktive Formel-1-Grafik:
includes interactive elements that contradict my Nonlinear Kurier
Fahrer, Strecken, Rekorde [40]
expectations and habits / includes interactive elements Linear-nonlinear 20min Eishockey [41]
that correspond to my expectations and habits. Chronologie der Katastrophe in
Linear-nonlinear Spiegel
Example 3: The principle "self-descriptiveness" is made up Fukushima [42]
of the following items: The interactive infographics …
offers no overview of interactive elements / offers a The usability test has been carried out in January 2017. Eight
good overview of interactive elements. persons at the age of 18 to 30 evaluated the six infographics..
Four test persons had already been using interactive infographics
uses vague and unclear terms and abbreviations / uses
before. The other four test persons did not have previous
terms and abbreviations that are easily understood.
experiences in using interactive infographics. However, all test
uses ambiguous and unclear symbols and icons /
persons were familiar with interactive systems and experienced
symbols and icons that can be easily understood.
users of the World Wide Web and its applications. Four persons
includes unnecessary comments and explanations / were female, four persons were male. [36]
includes helpful comments and explanations.
The test started by presenting the first interactive infographic
Example 4: The principle "controllability" consists of the to the test person. The test person explored the infographic
following items: The interactive infographics … individually, but had to perform a small number of specific
allows for a cumbersome adoption of navigation tools / tasks. According to the Thinking Aloud protocol he/she had to
allows for easy adoption of navigation tools. speak out loud, so the moderator could follow his/her comments
offers difficult actions and changes using buttons / easy and verbalized thoughts (and record them). Immediately after
offers actions and changes using buttons. finishing all tasks the questionnaire had to be filled in by the test
allows to undo single steps in a complicated way / person on his/her own. This cycle was repeated for all six
allows to undo single steps easily. interactive infographics. A test session had an average duration
provides complicated and insufficient sorting, filtering of 70 minutes. [36]
and selection of information / provides simple and In the following subsections some sample results of the
sufficient sorting, filtering and selection of information. usability test for selected infographics are presented. We have
chosen one representative for each type of the activity model:
V. USABILITY STUDY ‘So sank die „Titanic“’ (linear, IV.A), ‘Interaktive Formel-1-
Grafik: Fahrer, Strecken, Rekorde’ (nonlinear, IV.B), Eishockey
In the presented usability test six interactive infographics (linear-nonlinear, IV.C). We will present selected results of the
have been evaluated by eight test persons [36]. These
100
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
questionnaire for the most significant and meaningful
ergonomic principles introduced in section III.C.2: suitability Conformity with User Expectations
for the task, conformity with user expectations, self-
The interactive infographics:
descriptiveness, and controllability. complicates orientation and use due to a … 1 2 5
The interactive infographics:
contains text that is hard to read –… 1 1 3 3
A. Results on Case 1: So sank die “Titanic” The interactive infographics:
2 3 3
The interactive infographic “So sank die „Titanic“” has been complicates orientation and use due to bad …
The interactive infographics:
published by Spiegel Online in 2012 (Fig. 1) [38]. This reacts slowly and with unpredictable… 1 3 1 3
infographic of linear type provides a Forward button and a The interactive infographics:
2 3 3
includes interactive elements that contradict…
Backward button. Users can step forward and backwards step-
by-step among 11 individual images. Some images are 0 2 4 6 8
animated, but the animation cannot be controlled by the user.
-- - + ++
Fig. 3. Results for case 1: Conformity with user expectations
The response on the principle conformity with user
expectations (Fig. 3) show that seven test persons agreed that
this infographics makes orientation and use easy due to a
uniform design of interaction elements. The majority of users (6
out of 8) could read the text elements easily. The other two users
complained about the readability in animated images. Six
persons could navigate easily within the infographic due to good
color-coding. Two persons were irritated by the inconsistent use
of the color red. The turnaround times and reaction times have
been experienced differently. Four persons evaluated them
positively, while the other four persons had been annoyed by
slow reaction times. The majority (again 6 out of 8) persons
indicated that the interactive elements correspond to their
expectations and habits.
Fig. 1. Example linear type: So sank die "Titanic" [38]
Self-descriptiveness
Suitability for the Task
The interactive infographics:
offers no overview of interactive…
1 4 3
The interactive infographics:
2 6 The interactive infographics:
is complicated and confusing –… 3 5
uses hardly intelligible terms…
The interactive infographics: The interactive infographics:
is boring and unimaginative …
1 1 3 3 1 7
uses hardly intelligible symbols …
The interactive infographics:
includes unnecessary elements…
1 1 6 0 2 4 6 8
The interactive infographics:
offers too much and…
2 4 2 -- - + ++
0 2 4 6 8
Fig. 4. Results for case 1: Self-descriptiveness
-- - + ++
The ergonomic principle self-descriptiveness has been
Fig. 2. Results for case 1: Suitability for the task experienced by all test persons in a very positive way (Fig. 4).
Seven persons could easily identify the interactive elements.
The results on the principle suitability for the task (Fig. 2) Terms and abbreviations, as well as symbols and icons could
show that this infographic is reviewed as straightforward and easily be understood and interpreted by all test persons.
clearly structured by all test persons (i.e., rated + or ++). Six
persons had been motivated to interact and experienced its B. Results on Case 2: Interaktive Formel-1-Grafik
design exciting and creative. Furthermore, also six persons The interactive infographic “Interaktive Formel-1-Grafik:
agreed that the number of elements for control and interaction Fahrer, Strecken, Rekorde” has been published by the Austrian
are appropriate. Six persons indicated that this infographics newspaper Kurier on its website in 2016 (Fig. 5) [40]. It is an
offers exactly fitting and necessary information while two infographic of the nonlinear type that enables users to fully
persons disagreed. The response during the Thinking Aloud explore the infographic. Users can select a Formula-1 driver
protocol confirmed these results and provided more detailed with a drop-down menu which provides them with information
qualitative feedback on those issues. on the team, the Formula-1 debut, the results of the last season,
101
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
etc.. Additionally, users can navigate through all Grand-Prix
tracks by clicking on flags. The users can go forward and Conformity with User Expectations
backwards using buttons and reset the infographic.
The interactive infographics:
complicates orientation and use due to a … 1 4 2 1
The interactive infographics:
contains text that is hard to read –… 2 3 3
The interactive infographics:
complicates orientation and use due to bad … 1 3 1 3
The interactive infographics:
reacts slowly and with unpredictable… 2 3 1 2
The interactive infographics:
includes interactive elements that contradict… 1 6 1
0 2 4 6 8
-- - + ++
Fig. 7. Results for case 2: Conformity with user expectations
While evaluating the principle conformity with user
expectations (Fig. 7) we found that five test persons mentioned
that this infographic complicates orientation and use due to a
non-uniform design of interaction elements. Some users simply
could not find out where to click to induce a reaction by the
infographic. The readability of text is good (for six persons).
Opinions diverged on the ease of orientation and ease of use due
to color-coding. While four people were satisfied, four people
were irritated, especially by the usage of the same color for
Fig. 5. Example nonlinear type: Interaktive Formel-1-Grafik: Fahrer,
interactive and non-interactive elements. The majority (five out
Strecken, Rekorde [40] of eight) experienced the turnaround times and reaction times as
to long and unpredictable. A large majority of the test persons
(seven out of eight) was disappointed because several interactive
elements contradicted their expectations and habits.
Suitability for the Task
The interactive infographics:
is complicated and confusing –…
3 2 2 1 Self-descriptiveness
The interactive infographics:
is boring and unimaginative …
2 2 4 The interactive infographics:
offers no overview of interactive…
4 2 1 1
The interactive infographics:
includes unnecessary elements…
7 1 The interactive infographics:
uses hardly intelligible terms…
4 1 3
The interactive infographics:
offers too much and…
1 2 2 3 The interactive infographics:
uses hardly intelligible symbols …
4 2 2
0 2 4 6 8 The interactive infographics:
includes unnecessary comments… 4 0 2 2
-- - + ++
0 2 4 6 8
Fig. 6. Results for case 2: Suitability for the task -- - + ++
The answers in the questionnaire concerning the principle Fig. 8. Results for case 2: Self-descriptiveness
suitability for the task (Fig. 6) show that the majority of test
persons experienced this infographics as being complicated and The results on the ergonomic principle self-descriptiveness
confusing (5 out of 8). This negative experience might be caused (Fig. 8) show that six test persons expressed their opinion that
by the fact that all test persons identified unnecessary elements this infographic does not offer a good overview of interactive
for control and interaction. All users had been confused by the elements. Half of the test persons was satisfied with the terms
fact that additional functions (“maintain”, “exclude”) were and abbreviations being used, but the other half was not. Only
provided when clicking on some interactive elements – with two test persons were satisfied with the use of symbols and
unclear functionality. Half of the users found this infographics icons. The others were dissatisfied with the use of symbols
as boring and unimaginative while the other half found it (especially the usage of flags). Although the infographic
exciting. Three persons mentioned that this infographic does not provides some comments and advices, four test persons felt the
provide the right amount of information. comments and explanations as being not helpful.
102
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
Controllability Suitability for the Task
The interactive infographics: The interactive infographics:
allows for a cumbersome…
4 1 1 2 is complicated and confusing –…
4 4
The interactive infographics: The interactive infographics:
allows to undo single steps in a…
1 2 3 2 is boring and unimaginative …
2 6
The interactive infographics: The interactive infographics:
provides complicated and…
2 2 2 2 includes unnecessary elements…
2 3 3
The interactive infographics:
0 2 4 6 8 1 3 1 3
offers too much and…
-- - + ++ 0 2 4 6 8
-- - + ++
Fig. 9. Results for case 2: Controllability
The answers in the questionnaire regarding the principle Fig. 11. Results for case 3: Suitability for the task
controllability (Fig. 9) show that the majority of test persons
(five out of eight) experienced the adoption of navigation tools Again we start with the results on the ergonomic principle
being cumbersome. The Thinking Aloud protocol revealed that suitability for the task (Fig. 11). The test persons agree that this
some persons had been confused by the fact that detailed infographic is straightforward and clearly structured. The
information on the racing drivers as well as information on the majority found the number of elements for control and
tracks can be retrieved – but they do not influence each other. interaction appropriate. However, two persons mentioned that
Users can undo their actions and use forward and backward the infographic also includes unnecessary elements for control
buttons, but not all of them could find these buttons. and interaction. Although half of the test persons was
overwhelmed by too much information, the other half was quite
C. Results on Case 3: Eishockey satisfied with the amount of information provided.
The third case that will be presented is an interactive
infographic that has been published by the Swiss online news Conformity with User Expectations
platform 20min on ice hockey in 2015 (Fig. 10) [41]. The type
of the activity model is linear-nonlinear which combines the The interactive infographics:
other two approaches. The infographic consists of a start page complicates orientation and use due to a … 8
and graphics describing different issues on ice hockey. The The interactive infographics:
contains text that is hard to read –… 1 7
navigation bar on the top enables users to freely move within the The interactive infographics:
complicates orientation and use due to bad … 1 7
infographic. A linear progress is supported by forward and
The interactive infographics:
backward buttons. Red circle icons provide the users with reacts slowly and with unpredictable… 1 2 5
additional information. The interactive infographics:
1 1 2 4
includes interactive elements that contradict…
0 2 4 6 8
-- - + ++
Fig. 12. Results for case 3: Conformity with user expectations
The feedback concerning the principle conformity with user
expectations (Fig. 12) was quite biased. All test persons were
very satisfied. Only slight criticism could be found on the use of
interactive elements because they contradicted the expectations
and habits of two test persons.
The test persons had also been very satisfied with this
infographic when responding to the questionnaire on the
ergonomic principle self-descriptiveness (Fig. 13). Only one
person did not identify the forward and backward buttons right
from the beginning and rated the item “overview of interactive
elements” negative. Most test persons commented the issues of
Fig. 10. Example linear-nonlinear type: Eishockey [41] self-descriptiveness very positive during the Thinking Aloud
protocol.
103
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
way. However, this type risks to be perceived as being
Self-descriptiveness complicated and confusing. The test persons noticed that both
examples that have been analyzed offer unnecessary control
The interactive infographics: elements for interaction and non-essential information. They
offers no overview of interactive…
1 1 6
needed much more time to get familiar with the interaction
The interactive infographics:
uses hardly intelligible terms…
1 7 elements and control tools. Nevertheless, a uniform design of the
The interactive infographics:
interactive elements can increase the user experience
uses hardly intelligible symbols …
1 7 significantly. Users appreciate that they can move around and
The interactive infographics: navigate within the infographics – as long as the infographics
includes unnecessary comments… 3 5 remains user-friendly.
0 2 4 6 8 Whereas the nonlinear model often requires previous
experience – depending on the usability of the implementation –
-- - + ++
both examples of the linear-nonlinear type have shown that they
can be easily used. The test persons found both examples of this
Fig. 13. Results for case 3: Self-descriptiveness type to be exciting, but at the same time evaluated them as being
straightforward and having a clear design. An easy and fast
adoption is supported by a combination of structured
Controllability information delivery (linear activity) and individual exploration
(nonlinear activity). Additionally, this effect is assisted by good
The interactive infographics: usability, e.g., by uniform design and color-coding that facilitate
allows for a cumbersome…
2 6 orientation.
The interactive infographics:
offers difficult actions and…
8 The most compelling success factors for appealing and
The interactive infographics: usable interactive infographics are a clear and straightforward
allows to undo single steps in a…
8 structure, an appropriate number of elements for control and
0 2 4 6 8
interaction, a uniform design of interaction elements that have to
correspond to the user expectations and habits, good color-
-- - + ++ coding, terms and abbreviations as well as symbols and icons
that can be easily understood, and navigation tools that can be
easily adopted.
Fig. 14. Results for case 3: Controllability
The interactive infographic on ice hockey performed also VI. FUTURE WORK
very well concerning the principle controllability (Fig. 14). It The presented results are based on the analysis of a quite
was very easy for all test persons to adopt to the navigation tools. small sample of six interactive infographics. A continuative
Only two test persons needed more time than the others to adopt study will analyze a much larger number of infographics. Again,
because they did not identify the navigation bar at the top a balanced usability test will have to analyze infographics of the
immediately. Nevertheless, all test persons could perform linear, nonlinear, and linear-nonlinear type (activity model). To
changes and actions using the provided buttons without avoid biased results a widespread selection of infographics on
difficulties, including undoing operations. various topics, published in different media (online newspapers
and online magazines) will be tested to cover different style and
D. Summary of Results quality of the implementation. Thus, there are a number of
Summarizing the results of the usability test of all six criteria that have to be considered while selecting the
infographics (including the other three examples not presented infographics, e.g., type of activity model, type of media, design,
in detail here, but in [36]) reveals that users of infographics of depth of content, publishing date, technology, and topic.
the linear type do not necessarily need previous experience. The Furthermore, a larger number of test persons will be
test persons perceived this type as straightforward and simple. involved: persons of different age, different internet skills and
The linear type provides a step-by-step experience and there are familiarity with interactive designs, and different levels of
no additional, unnecessary elements of interaction which has experience of using (interactive) infographics. Since a test
been confirmed by the test results. Exploring the infographics session should not last much longer than in this test (70 minutes),
corresponds to the expectations and habits of users (i.e., high we will have to assign a number of infographics (e.g., six to a
conformity with user expectations). Both analyzed examples maximum of eight) at random while still ensuring a proper
made orientation easy due to a uniform design and color-coding overall distribution of the criteria mentioned above.
of the interaction elements. However, this type of infographics
has a major problem in user experience: Since this kind of Although the test method using Thinking Aloud and the
interaction is very simple, it risks being boring and uninspired adapted questionnaire based on ISO 9241/110-S was quite
because users would like to have more means of interaction suitable, the continuative study will use eye tracking and/or
another approach that is capable to track interactions (e.g., based
In contrary, nonlinear interactive infographics are found to on time stamps). An additional short questionnaire two weeks
be exiting and creative because they offer a large variety of after the first questionnaire might help to analyze the influence
possibilities to fully explore the infographics in a very individual of the usability and other parameters on the information recall.
104
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
VII. CONCLUSIONS [15] R. Mazza, Introduction to Information Visualization. London: Springer,
2009.
The user experience – notably the usability – of interactive [16] W. Weber, "Typen, Muster und hybride Formen: Ein
information graphics in interactive online media was evaluated Typologisierungsmodell für interaktive Infografiken" In W. Weber, M.
by performing a usability test on six interactive infographics. Burmester, R. Tille (Eds.), Interaktive Infografiken. Berlin, Heidelberg:
Eight test persons evaluated those infographics that had been Springer Vieweg, 2013, pp. 25-37.
published in online newspapers in Austria, Germany and [17] D. Roam, The Back of the Napkin: Solving problems and selling ideas
Switzerland according to the Thinking Aloud protocol and a with pictures. London: Marshall Cavendish, 2008, pp. 138-139.
questionnaire following a modified version of the ISO [18] E. Segel, J. Heer, "Narrative Visualization: Telling Stories with Data" In
IEEE Transactions on Visualization and Computer Graphics, vol. 16, no.
9241/110-S questionnaire. Based on the results of the usability 6, 2010, pp. 1139-1148.
test a number of success factors have been identified. [19] R. Sims, „Interactivity: A Forgotten Art?“ In Computers in Human
Data journalists and designers that use interactive Behavior, vol. 13, no. 2, 1997, pp. 157-180.
infographics for storytelling in online newspapers and [20] H-J. Bucher, "Online-Interaktivität - ein hybrider Begriff für eine hybride
Kommunikationsform. Begriffliche Klärungen und empirische
magazines – daily news or infographics for scrollytelling – may Rezeptionsbefunde" In: C. Bieber, C- Leggewie, Interaktivität. Ein
apply those success factors when designing new, compelling transdisziplinärer Schlüsselbegriff. Frankfurt am Main: Campus Verlag,
infographics. Although there are some fundamental influencing 2004, pp. 132-167.
factors like the degree of interactivity (low, high, medium) and [21] W. Weber, M. Burmester, R. Tille (Eds.), Interaktive Infografiken. Berlin,
the activity model (linear, nonlinear, linear-nonlinear) the Heidelberg: Springer Vieweg, 2013.
specific design and implementation improving the usability will [22] J. George-Palilonis, and M. Spillman, "Storytelling with Interactive
boost the acceptance of the infographics – and the stories being Graphics: An Analysis of Editors' Attitudes and Practices" In Visual
Communication Quarterly, vol. 20, no. 1, 2013, pp. 20-27.
told – among readers enormously.
[23] P. Schumacher, Rezeption als Interaktion: Wahrnehmung und Nutzung
multimodaler Darstellungsformen im Online-Journalismus. Baden-
REFERENCES Baden: Nomos, 2010.
[1] C. Ware, Information Visualization: Perception for Design, 3rd ed. [24] M. Burmester, M. Mast, R. Tille, and W. Weber, "How users perceive and
Waltham: Elsevier, 2012. use interactive information graphics: an exploratory study" In
Proceedings International Conference Information Visualization 2010,
[2] C. Chen, Information Visualization: Beyond the Horizon, 2nd ed.
July 26-29, London, 2010, pp. 361-368.
London: Springer, 2006.
[25] S. Zwinger, J. Langer, M. Zeiller, "Acceptance and Usability of
[3] N. Yau, Visualize This: The FlowingData Guide to Design, Visualization,
Interactive Infographics in Online Newspapers" In Proceedings
and Statistics. Weinheim: Wiley Publishing, 2011.
International Conference Information Visualisation 2017, July 11-14,
[4] R. Spence, Information Visualization: An Introduction, 3rd ed. London, 2017, pp. 176-181.
Heidelberg: Springer, 2014.
[26] J. Nielsen, Usability 101: Introduction to Usability. Nielsen Norman
[5] B. Shneiderman, "The Eyes Have It: a task by data type taxonomy for Group, 2012. https://www.nngroup.com/articles/usability-101-
information visualization" Proceedings of the 1996 IEEE Symposium on introduction-to-usability/
Visual Languages, 1996, pp.336-343.
[27] International Organization for Standardization, Ergonomics of human-
[6] J. Gray, L. Bounegru, and L. Chambers (Eds.), Data Journalism system interaction – Part 11: Usability: Definitions and concepts.
Handbook. European Journalism Centre and the Open Knowledge ISO/DIS 9241-11:2015.
Foundation, 2012. http://datajournalismhandbook.org/
[28] F. Sarodnik, and H. Brau, Methoden der Usability Evaluation:
[7] L. Matzat, Datenjournalismus: Methode einer digitalen Welt. Konstanz, Wissenschaftliche Grundlagen und praktische Anwendungen, 2nd ed.
München, UVK Verlagsgesellschaft, 2016. Bern: Hans Huber, 2011.
[8] M. Lorenz, "Status and outlook for data-driven journalism" In Data- [29] D. Norman, J. Nielsen, The Definition of User Experience (UX). Nielsen
driven journalism: What is there to learn? Amsterdam, European Norman Group, n.d. https://www.nngroup.com/articles/definition-user-
Journalism Centre, 2010, pp. 8-17. experience/
[9] W. Weber, H. Rall, "“We Are Journalists.” Production Practices, [30] International Organization for Standardization, Ergonomics of human
Attitudes and a Case Study of the New York Times Newsroom" In W. system interaction - Part 210: Human-centered design for interactive
Weber, M. Burmester, R. Tille (Eds.), Interaktive Infografiken. Berlin, systems. ISO/DIS 9241-210:2009, 2009.
Heidelberg: Springer Vieweg, 2013, pp. 161-172.
[31] T. Tullis, B. Albert, Measuring the User Experience: Collecting,
[10] M. Burmester, A. Wenzel, and R. Tille, "Navigation – Interaktion – Analyzing, and Presenting Usability Metrics, 2nd ed. Waltham, MA:
Informationspräsentation. Rezeptions- und Nutzungsprobleme bei Morgan Kaufmann, 2013.
interaktiven Infografiken" In W. Weber, M. Burmester, R. Tille (Eds.),
[32] J. Nielsen, and K. Pernice, Eyetracking Web Usability. Berkeley, CA:
Interaktive Infografiken. Berlin, Heidelberg: Springer Vieweg, 2013, pp.
New Riders, 2010.
105-134.
[33] J. Nielsen, Thinking Aloud: The #1 Usability Tool. Nielsen Norman
[11] S. Zwinger, and M. Zeiller, "Interactive Infographics in German Online
Group, 2012. https://www.nngroup.com/articles/thinking-aloud-the-1-
Newspapers", In W. Aigner, G. Schmiedl, K. Blumenstein, M.
usability-tool/
Zeppelzauer, and M. Iber (Eds.), Forum Media Technology 2016, CEUR
Workshop Proceedings, vol. 1734, pp. 54-64. urn:nbn:de:0074-1734-7. [34] C.H. Lewis, Using the "Thinking Aloud" Method In Cognitive Interface
Design (Technical report). IBM. RC-9265, 1982.
[12] A. Cairo, The functional art: an introduction to information graphics and
visualization. Berkeley, CA: New Riders, 2013. [35] T. Stapelkamp, Informationsvisualisierung: Web-Print-Signaletik. Berlin:
Springer Vieweg, 2013.
[13] W. Weber, and A. Wenzel, "Interaktive Infografiken:
Standortbestimmung und Definition" In W. Weber, M. Burmester, R. [36] J. Langer, Analyse der Usability von interaktiven Infografiken im
Tille (Eds.), Interaktive Infografiken. Berlin, Heidelberg: Springer deutschsprachigen Online-Journalismus. Bachelor thesis, Eisenstadt:
Vieweg, 2013, pp. 3-23. University of Applied Sciences Burgenland, 2017.
[14] E. Tufte, The Visual Display of Quantitative Information, 2nd ed. [37] Kurier. Vegan auch bei Fleischessern beliebt, 2015. Retrieved December
Cheshire, CT: Graphics Press, 2001. 15, 2016 from https://kurier.at/genuss/vegan-auch-beifleischessern-
beliebt (offline).
105
Evaluation of the User Experience of Interactive Infographics in Online Newspapers
[38] Spiegel. So sank die Titanic, 2012. Retrieved September 15, 2017 from https://kurier.at/sport/motorsport/interaktive-formel-1-grafik-fahrer-
http://www.spiegel.de/flash/flash-28319.html. streckenrekorde-zum-durchklicken/187.685.483.
[39] Berliner Morgenpost. WM 2014, 2014. Retrieved September 15, 2017 [41] 20min. Eishockey, 2015. Retrieved September 15, 2017 from
from http://interaktiv.morgenpost.de/wmspieler. Checked September 15, http://www.20min.ch/interaktiv/vizualne/2015_03_Eishockey/index_CH
2017. E.html.
[40] Kurier. Interaktive Formel-1-Grafik: Fahrer, Strecken, Rekorde, 2016. [42] Spiegel. Chronologie der Katastrophe in Fukushima I (Daiichi), n.d..
Retrieved September 15, 2017 from Retrieved September 15, 2017 from http://www.spiegel.de/flash/flash-
25484.html.
106