<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>November</journal-title>
      </journal-title-group>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Vertical Context of Geographic Locations: An Empirical Comparison of Three Visualization Approaches</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Prasad Madushanka</string-name>
          <email>prasad.dream13@gmail.com</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Auriol Degbelo</string-name>
          <email>auriol.degbelo@tu-dresden.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Chair of Geoinformatics</institution>
          ,
          <addr-line>TU Dresden</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute for Geoinformatics, University of Münster</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>1</volume>
      <fpage>1</fpage>
      <lpage>15</lpage>
      <abstract>
        <p>The vertical context of a geographic location encompasses all known information about that location. Though linked data is suitable for representing the vertical context of geographic locations, there is still a need for means to help users explore this vertical context visually and guidelines for designers of vertical context visualizations. To address this gap, this article compared three visualization approaches: map + table at the location of interest, map + markers at the location of interest and map + circular treemap at the location of interest. The three approaches were tested using two datasets: DBpedia (vertical context of places), and Umweltbundesamt data (vertical context of environmental data). While the approaches were comparable in terms of eficiency and efectiveness for most tasks, the map + circular treemap approach received higher ratings from participants (N=18) for enjoyment, usefulness, and satisfaction. The findings from this empirical study are an initial step towards developing guidelines for visualizing vertical context information extracted from geolinked data and beyond. linked data visualization, geolinked data, geovisualization, vertical context of geographic location Geographic locations are more than points on a map; they are only one dimension of the more complex notion of place [1], and often act as the connecting link between many attributes. Following [2], these attributes can be divided into two groups: those belonging to the horizontal context and those belonging to the vertical context. The horizontal context refers to the context established by information about surrounding locations. This could involve attributes such as the physical proximity to other landmarks, the accessibility to transportation networks, or the cultural and economic ties with neighbouring regions. By contrast, the vertical context pertains to the context established by all things that are known about a location. It encapsulates various attributes such as topography, climate, land use, historical significance, etc. and any other feature that adds depth to the understanding of a location. The topic of this article is the visualization of the vertical context of a geographic location. While linked data is suitable to represent VOILA 2024: The 9th International Workshop on the Visualization and Interaction for Ontologies, Linked Data and Knowledge Graphs co-located with the 23rd International Semantic Web Conference (ISWC 2024), Baltimore, USA, ∗Corresponding author.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>CEUR
ceur-ws.org</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>
        contextual information in general [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and hence, the vertical context of geographic locations in
particular, there are currently few means to help users explore this vertical context visually,
and limited empirically derived guidelines for designers of vertical context visualizations. Since
the spatial dimension is important in organizing knowledge (see e.g. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]), investigating means
to visualize the vertical context is a prerequisite to getting a holistic picture of what happens at
a location. Hence, research on visualizing the vertical context of geographic location is relevant
to work on linked geographic data [
        <xref ref-type="bibr" rid="ref5 ref6 ref7">5, 6, 7</xref>
        ], linked science [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], open (geo)data reuse [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and
spatial data infrastructures [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], to name a few. Table 1 shows a few examples of applications
displaying all attribute information about a place, along with their strategy. While the study
presented next considers visualization approaches to answer basic questions as a first step, the
work’s long-term goal is to build visualization techniques that help users “digest” all information
available about a location, to the end of formulating spatial hypotheses about places.
      </p>
      <p>This article presents an empirical study addressing the question: ‘How to efectively visualize
the vertical context of geographic locations?’. The article compared three approaches: map
+ table at the location of interest (hereafter MT), map + markers at the location of interest
(hereafter MM) and map + circular treemap at the location of interest (hereafter MCT). The
map is present in the three approaches because it is the most suitable medium to communicate
spatial knowledge [15]. Besides, in all three cases, the content is grouped according to thematic
categories (e.g. political, weather, administrative) that can be explored. The exploration of the
attribute values within the thematic categories happens through diferent interaction techniques
respectively: scrolling (MT), panning (MM) and zooming (MCT). The three approaches were
tested using two datasets: DBpedia [16] (vertical context of places), and Umweltbundesamt data
(vertical context of environmental data). The Umweltbundesamt is the environment agency of
the German government. DBpedia (knowledge graph data) was accessed through the DBpedia
Live Sync API1, while the Umweltbundesamt data (structured data as JSON) was accessed
through the ‘Air Data API (UBA)’2. The contribution of this work is an empirical investigation
informing about the respective merits of the three approaches.
1https://www.dbpedia.org/resources/live/dbpedia-live-sync/ (accessed: July 10, 2024).
2https://www.umweltbundesamt.de/daten/luft/luftdaten/doc (accessed: July 10, 2024).</p>
    </sec>
    <sec id="sec-3">
      <title>2. Background</title>
      <p>As discussed in [17], there are at least 16 linked data visualization use cases. Visualizing the
vertical context can be seen as a case of visualizing the information related to a specific instance
[17], when the specific instance is a location of interest. The necessity of communicating the
spatial context of the data instances is an additional requirement for tools visualizing the vertical
context of geographic locations, next to the visualization of all attributes available for that
location (the attributes may be provided as linked data or not). Ideally, these tools should also
support the follow-your-nose principle, which adds another dimension of complexity. As a first
step, all attributes retrieved for a location were treated as RDF-literals in this work, i.e. available
links were removed for simplicity.</p>
      <p>
        The existing literature ofers several reviews about tools and approaches to visualize linked
data (e.g. [18, 19, 20, 21]), but ‘vertical context exploration’ as a use case is mostly absent
from these reviews. Tools visualizing geo-linked data exist, but they often have a diferent
focus than user interfaces’ impact on the exploration of all things known about a location. For
instance, Mai et al. [22] proposed a multi-view interface (i.e. table view + graph view + map
view) to enable the exploration of scientific geographic data sources. Here, the focus was on the
discovery of detailed information about an entity, relationships between entities and between
entity types, and the spatial distribution of entities. In [23], diferent configurations can be
loaded (and a plugin that interprets the point geometry for locations), to display information
about DBpedia cities in the Phuzzy.link browser. Attribute information is displayed as a table
(similar to DBpedia Page), and hence the Phuzzy.link interface would qualify as a vertical context
visualization. Nonetheless, vertical context exploration was not the focus of the work, but
rather the concept of an adaptable interface to explore SPARQL endpoints from the browser.
At last, Potnis and Durbha [
        <xref ref-type="bibr" rid="ref11">24</xref>
        ] illustrate how to show country information on a 3D globe, but
their focus was on the display of geographic relationships (e.g. “hasBorderWith”) between
geographical entities.
      </p>
      <p>The study presented next is a user-based evaluation of three strategies to retrieve information
about the vertical context of a location (MT, MM and MCT). We consider three
informationseeking questions: elementary-level questions (questions about specific values of a property
at a given location), intermediate-level questions (questions about a specific category and its
subelements at the location of interest) and global-level questions (questions about all categories
of topics at the location of interest).</p>
    </sec>
    <sec id="sec-4">
      <title>3. Method</title>
      <p>We compared the merits of the three approaches through a controlled experiment. The following
variables were considered during the experiment:
• Independent variables: the three visualization approaches (map+table, map+markers,
map+circular treemap), the type of questions asked (specific attribute values, overview
[count of attributes per category], overview [attribute categories]), and the six attribute
levels (60, 90, 105, 140, 180, 200).
• Dependent variables: eficiency (task completion time), efectiveness (accuracy of tasks),
perceived enjoyment, perceived usefulness, perceived satisfaction and perceived ease of
use regarding the visualization approaches.
• Controlled variables: number of screens used (N=1) and screen size (all between 13 inches
and 27 inches).
• Subject variables: age, gender, educational background, English proficiency, computer
literacy and prior experience with visualization tools (i.e. web maps, Leaflet.js markers
and D3.js zoomable circle packing).</p>
      <p>We anticipated before the study that the magnitude of attributes displayed might influence
the performance of the techniques assessed and introduced the notion of ‘attribute level’ to
capture this potential efect. ‘Attribute level X’ stands for ‘number of attribute values in the
order of X’ or ‘number of attribute values roughly equal to X’. The initial plan was to start
with attribute level 60 and increase it incrementally by 30 attributes for each subsequent level.
However, due to attributes actually ofered by the datasets, we adjusted the final attribute levels
slightly, i.e. 60, 90, 105 (instead of 120), 140 (instead of 150), 180, and 200 (instead of 210).</p>
      <sec id="sec-4-1">
        <title>3.1. Datasets</title>
        <p>Two datasets were used: DBpedia and the Umweltbundesamt data (hereafter UBA). We used
each to collect information about the vertical context of 10 cities in Germany (Table 2). While
both provide information about the vertical context of places, a diference between the two is
that DBpedia provides mostly information in the form of text, while Umweltbundesamt ofers
primarily numerical information, e.g., for attributes such as Fine dust, Carbon monoxide (CO),
Sulphur dioxide (SO2), Ozone (O3) and Nitrogen dioxide (NO2). The DBpedia Live Sync API was
used by inputting the city names as query parameters and the data was obtained in N-triples
format. Relevant data from Umweltbundesamt was obtained in JSON format from the “Air Data
API (UBA) 3.0.0”. Raw data obtained from the two APIs underwent cleaning and post-processing
before rendering through the visualization approaches.</p>
      </sec>
      <sec id="sec-4-2">
        <title>3.2. Prototype</title>
        <p>
          The application was built using Web technologies (HTML, CSS, JavaScript), a web mapping
library (Leaflet) and a visualization library (D3.js). We used Leaflet (version 1.9.4) for the base
maps and location markers, D3.js (Version 4) for the circular treemap, and Bootstrap (version
5.3.2) as the CSS framework. The application features search for locations, the choice of datasets,
the choice of a visualisation approach, as well as short answers to frequently asked questions.
The approaches were renamed into “Visualization Approach 1”, “Visualization Approach 2”
and “Visualization Approach 3” during the experiment to avoid potential participant bias. A
demo of the application is available at https://youtu.be/zLCy3shSOoU. Figures 1, 2 and 3 show
screenshots of the three visualization approaches on the two datasets respectively.
• Map+table: The information is presented in a table format, grouped by the main attribute
category (Figure 1a). This arrangement enables users to eficiently search for specific
data by scrolling through the table. Categorizing the data facilitates swift navigation
City
Berlin
Hamburg
Munich
Cologne
Frankfurt am Main
Stuttgart
Düsseldorf
Leipzig
Dortmund
Essen
to sections of interest, enhancing the overall eficiency of data retrieval. Additionally,
attribute indexing improves the ease of counting the attributes relevant to each group or
attribute within the dataset.
• Map+markers: Instead of a traditional table format, the data is represented by leaflet
markers arranged in a spiral pattern, i.e. the makers are ‘spiderfied’ [
          <xref ref-type="bibr" rid="ref12">25</xref>
          ]. The spiderify
method, described in [
          <xref ref-type="bibr" rid="ref12">25</xref>
          ], was originally suggested to tackle the problem of visualizing
multiple markers at the same location. Thus, we reused it in this context to convey the
idea that all attributes are attached to the same location. The spiderify method takes the
markers placed in the same position and arranges them in a spiral. The advantage of
the spiral pattern is its compactness, which enables the display of numerous attributes.
Each marker represents diferent attributes, making it possible for users to identify and
diferentiate between them. When users click on a marker, a pop-up message appears,
showing the name of the attribute category and its corresponding values. Colour hue is
used to highlight markers belonging to the same category (Figure 2a).
• Map+circular treemap: The visualization organizes information into circles representing
distinct attribute categories for a city (Figure 3a). Upon a user’s click on a category, the
visualization zooms in to display the attribute-value pair information available in that
specific circle (Figure 3b). We opted for a circular treemap over a traditional treemap
because, for the two hierarchy levels required to display the datasets in this work, the
circular treemap reveals hierarchical structures and facilitates interaction between the
levels more efectively.
3.3. Tasks
We generated six tasks for the experiment. A task has three dimensions: a city, an attribute
level for the vertical context, and a dataset. The six tasks were: Dortmund-60-DBpedia (T1),
Düsseldorf-105-DBpedia (T2), Essen-90-DBpedia (T3), Hamburg-200-UBA (T4), Berlin-180-UBA
(a) DBpedia Data
(b) UBA Data
(T5), and Munich-140-UBA (T6). Each task features three types of questions: q1-q3 all deal with
retrieving specific attribute values, q4 is about counting the number of attributes belonging
to a category, and q5 is about listing all thematic categories of attributes available for a city.
Adapting Bertin [
          <xref ref-type="bibr" rid="ref13">26</xref>
          ]’s distinction to the current context, we can distinguish questions about a
single element (elementary level of reading), questions about a group of elements (intermediate
level of reading), and questions about the whole set of visual elements (overall or global level
of reading). q1-q3 correspond to the elementary level of reading, q4 touches the intermediate
level of reading, and q5 addresses the global level of reading of the dataset at hand. The order
of the questions (q1-q5) remained identical for all participants during the study. We provide
two examples of tasks below (T1, T4). A description of the remaining tasks (T2, T3, T5, T6) can
be found in the supplementary material, Section 7.
        </p>
        <p>Task (T1) is described as follows (Dortmund, 60 attributes, DBpedia).</p>
        <p>specific attribute values
• q1: What is the lowest temperature in Dortmund for September?
• q2: What is the recorded highest temperature in Dortmund for May?
• q3: Who is the leader of Dortmund (Leader Name)?</p>
        <p>overview (count of attributes per category)
• q4: How many attributes belong to the ‘Weather’ category?</p>
        <p>overview (attribute categories)
• q5: What are the attribute categories for Dortmund that you can access through this
approach?</p>
        <sec id="sec-4-2-1">
          <title>Task (T4) is described as follows (Hamburg, 200 attributes, UBA).</title>
          <p>specific attribute values
• q1: What is the monthly maximum (μg/m³) of “Ozone (O3)” recorded in the “Hamburg</p>
          <p>Sternschanze” station for April?
• q2: What is the monthly maximum (μg/m³) of “Nitrogen dioxide (NO2)” recorded in the
“Hamburg Max-Brauer-Allee II (Straße)” station for October?.
• q3: What is the monthly average (μg/m³) of “Fine dust (PM10)” recorded in the “Hamburg
Habichtstrasse” station for December?
overview (count of attributes per category)
• q4: How many data records (vertical attributes) are available for the air pollutant “Fine
dust (PM10)”?</p>
          <p>overview (attribute categories)
• q5: What are the categories of air pollutants that exist in Hamburg city?</p>
          <p>The experiment followed a within-group design. The order of the visualization approaches
and of the tasks was counterbalanced. The randomization and distribution of the subjects across
conditions helped collect 18 data points per visualization approach and 9 data points per attribute
level. Additional details about the randomization strategy are shown in the supplementary
material, Section 7. Figure 4 shows all experiment steps, including the tasks.</p>
        </sec>
      </sec>
      <sec id="sec-4-3">
        <title>3.4. Procedure</title>
        <p>
          The experiment was held online (Google Meet) and was conducted each time with one
participant and one examiner (the first author). The procedure started with a brief explanation of the
experiment’s goals. Then the participants were asked to provide a video consent before
proceeding with the experimental tasks. Afterwards, participants filled in a background questionnaire
about personal details, computer literacy, familiarity with web maps, Leaflet.js marker patterns,
and the D3.js zoomable circle-packing visualization approach. Once the background
questionnaire was completed, the participants performed three tasks using a diferent visualization
approach each time to find information about one of the ten cities (Figure 4). The examiner only
observed the entire process. Upon completion of each task, participants were asked to answer
one question related to the enjoyment [
          <xref ref-type="bibr" rid="ref14">27</xref>
          ] of the visualization approach and three questions
selected from the USE questionnaire [
          <xref ref-type="bibr" rid="ref15 ref16">28, 29</xref>
          ] to measure usefulness, satisfaction and ease of
use on a 7-point Likert scale. The three questions selected were: “X makes the things I want to
accomplish easier to get done” (Question UU5, Usefulness); “I don’t notice any inconsistencies
as I use X” (Question UE8, Ease of Use); and “I am satisfied with X” (Question US1, Satisfaction),
where X refers to the visualization approach, to find out the extent to which X helps users
answer the questions considered (UU5), smoothly navigate the attribute values (UE8) and brings
about satisfaction in the process (US1). At the end of the session, the participants were asked to
answer three questions: ‘Considering the three visualization approaches you’ve interacted with,
could you please rank them in order of preference based on which one you found most efective
in helping you answer the questions?’, ‘Could you please provide reasons for ranking them?’
And ‘Do you have any suggestions regarding further improvements for the Web Application?’.
All answers to questions were recorded through the LimeSurvey platform, which was also
used to record the time participants took to answer specific questions . The experiment was
pilot-tested with two participants and approved by the institutional ethics board.
        </p>
      </sec>
      <sec id="sec-4-4">
        <title>3.5. Participants</title>
        <p>The study involved 18 participants (6F, 12M), which were recruited through word-of-mouth.
Participation was voluntary and the participants were not compensated for their participation.
Their age distribution varied: (12/18) participants fell within the age range of 21 to 30 years,
while (4/18) participants were between 31 and 40. Additionally, there was (1/18) participant in
the age groups of 41-50 and 51-60 respectively. Regarding computer literacy, the participants
reported varying levels of proficiency, with (2/18) participants self-identifying as beginners,
(7/18) as intermediate, and (9/18) as advanced users. The familiarity with Leaflet.js markers
and D3 zoomable circle packing also exhibited a range of expertise among the participants.
Specifically, (1/18) participants were highly familiar with Leaflet.js marker patterns, while (8/18)
participants reported moderate familiarity and (9/18) participants indicated no familiarity. The
familiarity distribution was similar for D3.js zoomable circle packing, indicating a diverse range
of expertise levels within the participant pool. All participants used their own laptops with
one screen to complete the study and confirmed that their screen size was between 13 and 27
inches, to keep the experimental setup somewhat similar.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>4. Results</title>
      <p>We now present the experiment’s results, starting with the outcomes of the quantitative analysis,
before proceeding with the qualitative feedback.</p>
      <sec id="sec-5-1">
        <title>4.1. Results of the pairwise comparison of the three visualization approaches</title>
        <p>• Eficiency [elementary-level]: while the data did not suggest any significant diference
cross-attribute-levels, the Map+Table approach recommends itself for the attribute levels
90 and 105, while the Map+Markers approach seems best for the attribute levels 180 and
200 (see also Table 4).
• Eficiency [intermediate-level]: the three approaches are comparable overall, but
Map+Table may be preferable at attribute level 105, while Map+Makers or Map+Circular Treemap
may be advantageous at attribute level 60 (see also Table 5).
• Eficiency [global-level]: Map+Circular Treemap is preferable at attribute level 60,
Map+Table at attribute level 105, and Map+Makers may be a good option for attribute
levels 105 and 180 (see also Table 6).</p>
        <p>• Efectiveness [elementary-level]: the approaches may be deemed comparable overall, but
Map+Markers provided higher accuracy values at attribute level 60, while Map+Circular
Treemap increased users’ accuracy during question answering at attribute level 200 (see
also Table 7).
• Efectiveness [intermediate-level]: the Map+Circular Treemap approach recommends
itself here overall, and at attribute levels 105 and 180. The Map+Markers approach seems
best at attribute level 90 (see also Table 8).
• Efectiveness [global-level]: the Map+Circular Treemap approach seems best at levels
60, 140 and 180, and the Map+Markers approach helped users increase their answers’
accuracy at level 90 (see also Table 9).
• Perceived enjoyment: there seems to be a clear user preference for the Map+Circular</p>
        <p>Treemap approach (see also Table 10).
• Perceived usefulness: there seems to be a clear user preference for the Map+Circular</p>
        <p>Treemap approach both overall and across all attribute levels (see also Table 11).
• Perceived satisfaction: here also, the Map+Circular Treemap approach seems to have
provided more satisfaction to participants during the question-answering tasks (see also
Table 12).
• Perceived ease of use: overall, the easiest approaches to use according to the participants
were either the Map+Circular Treemap or the Map+Table approach. The Map+Makers
approach was perceived as slightly easier to use by participants at attribute level 60 only
(see also Table 13).</p>
      </sec>
      <sec id="sec-5-2">
        <title>4.2. Participants’ subjective preference</title>
        <p>Here, the Map+Circular Treemap approach emerged as the most favoured by participants,
garnering 15 out of 18 votes as Rank 1 (Table 3). In contrast, the Map+Markers approach
received the least preference with 16 out of 18 votes as Rank 3. As for the reasons for their
ranking provided by the users, the key advantages mentioned were:
• Map+Circular Treemap: Users can zoom in for a detailed inspection of specific attributes
and easily zoom out to transition to other attributes. The attributes are organized into
clusters, grouping similar ones in close proximity. This systematic arrangement enhances
navigation, making it easy to locate specific attributes eficiently.
• Map+Markers: the participants did not mention any specific advantage.
• Map+Table: Users can navigate through attributes by employing a rapid scrolling feature,
allowing them to eficiently move through the content and explore diferent attributes
without delays.</p>
        <sec id="sec-5-2-1">
          <title>The key disadvantages mentioned were:</title>
          <p>• Map+Circular Treemap: the participants did not mention any specific disadvantage.
• Map+Markers: To locate a specific attribute, users need to click on multiple markers, as it
can be challenging to distinguish the desired attribute from others in the same category.
• Map+Table: The list view arrangement makes it dificult to identify attribute counts and
discern the attribute categories easily.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>5. Discussion</title>
      <sec id="sec-6-1">
        <title>5.1. Implications</title>
        <p>
          The display of attribute values as a table is currently the most common - and straightforward
way of presenting vertical context information (e.g. DBpedia Page, Table 1). The preliminary
observations from this study suggest that there are usually better alternatives, both in terms
of maximizing utilitarian (e.g. eficiency, efectiveness) and hedonic objectives (e.g. perceived
user satisfaction). Another consistent observation of the study is that both attribute level and
question types matter when comparing the merits of the diferent visualisation approaches.
Since this study is a first of this kind, it is still unclear at this point whether the variability
observed can be truly attributed to the performance of an approach at an attribute level, or to
the current (limited) sample of participants. Still, the study has highlighted that attribute level
may be a confounding variable for similar experiments in the future, and this suggests that this
dimension should be explicitly controlled for. Moving forward, there are two possible ways of
reusing the results from Figure 5: reading a column to find out the most suitable visualization
at an attribute level or reading a line to find the best suitable row to maximize the outcomes
for a dependent variable (e.g. efectiveness, perceived ease of use). These two ways of reading
can be used by visualization designers in the future to formulate hypotheses, as they use any
of the three visualization approaches considered in the current work as a baseline. Finally, as
mentioned in [
          <xref ref-type="bibr" rid="ref17">30</xref>
          ], spatial data collection, processing and sharing is a common thread of various
disciplines within the Earth System Sciences. Hence, the observations made here about vertical
context visualization are relevant to ongoing eforts to establish (national) infrastructures for
the Earth System Sciences.
        </p>
      </sec>
      <sec id="sec-6-2">
        <title>5.2. Limitations</title>
        <p>The prototype was designed exclusively for desktop-size screens (e.g. personal computers) and
is not yet optimized for use on mobile devices such as phones and tablets. Hence, no claim
can be made as to the generalizability of the results to these devices. Another limitation of the
study was the relatively small number of participants and their homogeneous backgrounds.
This was necessary because of the exploratory nature of the study, but a larger-scale study
would be needed, with a more diversified user base, in follow-up work to learn about the
applicability of the results to diferent user groups. At last, all three techniques overlay the
vertical context information on top of the location selected. While this has the advantage that - in
the context of the navigation - the location for which the vertical context is currently visualized
is unambiguous, this comes at the cost of the occlusion of the map. An alternative worth
considering would be the juxtaposition of the vertical context display and the map (i.e. placing
the vertical context display alongside the map). This would come at the extra cost of designing
efective location emphasis techniques to highlight the current location unambiguously but is
worth further investigation in future work.</p>
      </sec>
      <sec id="sec-6-3">
        <title>5.3. Future work</title>
        <p>
          We have mentioned three key requirements of visualizations of the vertical context of geographic
locations in Section 2: communicate the spatial context (R1), enable intuitive and efective
navigation through the wealth of attributes and their values (R2), and enable the traversal of
paths between diferent datasets, and paths between properties of the same dataset (R3, a.k.a.
follow-your-nose principle). The three techniques in this work addressed R1 (through the
map) and R2 (through either the table, the marker, or the circular treemap). Hence, an open
question is how to address R3 through existing or new visualization approaches. Given the
number of attributes to display, the categorization of content comes in handy. Hence, this work
tested circular treemaps, which are only one way of visualizing hierarchical data, to display
the attribute values. It is an open question whether or not alternative ways of visualizing
hierarchical data (for examples, see [
          <xref ref-type="bibr" rid="ref18 ref19">31, 32</xref>
          ]) would be equally efective in this context, if not
more efective. Furthermore, the three approaches have in common that attribute information
is displayed in a popup after clicking on the map. Given the necessity for at least two views
for the display of vertical context information, geodashboards (i.e. multiple-view systems of
geographic data, arranged on a single screen so that the information can be perceived at a
glance [
          <xref ref-type="bibr" rid="ref20">33</xref>
          ]) may be considered to fulfil the three requirements as well. Besides, Figure 5 has
shown that attribute levels matter, hence scalability across several attribute levels needs more
scrutiny. In addition, the work only tested the visualization approaches for up to 200 attribute
values. The extent to which the observations hold for even greater levels of attribute values
also needs to be investigated in future work. At last, we mentioned in Section 1 that, beyond
simple question-answering about attribute values at a location, one higher-level goal of vertical
context exploration is the formulation of spatial hypotheses. The extent to which existing and
novel visualization techniques support that goal would need systematic testing in future work.
        </p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>6. Conclusion</title>
      <p>There are two types of contextual information related to a geographical location: the vertical
context (all things known about a location) and the horizontal context (all things known about
surrounding locations to a location). While linked data is suitable to represent contextual
information in general and the vertical context of geographic locations in particular, we still
lack means to help users explore this vertical context visually, and guidelines for designers
of vertical context visualizations. This work has provided an empirical comparison of three
approaches (map+table, map+marker, map+circular treemap) to address this gap. The merits
of the approaches were compared for question-answering tasks involving 60 to 200 attribute
values. The map+circular treemap approach yielded the best outcomes in most cases, followed
by the map+markers approach, and both can be used as a starting point in further investigations
of visualizations that help users explore all that is known about a place.</p>
    </sec>
    <sec id="sec-8">
      <title>7. Supplementary material</title>
      <p>The supplementary material showing the randomization approach during the experiment, all
tasks completed by the participants, as well as the data collected during the experiment, is
available at https://doi.org/10.6084/m9.figshare.26264594. The code of the prototype is available
on GitHub (https://github.com/Prasadmadhusanka/VerticalGeoVis-prototype).</p>
    </sec>
    <sec id="sec-9">
      <title>Acknowledgments</title>
      <p>The work has been partly funded by the European Commission through the Erasmus
Mundus Master in Geospatial Technologies (Erasmus+/Erasmus Mundus program, project
no. 101049796, http://mastergeotech.info/) and the German Research Foundation through the
project NFDI4Earth (DFG project no. 460036893, https://www.nfdi4earth.de/) within the German
National Research Data Infrastructure (NFDI, https://www.nfdi.de/).
[11] DBpedia, About:Baltimore, 2024. URL: https://dbpedia.org/page/Baltimore, accessed: July
10, 2024.
[12] Alfred Wegener Institute, GIS Viewer - Arctic Coastal Dynamics, 2024. URL: https://maps.</p>
      <p>awi.de/awimaps/projects/public/?cu=arctic_coastal_dynamics, accessed: July 10, 2024.
[13] Leibniz-Institut für ökologische Raumentwicklung (IÖR), Monitor der Siedlungs
und Freiraumentwicklung, 2024. URL: https://monitor.ioer.de/?ind=N01EG&amp;time=
2023&amp;baselayer=topplus&amp;opacity=0.8&amp;raeumliche_gliederung=gebiete&amp;zoom=8&amp;lat=
50.93073802371819&amp;lng=9.755859375000002&amp;glaettung=0&amp;raumgl=bld&amp;klassenanzahl=
7&amp;klassifizierung=haeufigkeit&amp;darstellung=auto&amp;ags_array=&amp;, accessed: July 10, 2024.
[14] TERENO, TERENO - Data Discovery Portal, 2024. URL: https://ddp.tereno.net/ddp/
dispatch?searchparams=freetext-Free%20Text%20Search, accessed: July 10, 2024.
[15] A. Degbelo, J. Wissing, T. Kauppinen, A comparison of geovisualizations and data tables
for transparency enablement in the open government data landscape, International Journal
of Electronic Government Research 14 (2018) 39–64. doi:10.4018/IJEGR.2018100104.
[16] J. Lehmann, R. Isele, M. Jakob, A. Jentzsch, D. Kontokostas, P. N. Mendes, S. Hellmann,
M. Morsey, P. Van Kleef, S. Auer, C. Bizer, DBpedia – A large-scale, multilingual knowledge
base extracted from Wikipedia, Semantic Web 6 (2015) 167–195. doi:10.3233/SW- 140134.
[17] F. Desimoni, N. Bikakis, L. Po, G. Papastefanatos, A comparative study of
state-of-theart linked data visualization tools, in: V. Ivanova, P. Lambrix, C. Pesquita, V. Wiens
(Eds.), Proceedings of the Fifth International Workshop on Visualization and Interaction
for Ontologies and Linked Data co-located with the 19th International Semantic Web
Conference (ISWC 2020), volume 2778 of CEUR workshop proceedings, CEUR-WS.org,
Virtual Event, 2020, pp. 1–13.
[18] E. Bernasconi, M. Ceriani, D. D. Di Pierro, S. Ferilli, D. Redavid, Linked Data Interfaces: A</p>
      <p>Survey, Information 14 (2023) 483. doi:10.3390/info14090483.
[19] A.-S. Dadzie, E. Pietriga, Visualisation of Linked Data - Reprise, Semantic Web 8 (2017)
1–21. doi:10.3233/SW- 160249.
[20] J. Klímek, P. Škoda, M. Nečaský, Survey of tools for Linked Data consumption, Semantic</p>
      <p>Web 10 (2019) 665–720. doi:10.3233/SW- 180316.
[21] M. Aguiar, S. Nunes, B. Giesteirad, A survey on user interaction with linked data, in:
P. Lambrix, C. Pesquita, V. Wiens (Eds.), Proceedings of the Sixth International Workshop
on the Visualization and Interaction for Ontologies and Linked Data co-located with the
20th International Semantic Web Conference (ISWC 2021), volume 3023 of CEUR workshop
proceedings, CEUR-WS.org, Virtual Event, 2021, pp. 13–28.
[22] G. Mai, K. Janowicz, Y. Hu, G. McKenzie, A linked data driven visual interface for the
multi-perspective exploration of data across repositories, in: V. Ivanova, P. Lambrix,
S. Lohmann, C. Pesquita (Eds.), Proceedings of the Second International Workshop on
Visualization and Interaction for Ontologies and Linked Data co-located with the 15th
International Semantic Web Conference (VOILA@ISWC 2016), volume 1704 of CEUR
workshop proceedings, CEUR-WS.org, Kobe, Japan, 2016, pp. 93–101.
[23] B. Regalia, K. Janowicz, G. Mai, phuzzy.link: A SPARQL-powered client-sided extensible
semantic web browser, in: V. Ivanova, P. Lambrix, S. Lohmann, C. Pesquita (Eds.),
Proceedings of the Third International Workshop on Visualization and Interaction for Ontologies
and Linked Data co-located with the 16th International Semantic Web Conference (ISWC</p>
    </sec>
    <sec id="sec-10">
      <title>A. Appendix - Detailed results</title>
      <p>
        This section presents detailed results about the pairwise comparison of the diferent approaches
across all dependent variables. The R package bootES [
        <xref ref-type="bibr" rid="ref21">34</xref>
        ] was used for the analysis. The
following applies to all tables. The first two columns of the tables represent the two visualization
approaches being compared. The third column represents the mean value diference between
the two groups:   ( 1) -   ( 2) . Positive values for both the lower and upper
confidence interval bounds (    and    ℎ values) suggest that the visualization in the first
column produced significantly higher values than the one in the second column. Conversely,
negative values for both    and    ℎ indicate that the visualization in the second column
resulted in significantly higher values than the one in the first column. A statistically significant
diference between the two groups is implied when the confidence interval of the mean diference
does not enclose zero. This is highlighted through a light yellow coloured background in the
tables. The bias is the diference between the mean of the resamples and the mean of the original
sample. The SE (standard error) is the standard deviation of the resampled means [
        <xref ref-type="bibr" rid="ref21">34</xref>
        ]. The
number of resamples used in the analysis was N=5000.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>W.</given-names>
            <surname>Kuhn</surname>
          </string-name>
          , E. Hamzei,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tomko</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Winter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <article-title>The semantics of place-related questions</article-title>
          ,
          <source>Journal of Spatial Information Science</source>
          (
          <year>2021</year>
          )
          <fpage>157</fpage>
          -
          <lpage>168</lpage>
          . doi:
          <volume>10</volume>
          .5311/JOSIS.
          <year>2021</year>
          .
          <volume>23</volume>
          .161.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>M. F.</given-names>
            <surname>Goodchild</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Guo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Annoni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Bian</surname>
          </string-name>
          ,
          <string-name>
            <surname>K. de Bie</surname>
            , F. Campbell,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Craglia</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Ehlers</surname>
            ,
            <given-names>J. van Genderen</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>D</given-names>
            .
            <surname>Jackson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. J.</given-names>
            <surname>Lewis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pesaresi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Remetey-Fulopp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Simpson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Skidmore</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Woodgate</surname>
          </string-name>
          ,
          <article-title>Next-generation digital earth</article-title>
          ,
          <source>Proceedings of the National Academy of Sciences</source>
          <volume>109</volume>
          (
          <year>2012</year>
          )
          <fpage>11088</fpage>
          -
          <lpage>11094</lpage>
          . doi:
          <volume>10</volume>
          .1073/pnas.1202383109.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>S.</given-names>
            <surname>Scheider</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Degbelo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Kuhn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Przibytzin</surname>
          </string-name>
          ,
          <article-title>Content and context description - How linked spatio-temporal data enables novel information services for libraries, gis</article-title>
          .
          <source>Science</source>
          (
          <year>2014</year>
          )
          <fpage>138</fpage>
          -
          <lpage>149</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>K.</given-names>
            <surname>Janowicz</surname>
          </string-name>
          ,
          <article-title>The role of space and time for knowledge organization on the semantic web</article-title>
          ,
          <source>Semantic Web</source>
          <volume>1</volume>
          (
          <year>2010</year>
          )
          <fpage>25</fpage>
          -
          <lpage>32</lpage>
          . doi:
          <volume>10</volume>
          .3233/SW-2010-0001.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>L. M. V.</given-names>
            <surname>Blázquez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Villazón-Terrazas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Saquicela</surname>
          </string-name>
          , A. de León,
          <string-name>
            <given-names>O.</given-names>
            <surname>Corcho</surname>
          </string-name>
          , A. GómezPérez,
          <article-title>GeoLinked data and INSPIRE through an application case</article-title>
          , in: D.
          <string-name>
            <surname>Agrawal</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Zhang</surname>
            ,
            <given-names>A. E.</given-names>
          </string-name>
          <string-name>
            <surname>Abbadi</surname>
            ,
            <given-names>M. F.</given-names>
          </string-name>
          <string-name>
            <surname>Mokbel</surname>
          </string-name>
          (Eds.),
          <source>18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems (ACM-GIS</source>
          <year>2010</year>
          ), ACM, San Jose, California, USA,
          <year>2010</year>
          , pp.
          <fpage>446</fpage>
          -
          <lpage>449</lpage>
          . doi:
          <volume>10</volume>
          .1145/1869790.1869858.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>W.</given-names>
            <surname>Kuhn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Kauppinen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Janowicz</surname>
          </string-name>
          ,
          <article-title>Linked Data - A paradigm shift for Geographic Information Science</article-title>
          , in: M.
          <string-name>
            <surname>Duckham</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Pebesma</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Stewart</surname>
          </string-name>
          , A. U. Frank (Eds.), Geographic Information Science - Eighth International Conference, Springer International Publishing, Vienna, Austria,
          <year>2014</year>
          , pp.
          <fpage>173</fpage>
          -
          <lpage>186</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>319</fpage>
          -11593-1_
          <fpage>12</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Shvaiko</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Farazi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Maltese</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ivanyukovich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Rizzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Ferrari</surname>
          </string-name>
          , G. Ucelli,
          <article-title>Trentino government linked open geo-data: A case study</article-title>
          , in: P.
          <string-name>
            <surname>Cudré-Mauroux</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Heflin</surname>
            , E. Sirin,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Tudorache</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Euzenat</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Hauswirth</surname>
            ,
            <given-names>J. X.</given-names>
          </string-name>
          <string-name>
            <surname>Parreira</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Hendler</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <string-name>
            <surname>Schreiber</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Bernstein</surname>
          </string-name>
          , E. Blomqvist (Eds.),
          <source>The Semantic Web - ISWC 2012 - 11th International Semantic Web Conference</source>
          , volume
          <volume>7650</volume>
          of Lecture notes in computer science, Springer, Boston, Massachusetts, USA,
          <year>2012</year>
          , pp.
          <fpage>196</fpage>
          -
          <lpage>211</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>642</fpage>
          -35173-0_
          <fpage>13</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>C.</given-names>
            <surname>Keßler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Janowicz</surname>
          </string-name>
          , T. Kauppinen, Spatial@linkedscience
          <article-title>- Exploring the research field of GIScience with linked data</article-title>
          , in: N.
          <string-name>
            <surname>Xiao</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Kwan</surname>
            ,
            <given-names>M. F.</given-names>
          </string-name>
          <string-name>
            <surname>Goodchild</surname>
          </string-name>
          , S. Shekhar (Eds.),
          <source>Proceedings of the Seventh International Conference on Geographic Information Science (GIScience2012)</source>
          , Springer-Verlag New York, Columbus, Ohio, USA,
          <year>2012</year>
          , pp.
          <fpage>102</fpage>
          -
          <lpage>115</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>642</fpage>
          -33024-
          <issue>7</issue>
          _
          <fpage>8</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Degbelo</surname>
          </string-name>
          ,
          <article-title>Open data user needs: a preliminary synthesis, in:</article-title>
          <string-name>
            <given-names>A. E. F.</given-names>
            <surname>Seghrouchni</surname>
          </string-name>
          , G. Sukthankar, T.-Y. Liu,
          <string-name>
            <surname>M.</surname>
          </string-name>
          v. Steen (Eds.),
          <source>Companion Proceedings of the Web Conference</source>
          <year>2020</year>
          , ACM, Taipei, Taiwan,
          <year>2020</year>
          , pp.
          <fpage>834</fpage>
          -
          <lpage>839</lpage>
          . doi:
          <volume>10</volume>
          .1145/3366424.3386586.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>L.</given-names>
            <surname>Diaz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Remke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Kauppinen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Degbelo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Foerster</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Stasch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Rieke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Schaefer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Baranski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bröring</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Wytzisk</surname>
          </string-name>
          ,
          <string-name>
            <surname>Future</surname>
            <given-names>SDI</given-names>
          </string-name>
          -
          <article-title>Impulses from Geoinformatics research</article-title>
          and IT trends,
          <source>International Journal of Spatial Data Infrastructures Research</source>
          <volume>7</volume>
          (
          <year>2012</year>
          )
          <fpage>378</fpage>
          -
          <lpage>410</lpage>
          . doi:
          <volume>10</volume>
          .2902/
          <fpage>1725</fpage>
          -
          <lpage>0463</lpage>
          .
          <year>2012</year>
          .
          <volume>07</volume>
          .
          <year>art18</year>
          .
          <year>2017</year>
          ), volume
          <volume>1947</volume>
          <source>of CEUR workshop proceedings, CEUR-WS.org</source>
          , Vienna, Austria,
          <year>2017</year>
          , pp.
          <fpage>34</fpage>
          -
          <lpage>44</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>A.</given-names>
            <surname>Potnis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. S.</given-names>
            <surname>Durbha</surname>
          </string-name>
          ,
          <article-title>Exploring visualization of geospatial ontologies using cesium</article-title>
          , in: V.
          <string-name>
            <surname>Ivanova</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Lambrix</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Lohmann</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          Pesquita (Eds.), Proceedings of the Second International Workshop on Visualization and
          <article-title>Interaction for Ontologies and Linked Data co-located with the 15th International Semantic Web Conference (VOILA@ISWC 2016)</article-title>
          , volume
          <volume>1704</volume>
          <source>of CEUR workshop proceedings</source>
          , CEUR-WS.org, Kobe, Japan,
          <year>2016</year>
          , pp.
          <fpage>143</fpage>
          -
          <lpage>150</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>L.</given-names>
            <surname>Fürhof</surname>
          </string-name>
          ,
          <article-title>Rethinking the usage and experience of clustering markers in web mapping</article-title>
          ,
          <source>PeerJ Preprints</source>
          (
          <year>2019</year>
          ). doi:
          <volume>10</volume>
          .7287/peerj.preprints.27858v3.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>J.</given-names>
            <surname>Bertin</surname>
          </string-name>
          ,
          <article-title>Semiology of graphics: diagrams, networks, maps (Translated by William J</article-title>
          . Berg), The University of Wisconsin Press, Madison,
          <year>1983</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>B.</given-names>
            <surname>Saket</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Endert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Stasko</surname>
          </string-name>
          ,
          <article-title>Beyond usability and performance: A review of user experience-focused evaluations in visualization</article-title>
          , in: M.
          <string-name>
            <surname>Sedlmair</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Isenberg</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Isenberg</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Mahyar</surname>
          </string-name>
          , H. Lam (Eds.),
          <source>Proceedings of the Beyond Time and Errors on Novel Evaluation Methods for Visualization - BELIV '16</source>
          , ACM Press, Baltimore, Maryland, USA,
          <year>2016</year>
          , pp.
          <fpage>133</fpage>
          -
          <lpage>142</lpage>
          . doi:
          <volume>10</volume>
          .1145/2993901.2993903.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>M.</given-names>
            <surname>Gao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Kortum</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Oswald</surname>
          </string-name>
          ,
          <article-title>Psychometric evaluation of the USE (Usefulness, Satisfaction, and Ease of use) questionnaire for reliability and validity</article-title>
          ,
          <source>Proceedings of the Human Factors and Ergonomics Society Annual Meeting</source>
          <volume>62</volume>
          (
          <year>2018</year>
          )
          <fpage>1414</fpage>
          -
          <lpage>1418</lpage>
          . doi:
          <volume>10</volume>
          .1177/1541931218621322.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [29]
          <string-name>
            <surname>A. M. Lund</surname>
          </string-name>
          ,
          <article-title>Measuring usability with the USE questionnaire</article-title>
          ,
          <source>Usability Interface 8</source>
          (
          <year>2001</year>
          )
          <fpage>3</fpage>
          -
          <lpage>6</lpage>
          . ISBN:
          <fpage>1078</fpage>
          -
          <lpage>0874</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>L.</given-names>
            <surname>Bernard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Henzen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Degbelo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Nüst</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Seegert,</surname>
          </string-name>
          <article-title>NFDI4Earth: Improving Research Data Management in the Earth System Sciences</article-title>
          , in: Y.
          <string-name>
            <surname>Sure-Vetter</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          Goble (Eds.),
          <source>Proceedings of the Conference on Research Data Infrastructure</source>
          , volume
          <volume>1</volume>
          , Karlsruhe, Germany,
          <year>2023</year>
          . doi:
          <volume>10</volume>
          .52825/cordi.v1i.
          <fpage>288</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>W.</given-names>
            <surname>Scheibel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Trapp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Limberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Döllner</surname>
          </string-name>
          ,
          <article-title>A taxonomy of treemap visualization techniques</article-title>
          , in: A.
          <string-name>
            <surname>Kerren</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Hurter</surname>
          </string-name>
          , J. Braz (Eds.),
          <source>Proceedings of the 15th International Joint Conference on Computer Vision</source>
          , Imaging and
          <source>Computer Graphics Theory and Applications</source>
          ,
          <string-name>
            <surname>VISIGRAPP</surname>
          </string-name>
          <year>2020</year>
          , Volume
          <volume>3</volume>
          : IVAPP,
          <string-name>
            <surname>SCITEPRESS</surname>
          </string-name>
          , Valletta, Malta,
          <year>2020</year>
          , pp.
          <fpage>273</fpage>
          -
          <lpage>280</lpage>
          . doi:
          <volume>10</volume>
          .5220/0009153902730280.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>B.</given-names>
            <surname>Zheng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Sadlo</surname>
          </string-name>
          ,
          <article-title>On the visualization of hierarchical multivariate data</article-title>
          ,
          <source>in: 2021 IEEE 14th Pacific Visualization Symposium (PacificVis)</source>
          , IEEE, Tianjin, China,
          <year>2021</year>
          , pp.
          <fpage>136</fpage>
          -
          <lpage>145</lpage>
          . doi:
          <volume>10</volume>
          .1109/PacificVis52677.
          <year>2021</year>
          .
          <volume>00026</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>S.</given-names>
            <surname>Meißner</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. Degbelo,</surname>
          </string-name>
          <article-title>User performance modelling for spatial entities comparison with geodashboards: Using view quality and distractor as concepts</article-title>
          , in: M.
          <string-name>
            <surname>Nebeling</surname>
            ,
            <given-names>L. D.</given-names>
          </string-name>
          <string-name>
            <surname>Spano</surname>
            ,
            <given-names>J. C.</given-names>
          </string-name>
          Campos (Eds.),
          <source>Companion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS Companion</source>
          <year>2024</year>
          ), ACM, Cagliari, Italy,
          <year>2024</year>
          , pp.
          <fpage>7</fpage>
          -
          <lpage>14</lpage>
          . doi:
          <volume>10</volume>
          .1145/3660515.3661325.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>K. N.</given-names>
            <surname>Kirby</surname>
          </string-name>
          , D. Gerlanc,
          <article-title>BootES: An R package for bootstrap confidence intervals on efect sizes</article-title>
          ,
          <source>Behavior Research Methods</source>
          <volume>45</volume>
          (
          <year>2013</year>
          )
          <fpage>905</fpage>
          -
          <lpage>927</lpage>
          . doi:
          <volume>10</volume>
          .3758/s13428-013-0330-5.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>