=Paper=
{{Paper
|id=Vol-3140/hurtienne
|storemode=property
|title=Supporting User Interface Design with Image Schemas: The ISCAT Database as a Research Tool
|pdfUrl=https://ceur-ws.org/Vol-3140/paper4.pdf
|volume=Vol-3140
|authors=Jörn Hurtienne,Stephan Huber,Cordula Baur
|dblpUrl=https://dblp.org/rec/conf/isd2/HurtienneHB22
}}
==Supporting User Interface Design with Image Schemas: The ISCAT Database as a Research Tool==
https://ceur-ws.org/Vol-3140/paper4.pdf
Supporting User Interface Design with Image Schemas:
The ISCAT Database as a Research Tool
Jörn Hurtienne 1, Stephan Huber 1 and Cordula Baur 1
1
Julius-Maximilians-Universität, Chair of Psychological Ergonomics, Oswald-Külpe-Weg 82, 97074 Würzburg,
Germany
Abstract
This paper introduces ISCAT – image schema catalogue – that has been developed for
supporting image schema research in human-computer interaction. It contains image schema
definitions collected from the literature and image schema analyses of a wide range of user
interfaces. It also contains metaphorical extensions of image schemas with linguistic examples
and associated studies into the psychological reality of metaphors.
Keywords 1
Image schemas, image-schematic metaphors, research database
1. Introduction: Image Schemas for User Interface Design
In the field of human-computer interaction (HCI), image schemas and image-schematic metaphors
have proven to be valuable tools for the design of user interfaces. As mental patterns of thought, it is
believed that mirroring image-schematic metaphors at the user interface makes the user interface more
intuitive to use. Also, because of their relatively frequent occurrence in daily sensorimotor experience
and domains of human judgment they have become firmly anchored in cognition and are activated
automatically. Their early acquisition and multimodality make them suitable also for designing
inclusive user interfaces, e.g. when designing user interfaces for the elderly. Their abstractness leaves
space for the creativity of user interface designers because they can see what is essential while keeping
the concrete instantiation of an image schema open to be designed (e.g. UP-DOWN can be instantiated
in a vertical slider, a string to be pulled from above, or a vertical sliding gesture on a touchscreen).
Across several studies, all three criteria, intuitive use, inclusiveness and innovativeness, have been
shown to be positively influenced when applying image-schematic metaphors to user interface design
(e.g. [4, 7]; see [3] for a review of the HCI literature). To better support user interface design with image
schemas we created the ISCAT database as a central image schema repository. In the following we
report on the requirements, the structure of the database and more recent spin-offs from this work.
2. Requirements for a Central Image Schema Repository
From the results of previous studies applying image schemas in the design of user interfaces, several
requirements were formulated that could make an image-schema database useful in different phases of
a user-centred design process [2] (p. 191):
• Supporting the phase of context-of-use analysis. To enhance the reliability of image-schema
extractions, analysts need a good command of image-schema definitions and their interrelations.
As image-schema definitions sometimes make subtle distinctions (e.g. the distinction between
active and passive ENABLEMENT or the object and observer view of NEAR-FAR), analysts need a
The Sixth Image Schema Day (ISD6), March 24–25, 2022, Jönköping, Sweden
EMAIL: joern.hurtienne@uni-wuerzburg.de (J. Hurtienne); stephan.huber@uni-wuerzburg.de (S. Huber); cordula.baur@uni-wuerzburg.de
(C. Baur)
ORCID: 0000-0001-5295-9772 (J. Hurtienne); 0000-0002-5619-3222 (S. Huber); 0000-0002-0228-9747 (C. Baur)
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
possibility to consult definitions of image schemas. Hence, the database should provide a central
repository of definitions and descriptions of image schemas, their relations to other image
schemas and typical user tasks.
• Supporting the phases of producing design solutions and evaluation with inspirational and
evaluated examples. If using requirements lists including image schemas, designers may wish to
get inspired by image-schema instantiations in current user interfaces. To support this, the
database needs to hold (and make available) a range of user interface examples from a variety of
interactive products, along with evaluations of the effects these image-schema instances have and
possible constraints of their use.
• Supporting the phase of producing design solutions with image-schematic metaphors. Even
without employing image schemas during the whole design process, designers might want to
look up how user interface elements can be designed to convey certain abstract meanings. A
question might be: “How can I convey the idea of importance to the user?” As this relates to
metaphorical extensions (e.g. IMPORTANT IS CENTRAL, IMPORTANT IS UP, IMPORTANT IS BIG),
designers would benefit from a centralised collection of metaphorical extensions already known
that are currently scattered throughout the literature.
• Supporting the extension and customisation of the database. The research community should be
able to add to the database other and new instances of image schemas in user interfaces.
Taking these requirements into account, a database was built that provides a rich set of information
on image schemas and their applications in user interface design. The database is realised as a Zope /
Plone content management system with a custom add-on written in Python. It is currently available at
https://iscat.psyergo.uni-wuerzburg.de/ and new users can be registered when they contact the first
author.
3. ISCAT Overview
The user interface of ISCAT (Image-Schema CATalogue) is structured in two main parts. The first
part contains image-schema descriptions and metaphorical extensions found in language (Figure 1).
Each image-schema description contains a definition, its entailments, and synonyms. It contains notes
on the experiential grounding of the image schema and lists relations to other image schemas. Attached
to image-schema entries are their metaphorical extensions as found in the linguistic and psychological
literature. Metaphorical extensions are documented with their target domains, experiential grounding,
and linguistic examples from different languages (not shown in Figure 1). As of today, the database
contains 49 image schemas and 349 metaphorical extensions. Each description of an image schema is
linked to image-schema instances from user interface studies (right hand column in Figure 1).
The second part, user interface studies, contains a number of user interface examples. A user
interface study can be divided into sub-studies and user interfaces. For instance, Figure 2, shows a flap
lever (user interface) in the sub-study Centre Pedestal of the study Cockpit A320. Each user interface
contains one or several images depicting the user interface element and a short description of what the
UI element is for and what it does. All image-schema instances that were extracted from the UI element
are shown below this information. They include a description of why the analyst thought that the image-
schema category applies to the interface, the effect that is achieved by the image-schema instance, how
strong the effect is, whether it supports, hinders or is neutral to usability (indicated by a smiley icon),
and whether the analyst thought this is a prototypical instantiation of the image schema (indicated by
the light-bulb icon). Further notes can be given regarding the decisions the analyst had made in
analysing an instance of the image schema.
User interface examples were collected by extracting image-schema instances from products as far
ranging like cash and ticket machines, enterprise resource planning systems, websites, CAD systems,
Virtual Reality interfaces, tangible interfaces, data visualisations and an Airbus A320 cockpit. As of
today, the database contains 116 studies and sub-studies with 754 user interfaces, and 2693 instances
of image schemas in user interface elements.
Figure 1: Example page from the ISCAT database – information on the image schema PATH
Figure 2: Example page from the ISCAT database – the user interface study of an Airbus A320
Figure 3: Example from ISCAT – empirical studies of the image-schematic metaphor SIMILAR IS NEAR –
DIFFERENT IS FAR
In addition, users have the ability to search the database via the search box provided in the upper
right corner (Figures 1, 2). Also, specific searches for image-schema instances are possible, the user
can set more specific filters, e.g. on specific user interfaces, effect strength, and so on. Further, the
advanced search offers possibilities to search for specific items like pictures, metaphorical extensions,
etc. The database has an added user management system, so that new users can become members of the
ISCAT community and contribute to the contents of the database.
4. ISCAT Spin-offs
Over time the image schema catalogue has become a central repository of all things image-
schematic. It has been well accepted by researchers, students and practitioners. Currently there are 725
registered users of which 40 appear as contributors and study authors in the database. Its
comprehensiveness, however, stands against the requirement of an easy-to-use inspirational tool for
designers. Thus, spin-off-projects have been developed to promote a better ease-of-use of image-
schemas and their metaphors in design:
1. Supporting researchers. Since the inception of the database in 2007, a great number of studies
have been conducted into the psychological reality of a number of metaphorical extensions.
These were mainly conducted in cognitive social psychology, consumer psychology, cognitive
linguistics and HCI. Since these are informative for researchers, we added the possibility in
ISCAT to collect empirical studies of each metaphorical extension (Figure 3). At the moment
this feature is experimental and only available to a registered subset of users.
2. Supporting designers. In a recent book project, 65 of the most salient metaphorical extensions
were selected and illustrated, along with linguistic examples, empirical evidence for their
psychological reality and examples of their applications in design [6]. The book can be easily
browsed due to its visual attractiveness and short texts. It thus makes for an attractive coffee
table book and inspiration for designers.
3. Supporting design processes. We are currently working on interactive toolkits that support
designers experiencing (and thus understanding) the features and entailments of single image
schemas for design. A first attempt in this direction was making force-dynamic image schemas
mentally and physically graspable through interactive tangible prototypes [5]. Our ongoing work
introduces a physical toolkit for image schemas aimed at people working on designing physical
data representations (aka data physicalisations) [1].
5. Conclusion and Outlook
The image-schema database ISCAT seems a good starting point for providing the HCI and design
communities with an information repository on image schemas and image-schematic metaphors. But
also psychologists and cognitive linguists have found the database useful for their research. Image-
schema definitions, for example, can help to enhance the agreement between coders and reduce
subjectivity when extracting image schemas during a context-of-use analysis. Examples of image-
schema instantiations could inspire user interface designers in designing future solutions. Both,
linguistic data and user interface examples, could contribute to new insights in user interface design.
Although such a database is practical, its users need to be reminded that the image schemas extracted
from user interfaces (as well as the metaphorical extensions from linguistics) reflect the subjective view
of the respective analyst. Thus, the examples in the database should be seen as a repository of
hypotheses for researchers and heuristics for designers.
The accumulation of a large number of image-schema instances leads to the emergence of new
research questions. The database could be used for mining general rules of image-schema application.
Thus, implicitly applied design rules can be detected across different user-interface domains and can be
subjected to further research. Early analyses of the current set of over 2600 image-schema instances
reveal
• rules of image-schema co-occurrences (e.g. BLOCKAGE needs to be followed by RESTRAINT
REMOVAL, ATTRACTION is resulting in DIVERSION);
• image-schema transformation rules (e.g. UP-DOWN is readily substituted for by FRONT-BACK
relations);
• typical problems (e.g. UI elements that belong to the same task are often far away from each
other without communicating their relation via a LINK or common CONTAINER image schema).
The database – tool and content – is open for further improvements. A future version could better
meet the specific requirements of design practitioners. Such a version would feature better
interconnections between the two parts of the database (linguistic and UI data). It would also provide
easier community support. The database could profit from including more kinds of data, for example
data on typical user tasks and their associated image schemas. Other information could include typical
constraints in using specific instantiations of image schemas that are determined by differing contexts
of use. A perennial topic is maintaining the quality and completeness of the data with a range of
contributors to the database. Finally, a formal evaluation of ISCAT will be necessary to validate its
usability and general value for its users.
6. References
[1] C. Baur, C. Wienrich, J. Hurtienne, Form follows mental models: Finding instantiations of image
schemas using a design research approach, in: Proceedings DIS’22 Designing Interactive Systems
Conference, ACM, New York, NY, 2022.
[2] J. Hurtienne, Image schemas and design for intuitive use: Exploring new guidance for user
interface design, Ph.D. thesis, Technische Universität Berlin, 2011. doi: 10.14279/depositonce-
2753.
[3] J. Hurtienne, How cognitive linguistics inspires HCI: Image schemas and image-schematic
metaphors, International Journal of Human-Computer Interaction, 33 (2017) 1-20.
doi:10.1080/10447318.2016.1232227.
[4] J. Hurtienne, K. Klöckner, S. Diefenbach, C. Nass, A. Maier, Designing with Image Schemas:
Resolving the Tension Between Innovation, Inclusion and Intuitive Use, Interacting with
Computers, 27 (2015) 235–255. doi:10.1093/iwc/iwu049
[5] J. Hurtienne, D. Löffler, P. Gadegast, S. Hußlein, Comparing Pictorial and Tangible Notations of
Force Image Schemas, in: Proceedings of the Ninth International Conference on Tangible,
Embedded, and Embodied Interaction, ACM, New York, NY, 2015, pp. 249–256.
doi:10.1145/2677199.2680553.
[6] J. Hurtienne, D. Löffler, C. Hüsch, D. Reinhardt, R. Tscharn, S. Huber, S., Happy is up, sad is
down. 65 metaphors for design, BIS Publishers, Amsterdam, 2020.
[7] R. Tscharn, Innovative and age-inclusive interaction design with image-schematic metaphors,
Ph.D. thesis, Julius-Maximilians-Universität Würzburg, 2019. doi:10.25972/OPUS-17576.