=Paper=
{{Paper
|id=Vol-3328/workshop4
|storemode=property
|title=Designing Data Physicalisations - using physical Image Schema Instantiations
|pdfUrl=https://ceur-ws.org/Vol-3328/workshop4.pdf
|volume=Vol-3328
|authors=Cordula Baur,Carolin Wienrich,Jörn Hurtienne
|dblpUrl=https://dblp.org/rec/conf/etis/BaurWH22
}}
==Designing Data Physicalisations - using physical Image Schema Instantiations==
https://ceur-ws.org/Vol-3328/workshop4.pdf
Designing Data Physicalisations – with Physical Image Schema
Instantiations
Cordula Baur 1, Carolin Wienrich2 and Jörn Hurtienne 1
1
Julius-Maximilians-Universität, Chair of Psychological Ergonomics, Oswald-Külpe-Weg 82, 97074 Würzburg,
Germany
2
Julius-Maximilians-Universität, Human-Technology-Systems, Oswald-Külpe-Weg 82, 97074 Würzburg,
Germany
Abstract
In an iterative design process, we created Image Schema Instantiations, an inspirational tool,
to make image schema theory easily accessible for designers of data physicalisations, with the
aim to foster creativity and intuitive mappings of abstract data to physical properties. The
workshop provides Image Schema Instantiations as inspirational tool for a data physicalisation
design session. It enables participants to gather experience in designing data physicalisations
and to compare design sessions with and without Image Schema Instantiations. A structured
reflection and feedback session helps to evaluate the impact of the Image Schema
Instantiations. Aim of this workshop is to provide researchers the opportunity to utilize the
Image Schema Instantiations directly in the design process and reflect on the usefulness of the
tool.
Keywords 1
Data physicalisation, image schemas, design research
1. Background
Data physicalisations are physical objects which encode data by their geometry or material
properties [1]. They aim to enhance cognition, communication, learning, problem solving and decision-
making by mapping abstract data to physical and spatial properties [1, 2]. Even if they provide the
opportunity to use different modalities, previous research found that they do not use their full potential
to facilitate the embodied understanding of complex data [3].
This could be enhanced by incorporating mental building blocks, so called image schemas, in the design
process of data physicalisations. Image schemas are abstract representations of recurring dynamic
patterns of bodily interactions [4]. They help us to structure and understand the world by addressing
mental models, like the image schemas UP-DOWN or CONTAINER, which we experience recurrently in
our everyday life. For example, a glass of water contains both image schemas: UP-DOWN because of the
rising and falling level of water when pouring water or drinking up, while the glass itself acts as
CONTAINER, separating an outside from an inside by a border of glass.
Previous research revealed image schemas to be useful to analyse and evaluate user interfaces, to
describe requirements and to support the user interface design process. Graphical and tangible user
interfaces which contain designed instantiations of image schemas showed that image schema theory
can work as design language and powerful tool offering insight and value [5-11]. Furthermore, image
schemas, when instantiated in user interfaces, enable more inclusive, intuitive and innovative
interaction [12]. These are also desirable properties for data physicalisations. As abstract
Proceedings of ETIS 2022, November 7–10, 2022, Toulouse, France
EMAIL: cordula.baur@uni-wuerzburg.de (C. Baur); carolin.wienrich@uni-wuerzburg.de (C. Wienrich); joern.hurtienne@uni-wuerzburg.de
(J. Hurtienne)
ORCID: 0000-0002-0228-9747 (C.Baur); 0000-0003-3052-7172 (C.Wienrich); 0000-0001-5295-9772 (J.Hurtienne)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
representations of sensorimotor experiences [4] image schemas are multimodal and able to represent
visual, aural and tactile qualities as well as kinaesthetic experiences [13]. This enhances not only
inclusiveness but is also promising for multisensory data physicalisations. They could support designers
in finding appropriate mappings of abstract data to physical properties or address the challenges found
in actual data physicalisations, like addressing primary vision, being static and passive [3]. Furthermore,
image schemas’ abstractness leaves enough space for designers to decide how to instantiate them. This
enables more innovative design ideas [12].
An analysis of current data physicalisations, the authors made in an earlier phase of this project
[paper submitted], showed that a range of image schemas is already incorporated in data
physicalisations and there is potential to use more and different image schemas to make physicalisations
more expressive or easier to understand. To make image schemas accessible for designers and easy to
incorporate in the design process of data physicalisations, we created physical instantiations of selected
image schemas. These are meant to be part of a toolkit which will be utilized in the design process of
data physicalisations, with the aim to foster the designer’s creativity and the user’s intuitive
understanding of the data physicalisation. In comparison to other data physicalisation workshops our
approach is not to provide tools which constrain the design space of data physicalisations [14], but to
enhance creativity and intuitive design.
In an iterative research oriented design process [15] including several feedback sessions with
different groups of participants (design experts, HCI experts and lay people) we went through the stages
of visual representations, physical representations, and interactive representations [16]. For each image
schema a final design solution for each stage was found, which then informed the following stage. For
the final set of Image Schema Instantiations (for some excerpts see Figure: 1) we choose either the final
interactive object or – when the interaction appeared to be hindering or confusing – the final physical
object.
Figure 1: Physical Image Schema Instantiations for the image schemas CENTER-PERIPHERY, SMOOTH-
ROUGH, UP-DOWN, CONTAINER, STRONG-WEAK, HARD-SOFT.
In total the toolkit contains instantiations of 14 image schemas: OBJECT, PAINFUL, LINKAGE, UP-
DOWN, LEFT-RIGHT, SMOOTH-ROUGH, STRONG-WEAK, HEAVY-LIGHT, STRAIGHT-CROOKED, HARD-
SOFT, CENTER-PERIPHERY, NEAR-FAR, CONTENT-CONTAINER, and PART-WHOLE. The latest evaluation
showed that this number tends to overwhelm some designers, so we are going to provide only a subset
of seven image schemas to each group.
The physical Image Schema Instantiations are promising for the design of data physicalisations
because they are already closer to the target medium, i.e., closer to the ideal form to be used in the
design process of data physicalisations. Further, they eliminate the need to cope with formal definitions
like those provided by the ISCAT database [17] and make image schemas more easily accessible. Also,
social engagement could be higher, because objects can be better explored and discussed in teams than
bare textual definitions. The physical instantiations make it easy to compare image schemas with each
other, to identify differences and to figure out which image schemas are the most appropriate for the
current purpose. They can be seen as inspiring design suggestions how image schemas can be
instantiated in one’s own data physicalisation project or can enhance creativity in a more abstract way,
also fostering the unconscious implementation of image schemas. Furthermore, they are promising to
enhance the (physical) engagement and to consider the effects of instantiating the image schema in a
data physicalisation.
The format of a full-day workshop offers participants to gather experience in designing data
physicalisations and to engage in depth with the Image Schema Instantiations. They experience the
design process of data physicalisations with and without the Image Schema Instantiations and get the
opportunity to compare both and reflect on the impact of the tool.
The first part of the workshop offers a brief introduction to the field of data physicalisation. This
opens the workshop to everyone, regardless of prior knowledge or background. A hands-on experience,
including brainstorming, conceptualizing, and building a data physicalisation in small teams offers
active engagement with the topic and afterwards the opportunity to reflect on the own design action and
the design outcome.
In the second part image schema theory and the Image Schema Instantiations are introduced. Now
participants can explore the Image Schema Instantiations on their own and utilize them in the design
process of data physicalisations in a second design session. Afterwards there is time to reflect on both
design sessions, the differences as well as the impact of the Image Schema Instantiations.
During the design process of the Image Schema Instantiations, we already collected first insights
about their usage in group settings. This workshop provides the opportunity to apply them in the design
process of data physicalisations. Our aim is to gather feedback about the impact of the Image Schema
Instantiations on the design process and to evaluate their usefulness. Further we want to create together
with the workshop participants data physicalisations designed with and without Image Schema
Instantiations, in order to analyse and compare them.
2. Organizers
2.1. Cordula Baur
is currently working toward the Ph.D. degree with the Chair of Psychological Ergonomics,
University of Wuerzburg, Wuerzburg, Germany. Initially coming from a design background, she
engaged in her master’s thesis with the topic of perception and the fields of sensory design as well as
research through design. In her PhD she deals with the fields of data physicalisation and image schemas.
For this aim she did an analysis of data physicalisations regarding image schemas and their potential
[paper submitted] and created in an iterative design process the Image Schema Instantiations [16].
Further she also offers lectures to this topic. Her research interests include tangible interaction, data
physicalisation, image schemas, sensory design, and design research. She is the contact person for this
workshop.
2.2. Carolin Wienrich
is currently a Professor for Psychology of Intelligent Interactive Systems with the University of
Wuerzburg, Wuerzburg, Germany, and a Co-Leader of the XR HUB Wuerzburg. Her research interests
focus on interaction paradigms between humans and digital entities as well as change experiences
during and after digital interventions.
2.3. Jörn Hurtienne
is currently a Full Professor and Chair Holder in Psychological Ergonomics with the University of
Wuerzburg, Wuerzburg, Germany. He is interested in data physicalisation, tangible interaction, design
for intuitive use, image schema theory and user experience.
3. Workshop Structure
Table 1
Workshop Structure
Time Activity Description Resources
10 am to 1045 am
30 welcome &
introduction participants introduce themselves
of the with a physical object
participants > mail in advance: to inform
participants to bring an object
1045 am to 1100 am input session brief introduction to the field of laptop
data data physicalisation projector
physicalisation wire
presentation
1100 am to 1230 pm design session participants team up in groups of datasets
1 two to four people. Each group task sheets
gets a dataset and the design task: sketching- and
1) generate different ideas to crafting-material
physicalize the dataset
2) choose one idea
3) build a data physicalisation
1230 pm to 1300 pm presentation each group presents their final
physicalisation
1300 pm to 1400 pm lunch break
1400 pm to 1430 pm input session brief introduction to the field of laptop
image image schemas & presentation of projector
schemas the Image Schema Instantiations wire
presentation
1430 pm to 1630 pm design session participants come back to their Image Schema
2 group, get a different dataset and a Instantiations (4x)
including design task: datasets
individual 1) generate different ideas to task sheets
coffee break physicalize the dataset sketching- and
use the Image Schema crafting-material
Instantiations for inspiration
2) choose one idea
3) build a data physicalisation
1630 pm to 1700 pm presentation each group presents their final
physicalisation
1700 pm to 1745 pm reflection reflection on the design process reflection cards
(differences, impact of the Image
Schema Instantiations) and
outcome (differences, incorporated
image schemas)
1745 pm to 1800 pm sum up and Reflection on the Image Schema reflection cards
closing Instantiations (usefulness,
improvement, and further
development)
4. Post-Workshop-Plans
We plan to analyse the participants reflections, how they perceived the design process, regarding
flow and task load, physical and social engagement, and their reflections regarding the created
physicalisations. Further, we analyse and compare the data physicalisations created by the workshop
participants with and without the Image Schema Instantiations, regarding creativity, intuitive use,
graspability, interactivity and the sensory modalities they address. We also evaluate the collected
reflections and suggestions to develop the Image Schema Instantiations further and write a workshop
report.
5. Material
Table 2
Material
to be provided we bring with us
room laptop with presentation
five tables 1-4 sets of Image Schema Instantiations
chairs (of number of participants & organizers) Instructions, datasets, reflection, cards, …
projector sketching- and crafting-material (paper, glue,
wire scissors, tape, …)
6. Number of Participants
To enable group building and provide exchange between the groups a minimum number of four
participants is required. In order to ensure each group can be supervised and everyone stays involved
in the design process we suggest a maximum number of four groups with four participants per group.
So, a maximum of 16 participants is suggested.
7. Implementation
We conducted the workshop with 14 participants coming from different backgrounds, ranging from
computer science to design. Each of the four groups created a physical data representation of CO2
emission data in the first design session. After the lunch break participants created data physicalisations
representing internet usage data. These physical representations incorporated the image schemas
SMOOTH-ROUGH, HEAVY-LIGHT, STRAIGHT-CROOKED (2x), LINKAGE (2x) and UP-DOWN. Further, the
image schema PAINFUL was used for inspiration two times.
Figure 2: Work in progress and final data physicalisations, representing internet usage data.
In the reflection session one group described the second design process as more linear - being
quicker, while creating more ideas. Some participants described the ideas created in the second design
process as more abstract, as well as less interactive and less multimodal. As reason for this the higher
complexity of the second data set was suggested. All participants experienced this data set as more
difficult to physicalize. Further, participants suggested to choose the Image Schema Instantiations by
their own, instead of providing a pre-defined selection of seven Image Schema Instantiations.
8. References
[1] Jansen, Y., Dragicevic, P., Isenberg, P., Alexander, J., Karnik, A., Kildal, J., Subramanian, S. and
Hornbæk, K. Opportunities and Challenges for Data Physicalization. In Proceedings of the 33rd
Annual ACM Conference on Human Factors in Computing Systems (CHI'15). Association for
Computing Machinery, Seoul, Republic of Korea, 2015.
[2] Wang, Y., Segal, A., Klatzky, R., Keefe, D., Isenberg, P., Hurtienne, J., Hornecker, E., Dwyer, T.,
Barrass, S. and Rhyne, T.-M. An Emotional Response to the Value of Visualization. IEEE
computer graphics and applications, 39 (09/01 2019), 8-17.
[3] Hogan, T. and Hornecker, E. Towards a Design Space for Multisensory Data Representation.
Interacting with Computers, 29, 2 (2016), 147-167.
[4] Johnson, M. The body in the mind: The bodily basis of meaning, imagination, and reason.
University of Chicago Press, Chicago, IL, US, 1987.
[5] Hurtienne, J. and Blessing, L. Design for intuitive use - Testing image schema theory for user
interface design. In Proceedings of the 16th International Conference on Engineering Design
(ICED 2007), Paris, France, 2007.
[6] Lund, A. Massification of the Intangible : An investigation into embodied meaning and information
visualization. Print & Media ,Umea University, 2003.
[7] Antle, A., Droumeva, M. and Corness, G. Playing with the sound maker: do embodied metaphors
help children learn?, In Proceedings of the 7th international conference on Interaction design and
children (IDC'08). Association for Computing Machinery, New York, NY, USA, 2008.
[8] Antle, A. N., Corness, G. and Droumeva, M. What the body knows: Exploring the benefits of
embodied metaphors in hybrid physical digital environments. Interacting with Computers, 21, 1–
2 (2009), 66–75.
[9] Van Rompay, T., Hekkert, P., Saakes, D. and Russo, B. Grounding abstract object characteristics
in embodied interactions. Acta psychologica, 119 (08/01 2005), 315-351.
[10] Wilkie, K., Holland, S. and Mulholland, P. Evaluating Musical Software Using Conceptual
Metaphors, In Proceedings of HCI 2009 - People and Computers XXIII, Cambridge, UK, 2009.
[11] Hurtienne, J. and Israel, J. H. Image schemas and their metaphorical extensions: intuitive patterns
for tangible interaction. In Proceedings of the 1st international conference on Tangible and
embedded interaction. Association for Computing Machinery, Baton Rrouge, Lousiana, 2007.
[12] Hurtienne, J., Klöckner, K., Diefenbach, S., Nass, C. and Maier, A. Designing with Image
Schemas: Resolving the Tension Between Innovation, Inclusion and Intuitive Use. Interacting with
Computers, 27, 3 (2015), 235-255.
[13] Hurtienne, J. Image Schemas and Design for Intuitive Use. Doctoral Thesis, Technische
Universität Berlin, Berlin, 2011.
[14] Huron, S., Gourlet, P., Hinrichs, U., Hogan, T. and Jansen, Y. Let's Get Physical: Promoting Data
Physicalization in Workshop Formats. In Proceedings of the 2017 Conference on Designing
Interactive Systems. Association for Computing Machinery, Edinburgh, United Kingdom, 2017.
[15] Fallman, D. Design-oriented human-computer interaction. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems. Association for Computing Machinery, Ft.
Lauderdale, Florida, USA, 2003.
[16] Baur, C., Wienrich, C. and Hurtienne, J. Form Follows Mental Models: Finding Instantiations of
Image Schemas using a Design Research Approach. In Proceedings of the Designing Interactive
Systems Conference (DIS'22). Association for Computing Machinery, Virtual Event, 2022.
[17] Hurtienne, J., Huber, S. and Baur, C. Supporting user interface design with image schmeas: The
ISCAT database as a research tool. CEUR Workshop Proceedngs (EUR-WS.org), Jönköping,
Sweden, 2022.