Designing Data Physicalisations - with Physical Image Schema Instantiations

Abstract

In an iterative design process, we created Image Schema Instantiations, an inspirational tool, to make image schema theory easily accessible for designers of data physicalisations, with the aim to foster creativity and intuitive mappings of abstract data to physical properties. The workshop provides Image Schema Instantiations as inspirational tool for a data physicalisation design session. It enables participants to gather experience in designing data physicalisations and to compare design sessions with and without Image Schema Instantiations. A structured reflection and feedback session helps to evaluate the impact of the Image Schema Instantiations. Aim of this workshop is to provide researchers the opportunity to utilize the Image Schema Instantiations directly in the design process and reflect on the usefulness of the tool. Data physicalisation, image schemas, design research

1. Background

Data physicalisations are physical objects which encode data by their geometry or material properties [ 1 ]. They aim to enhance cognition, communication, learning, problem solving and decisionmaking by mapping abstract data to physical and spatial properties [ 1, 2 ]. Even if they provide the opportunity to use different modalities, previous research found that they do not use their full potential to facilitate the embodied understanding of complex data [ 3 ].

This could be enhanced by incorporating mental building blocks, so called image schemas, in the design process of data physicalisations. Image schemas are abstract representations of recurring dynamic patterns of bodily interactions [ 4 ]. They help us to structure and understand the world by addressing mental models, like the image schemas UP-DOWN or CONTAINER, which we experience recurrently in our everyday life. For example, a glass of water contains both image schemas: UP-DOWN because of the rising and falling level of water when pouring water or drinking up, while the glass itself acts as CONTAINER, separating an outside from an inside by a border of glass.

Previous research revealed image schemas to be useful to analyse and evaluate user interfaces, to describe requirements and to support the user interface design process. Graphical and tangible user interfaces which contain designed instantiations of image schemas showed that image schema theory can work as design language and powerful tool offering insight and value [ 5-11 ]. Furthermore, image schemas, when instantiated in user interfaces, enable more inclusive, intuitive and innovative interaction [12]. These are also desirable properties for data physicalisations. As abstract

2022 Copyright for this paper by its authors. representations of sensorimotor experiences [ 4 ] image schemas are multimodal and able to represent visual, aural and tactile qualities as well as kinaesthetic experiences [13]. This enhances not only inclusiveness but is also promising for multisensory data physicalisations. They could support designers in finding appropriate mappings of abstract data to physical properties or address the challenges found in actual data physicalisations, like addressing primary vision, being static and passive [ 3 ]. Furthermore, image schemas’ abstractness leaves enough space for designers to decide how to instantiate them. This enables more innovative design ideas [12].

An analysis of current data physicalisations, the authors made in an earlier phase of this project [paper submitted], showed that a range of image schemas is already incorporated in data physicalisations and there is potential to use more and different image schemas to make physicalisations more expressive or easier to understand. To make image schemas accessible for designers and easy to incorporate in the design process of data physicalisations, we created physical instantiations of selected image schemas. These are meant to be part of a toolkit which will be utilized in the design process of data physicalisations, with the aim to foster the designer’s creativity and the user’s intuitive understanding of the data physicalisation. In comparison to other data physicalisation workshops our approach is not to provide tools which constrain the design space of data physicalisations [14], but to enhance creativity and intuitive design.

In an iterative research oriented design process [15] including several feedback sessions with different groups of participants (design experts, HCI experts and lay people) we went through the stages of visual representations, physical representations, and interactive representations [16]. For each image schema a final design solution for each stage was found, which then informed the following stage. For the final set of Image Schema Instantiations (for some excerpts see Figure: 1) we choose either the final interactive object or – when the interaction appeared to be hindering or confusing – the final physical object.

In total the toolkit contains instantiations of 14 image schemas: OBJECT, PAINFUL, LINKAGE, UPDOWN, LEFT-RIGHT, SMOOTH-ROUGH, STRONG-WEAK, HEAVY-LIGHT, STRAIGHT-CROOKED, HARDSOFT, CENTER-PERIPHERY, NEAR-FAR, CONTENT-CONTAINER, and PART-WHOLE. The latest evaluation showed that this number tends to overwhelm some designers, so we are going to provide only a subset of seven image schemas to each group.

The physical Image Schema Instantiations are promising for the design of data physicalisations because they are already closer to the target medium, i.e., closer to the ideal form to be used in the design process of data physicalisations. Further, they eliminate the need to cope with formal definitions like those provided by the ISCAT database [17] and make image schemas more easily accessible. Also, social engagement could be higher, because objects can be better explored and discussed in teams than bare textual definitions. The physical instantiations make it easy to compare image schemas with each other, to identify differences and to figure out which image schemas are the most appropriate for the current purpose. They can be seen as inspiring design suggestions how image schemas can be instantiated in one’s own data physicalisation project or can enhance creativity in a more abstract way, also fostering the unconscious implementation of image schemas. Furthermore, they are promising to enhance the (physical) engagement and to consider the effects of instantiating the image schema in a data physicalisation.

The format of a full-day workshop offers participants to gather experience in designing data physicalisations and to engage in depth with the Image Schema Instantiations. They experience the design process of data physicalisations with and without the Image Schema Instantiations and get the opportunity to compare both and reflect on the impact of the tool.

The first part of the workshop offers a brief introduction to the field of data physicalisation. This opens the workshop to everyone, regardless of prior knowledge or background. A hands-on experience, including brainstorming, conceptualizing, and building a data physicalisation in small teams offers active engagement with the topic and afterwards the opportunity to reflect on the own design action and the design outcome.

In the second part image schema theory and the Image Schema Instantiations are introduced. Now participants can explore the Image Schema Instantiations on their own and utilize them in the design process of data physicalisations in a second design session. Afterwards there is time to reflect on both design sessions, the differences as well as the impact of the Image Schema Instantiations.

During the design process of the Image Schema Instantiations, we already collected first insights about their usage in group settings. This workshop provides the opportunity to apply them in the design process of data physicalisations. Our aim is to gather feedback about the impact of the Image Schema Instantiations on the design process and to evaluate their usefulness. Further we want to create together with the workshop participants data physicalisations designed with and without Image Schema Instantiations, in order to analyse and compare them.

2. Organizers

2.1. Cordula Baur

is currently working toward the Ph.D. degree with the Chair of Psychological Ergonomics, University of Wuerzburg, Wuerzburg, Germany. Initially coming from a design background, she engaged in her master’s thesis with the topic of perception and the fields of sensory design as well as research through design. In her PhD she deals with the fields of data physicalisation and image schemas. For this aim she did an analysis of data physicalisations regarding image schemas and their potential [paper submitted] and created in an iterative design process the Image Schema Instantiations [16]. Further she also offers lectures to this topic. Her research interests include tangible interaction, data physicalisation, image schemas, sensory design, and design research. She is the contact person for this workshop. 2.2.

Carolin Wienrich

is currently a Professor for Psychology of Intelligent Interactive Systems with the University of Wuerzburg, Wuerzburg, Germany, and a Co-Leader of the XR HUB Wuerzburg. Her research interests focus on interaction paradigms between humans and digital entities as well as change experiences during and after digital interventions. 2.3.

Jörn Hurtienne

is currently a Full Professor and Chair Holder in Psychological Ergonomics with the University of Wuerzburg, Wuerzburg, Germany. He is interested in data physicalisation, tangible interaction, design for intuitive use, image schema theory and user experience.

3. Workshop Structure

4. Post-Workshop-Plans

each group presents their final physicalisation reflection on the design process (differences, impact of the Image Schema Instantiations) and outcome (differences, incorporated image schemas) Reflection on the Image Schema Instantiations (usefulness, improvement, and further development) reflection cards reflection cards

We plan to analyse the participants reflections, how they perceived the design process, regarding flow and task load, physical and social engagement, and their reflections regarding the created physicalisations. Further, we analyse and compare the data physicalisations created by the workshop participants with and without the Image Schema Instantiations, regarding creativity, intuitive use, graspability, interactivity and the sensory modalities they address. We also evaluate the collected reflections and suggestions to develop the Image Schema Instantiations further and write a workshop report.

5. Material

6. Number of Participants

To enable group building and provide exchange between the groups a minimum number of four participants is required. In order to ensure each group can be supervised and everyone stays involved in the design process we suggest a maximum number of four groups with four participants per group. So, a maximum of 16 participants is suggested.

7. Implementation

We conducted the workshop with 14 participants coming from different backgrounds, ranging from computer science to design. Each of the four groups created a physical data representation of CO2 emission data in the first design session. After the lunch break participants created data physicalisations representing internet usage data. These physical representations incorporated the image schemas SMOOTH-ROUGH, HEAVY-LIGHT, STRAIGHT-CROOKED (2x), LINKAGE (2x) and UP-DOWN. Further, the image schema PAINFUL was used for inspiration two times.

In the reflection session one group described the second design process as more linear - being quicker, while creating more ideas. Some participants described the ideas created in the second design process as more abstract, as well as less interactive and less multimodal. As reason for this the higher complexity of the second data set was suggested. All participants experienced this data set as more difficult to physicalize. Further, participants suggested to choose the Image Schema Instantiations by their own, instead of providing a pre-defined selection of seven Image Schema Instantiations.

8. References

[12] Hurtienne, J., Klöckner, K., Diefenbach, S., Nass, C. and Maier, A. Designing with Image Schemas: Resolving the Tension Between Innovation, Inclusion and Intuitive Use. Interacting with Computers, 27, 3 (2015), 235-255. [13] Hurtienne, J. Image Schemas and Design for Intuitive Use. Doctoral Thesis, Technische

Universität Berlin, Berlin, 2011. [14] Huron, S., Gourlet, P., Hinrichs, U., Hogan, T. and Jansen, Y. Let's Get Physical: Promoting Data Physicalization in Workshop Formats. In Proceedings of the 2017 Conference on Designing Interactive Systems. Association for Computing Machinery, Edinburgh, United Kingdom, 2017. [15] Fallman, D. Design-oriented human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Ft.

Lauderdale, Florida, USA, 2003. [16] Baur, C., Wienrich, C. and Hurtienne, J. Form Follows Mental Models: Finding Instantiations of Image Schemas using a Design Research Approach. In Proceedings of the Designing Interactive Systems Conference (DIS'22). Association for Computing Machinery, Virtual Event, 2022. [17] Hurtienne, J., Huber, S. and Baur, C. Supporting user interface design with image schmeas: The ISCAT database as a research tool. CEUR Workshop Proceedngs (EUR-WS.org), Jönköping, Sweden, 2022.

References

[1] Jansen , Y. , Dragicevic , P. , Isenberg , P. , Alexander , J. , Karnik , A. , Kildal , J. , Subramanian , S. and Hornbaek , K. Opportunities and Challenges for Data Physicalization . In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15) . Association for Computing Machinery , Seoul, Republic of Korea, 2015 .

[2] Wang , Y. , Segal , A. , Klatzky , R. , Keefe , D. , Isenberg , P. , Hurtienne , J. , Hornecker , E. , Dwyer , T. , Barrass , S. and Rhyne , T.-M. An Emotional Response to the Value of Visualization . IEEE computer graphics and applications , 39 ( 09 /01 2019), 8 - 17 .

[3] Hogan , T. and Hornecker , E. Towards a Design Space for Multisensory Data Representation . Interacting with Computers , 29 , 2 ( 2016 ), 147 - 167 .

[4] Johnson , M. The body in the mind: The bodily basis of meaning, imagination, and reason . University of Chicago Press, Chicago, IL, US, 1987 .

[5] Hurtienne , J. and Blessing , L. Design for intuitive use - Testing image schema theory for user interface design . In Proceedings of the 16th International Conference on Engineering Design (ICED 2007 ), Paris, France, 2007 .

[6] Lund , A. Massification of the Intangible : An investigation into embodied meaning and information visualization . Print & Media ,Umea University, 2003 .

[7] Antle , A. , Droumeva , M. and Corness , G. Playing with the sound maker: do embodied metaphors help children learn? , In Proceedings of the 7th international conference on Interaction design and children (IDC'08) . Association for Computing Machinery , New York, NY, USA, 2008 .

[8] Antle , A. N. , Corness , G. and Droumeva , M. What the body knows: Exploring the benefits of embodied metaphors in hybrid physical digital environments . Interacting with Computers , 21 , 1 - 2 ( 2009 ), 66 - 75 .

[9] Van Rompay , T. , Hekkert , P. , Saakes , D. and Russo , B. Grounding abstract object characteristics in embodied interactions . Acta psychologica , 119 ( 08 /01 2005), 315 - 351 .

[10] Wilkie , K. , Holland, S. and Mulholland , P. Evaluating Musical Software Using Conceptual Metaphors , In Proceedings of HCI 2009 - People and Computers XXIII , Cambridge, UK, 2009 .

[11] Hurtienne , J. and Israel , J. H. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction . In Proceedings of the 1st international conference on Tangible and embedded interaction. Association for Computing Machinery , Baton Rrouge, Lousiana, 2007 .