A Dynamic Representation of Physical Exercises on Inflatable Membranes : Making Walking Fun Again! Sujay Shalawadia , Eva Horneckera and Florian Echtlerb a Bauhaus Universität, Weimar, Germany b Aalborg University, Denmark Abstract Humans have been relying on multiple sensory channels to generate enriched meaning to reflect experiences. Imagine a tangible object that can change its texture and size based on the amount of walking done by users and engage their touch and vision sensory channels simultaneously for self reflection. This idea has been realised with a prototype that uses pneumatics on inflatable membranes to produce haptics and visual effects for users to experience their walking through step counts mapped on inflatables. An evaluation of the prototype is made to derive interpretations that could be a conceptual guideline for representing physical exercises using pneumatics. Keywords Personal Informatics, Pneumatics, Haptics 1. Introduction Over the last decade, smartphones have become ubiquitous through their effective multi func- tional features. This has resulted to an exponential rise in application software that support more than traditional information retrieval and productivity tasks. Amongst these applications, those supporting personal informatics (PI) has steadily seen a rise in users mostly in urban areas where obesity is on the rise due to their sedentary jobs and eating habits. PI refer to data features that represent the lifestyle of a person. The applications that support PI are mostly utilising commodity hardware of the smartphones such as IMU (Inertial Measurement Unit), GPS (Global Positioning System) and cameras to produce data that represent activities such as running, walking and sleeping. Wearable devices, mostly smart watches also contribute to generating data to represent physical exercises. However, generated data is mostly limited to screens which reduces the data perception to a single sensory channel of vision as seen in figure 1. Victor et al. [1] support this claim with ”picture under the glass effect” as an inclination of screen visualisations that are occluding the presence of other sensory channels for data perception. These representations can reduce personal connections to users due to the predominant reliance on the visual medium. PI data provides a powerful source for self reflection and by limiting data to mainly visualisations can also impede data perception of visually impaired people. Proceedings of ETIS 2020, November 16–20, 2020, Siena, Italy Envelope-Open sujay.shalawadi@uni-weimar.de (S. Shalawadi); eva.hornecker@uni-weimar.de (E. Hornecker); floech@cs.aau.dk (F. Echtler) © 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) Figure 1: Can this dashboard be understood easily? This is trying to show user step count. The idea about representing PI on physical objects is motivated towards providing an enjoy- able experience of data perception to users about their physical efforts[2]. Additionally, the direction of multi modal approach for data representation can engage differently abled users. A qualitative user study conducted on the prototype that represents step counts of users over 3 days on an inflatable membrane received a positive feedback towards self reflecting their day and the ability to physically feel their data. This is further discussed later in the paper. The important highlights made through this paper are : • A self reflection prototype representing the amount of walking done by users through inflatable objects using pneumatics, • A qualitative user study to understand user reactions towards PI data represented on physical objects, • Interpretations from observations made that can be incorporated in the next prototype and • A generalised design approach to represent physical activity patterns of users using pneumatics. 2. Background and Related Work The thought to consider data representations beyond a visual medium was inspired by the definition - a physical artifact whose geometry or material properties encode data provided by Jansen et al.[3] on data physicalisation (DP). In the context of PI, Khot et al. [4] have shown how human material interaction (HMI) can be intersected with DP and PI to engage new data perceptions through taste and touch. 3D printed physical objects with abstract shapes to show physical activity levels to engage the feeling of self created souvenirs is shown through sweat atoms [5]. But, 3D printed objects are not practical on the long run due to the amount of space needed to store them. Edible representations of different colored electrolyte drinks and printed chocolate induce a feeling of treating the body for the effort done [6] [7]. The perception of taste induced satisfaction when a brightly colored liquid or more chocolate was given to users while during a sedentary day when users received only water, they were motivated to exercise. These representations certainly show a stronger connection between users and their data. The edible representations require active maintenance which is not feasible on a routine basis. Betros et al. [8] have tried to bring about awareness about walking by embodying step counts to the amount of water a plant receives. The user study showed valuable insights to how the plant motivated users to stay active. Some users also developed personal connections to their plant by naming the plant, this also resulted in increased activity levels for users so they could see their plant healthy. The metaphor of using a living being to represent a user’s activity level introduces an emotional connection for the users and a plant can easily be integrated in a living environment. But, typically users are keen to compare their improvements on a daily basis which limits this representation to exclude short term improvements [9]. Additionally by using only a single data feature, the holistic view of a person’s health is limited to only a specific behaviour. We associate ourselves with smartphones that have become ubiquitous devices and are collecting data as step counts, number of floors climbed, distance covered and few other data features in the background making PI an integral part of our lives. Users are mostly unaware about such data due to their limited visualisations that make them non captivating for users. The above literature provides a sound platform to investigate designing prototypes that allow self reflection to users on such PI data features by engaging more than vision for stronger personal connection at taking suitable action for better well being. 3. Methodology An user centred process was used to create a prototype that can be tested for feedback on the relationship between the data and their users. The idea of choosing walking as the context for proof of concept of data representation was to involve non active users to become a part of the process and eventually understand if the inflatable membrane could motivate them to become more active and healthy. The following subsections describe the chronological steps taken to build and evaluate the prototype. 3.1. Choosing Inflatables During the preliminary research on choosing the design principle, the salient factors for a suitable physicalisation were the following : • Keeping the outcome of the physicalisation totally dependent on user data so that the representation can be a self created souvenir. • The idea of improving aesthetics for analysing data and keeping the sight pleasant and less chaotic compared to graphs. The idea of data mapping on inflatables was inspired by Gohlke et al.. [10]. This work shows the use of inflatable membranes as an input surface which prompted the idea of a vice versa representation to generate haptics for data perception. 3.2. Focus Group Discussion (FGD) In order to understand the actual relationship between users and their PI data, we conducted the FGD to understand the data features users associate with, their experience with existing smartphone applications and motivation to use these applications. The users participating were from different age groups, professions and had different levels of fitness. The common feature among the participants was their interest in self monitored data, the group had people who were experienced at collecting self monitored data and also novice people who recently started collecting data. The salient observations made were: • The most preferred data feature was distance covered. • Users were not very happy when their applications showed different distances for the same route on different days. • Users were also dissatisfied with notification burden prompting to use paid version. We shared our idea of representing the data through inflatables which made the users excited and were willing to try the prototype. 3.3. Implementation The focus of this section was to develop a prototype that can be appealing to users during the study. The size of the prototype should be considered for aesthetics and eventual integration into user living spaces. There are two parts in the implementation : the Hardware system that supports the pneumatic system and the material design that contains the pressurised air and generate haptics. The implementation was conducted in an iterative manner for both the parts. The hardware system was first tested with air pumps and then eventually settled with solenoid valves connected to an aquarium air pump for less noisy and consistent air flow. The material design began with using stretchable cloth over TPU (Thermoplastic polyurethane) which is the inflatable membrane. Stitches were made on the cloth that were expected to expand and contract based on the inflatable and generate change in texture for users. However, during the pilot study the outcome was not satisfactory due to the negligible change in texture when compressed and decompressed. We also tried variations of texture embedded directly on the inflatable membrane but that compromised the size for expansion. We eventually arrived at using a stretchable cloth over the TPU material with silicon balls in between the outer cloth and inflatable material as shown in figure 2. Figure 2: Inflatable Design This idea was inspired by air occupying space, the compressed air expands into the space between cloth and the inflatable membrane to produce a stiff texture of the silicon balls and vice versa when deflated. The number of steps were converted to the amount of time in seconds that the solenoid valves need to be opened to fill the inflatables. A user interaction is shown in 3. Figure 3: Three inflatables showing step counts of three different days 4. Evaluation Through the evaluation, we wanted to find answers to two important questions : • Can the data be comprehended by the users? • What do the different physical states of the inflatable relate to the user’s effort? For the study, 15 users were considered. 8 of them were already a part of the FGD. The mean age of the users was 27 years with a standard deviation of 3.6 years. The data represented on the inflatable was the step counts over 3 days represented on three different inflatables placed next to each other. The data for 7 participants were extracted using API (Application Programming Interface) from a fitness tracker application called Strava. While the data for 8 participants were manually hardcoded due to their discomfort in making their profiles in Strava public to use API. The inflatables were 3 cms in radius and could easily be grasped by an average sized human palm. The calibrations on the inflatables were hard-coded during the pilot study and was conducted in a laboratory setup. 4.1. Discussions and Interpretations During the prototype evaluation, a semi structured interview was conducted with users to understand their opinion on data interpretation, context of use and real time usage scenario. The salient observations are that, • Users associated the haptic feedback of the inflatable to a stress ball. The stiffness of the silicon balls as a result of more distance covered reminded them of lean and firm muscles. Users realised the silicon balls created a therapeutic feeling. • 9 users felt using the inflatable for self reflection in personal space while 6 users felt that it would be better having this set up in public space like gym. • Users preferred an inflatable representing their goal steps and another representing their step count to increase their motivation. Users also felt that if they could compare their step counts with their family, it could encourage a healthy competition. • 2 users were active in sports but due to injuries were unable to exercise. They were pleased to see their efforts of walking being represented on the inflatable as a positive feeling and were motivated to become fit again through gradually increasing their step counts. Based on the observations, the user behaviours can be interpreted as: • Self Reflection Tool : Users felt an immediate satisfaction to see fully compressed inflatable and were able to quickly decide if they have walked enough. • Healthy Competition : Users were keen to chase their own goals and would like to compare the size of the inflatables within family to engage an active family lifestyle. • Data Privacy : Majority of the participants were not keen to share their secret key for their Strava account and make their profile public to use API. This raises future directions of non intrusive data acquisition. • Haptic Feedback : The use of fabric prompted most users to touch the inflatables. The silicon balls engaged the users to grip and grasp the inflatables. 5. Results and Future Work The dynamic states of inflatable membranes such as size, stiffness and texture allow mapping of three data features of PI. But with our prototype we have utilised only one design dimension of stiffness to encourage haptics. This opens up discussions of extending the dynamic states to a generalised design of representing user lifestyles by scaling this design to other contexts of PI such as running, sleeping and food intake. Additional data features such as heart rate can be introduced through other forms of haptics such as vibrations to show intensity of activities. There can also be a consideration for showing inactivity by reducing the size and stiffness of the inflatable and keep the representation honest for users to take actions for their well being. Based on the feedback, the design of inflatables can be modified to show goal vs actual effort to support user motivation to stay fit. The user study can be further extended to a longer duration in user’s familiar setup. The data collection can also be further developed to support data privacy and non intrusive data acquistion by using self built pedometer devices that could also collect more data features to improve personalising of data to its users. For the context of correlating step counts to inflatable membranes, the user study provided sufficient information to interpret a successful acceptance. Users were mostly excited to touch their data and the inference to take action to walk more was immediate. For data features of step counts and days, the users felt very comfortable comprehending their state of activity levels. Most users wish to have a similar system in their living space. We also believe this work provides a platform to engage discussions for future prototypes in the following ways: • Can the interpretations derived be used as general guidelines while developing physicali- sations for personal informatics ? • Can other sensory channels in addition to vision and touch be engaged using pneumatics? References [1] B. Victor, A brief rant on the future of interaction design, Worry Dream. com (2011). [2] Y. Yang, H. Lee, C. Gurrin, Visualizing lifelog data for different interaction platforms, in: CHI’13 Extended Abstracts on Human Factors in Computing Systems, 2013, pp. 1785–1790. [3] Y. Jansen, P. Dragicevic, P. Isenberg, J. Alexander, A. Karnik, J. Kildal, S. Subramanian, K. Hornbæk, Opportunities and challenges for data physicalization, in: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015, pp. 3227–3236. [4] R. A. Khot, L. Hjorth, F. Mueller, Understanding material representations of physical activity, Melbourne, Australia: RMIT University 144 (2016). [5] R. A. Khot, F. F. Mueller, Sweat-atoms: turning physical exercise into physical objects, in: CHI’13 Extended Abstracts on Human Factors in Computing Systems, 2013, pp. 3075–3078. [6] R. A. Khot, R. Pennings, F. Mueller, Edipulse: supporting physical activity with chocolate printed messages, in: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, 2015, pp. 1391–1396. [7] R. A. Khot, J. Lee, D. Aggarwal, L. Hjorth, F. Mueller, Tastybeats: Designing palatable representations of physical activity, in: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015, pp. 2933–2942. [8] F. Botros, C. Perin, B. A. Aseniero, S. Carpendale, Go and grow: Mapping personal data to a living plant, in: Proceedings of the International Working Conference on Advanced Visual Interfaces, 2016, pp. 112–119. [9] J. Hattie, H. Timperley, The power of feedback, Review of educational research 77 (2007) 81–112. [10] K. Gohlke, E. Hornecker, W. Sattler, Pneumatibles: exploring soft robotic actuators for the design of user interfaces with pneumotactile feedback, in: Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, 2016, pp. 308–315.