Personal Reflection as Creative Practice in Collaboration with Biosensing Machines Noura Howell Abstract University of California, Berkeley Personal reflection with biosensing machines, such as Berkeley, CA 94720, USA step counters or heart rate sensors, places human and noura@berkeley.edu machine into a collaborative relationship of creating and interpreting data. Biosensory data are at once intimate and interpersonal, embodied and abstracted, objective and subjective, posing unique challenges for data analysis for personal meaning making. My work explores collaborative human-machine interpretation with biosensing. I design material representations of biosensory data and study the interpretive relationships humans develop with these in daily life, such as color- changing clothing that responds to the skin conductance of the wearer. Different materials suggest different associations and interpretations, fostering different kinds of “interpretive relationships” than typical data analysis of a graph on a screen. Although personal reflection may not typically be construed as creative practice, I posit that important parallels exist and would like to explore these further by participating in this workshop. Author Keywords Biosensing; thermochromic; reflective design. ACM Classification Keywords Copyright © 2017 for this paper is held by the author(s). Proceedings of MICI 2017: CHI Workshop on Mixed-Initiative Creative H.5.m. Information interfaces and presentation (e.g., Interfaces. HCI): Miscellaneous. Biosensing & Personal Reflection explorations manifest in the mind, in social interactions, Biosensing is on the rise in daily life. For example, the and through creative expression. Some core aspects of Feel wristband monitors skin conductance, pulse, and personality or identity may be fixed, but personal temperature to track mood and give wellness advice growth is a lifelong creative process. [18], and Affectiva uses facial analysis to detect emotions [19]. Prevailing trends in biosensing promote Beyond personal reflection, other work has explored individualistic, algorithmically defined emotions, enrolling biosensing for creative expression, such as wellness, and self-improvement. My stance is that this accelerometers or heart rate sensors worn by dancers alone is far too limiting, and that biosensing designs that influence timbre or rhythm for musicians (e.g., should foreground the subjective, creative process of [8,14]), or installations that transform and combine personal meaning making in order to support a participants’ heart rate into interactive visualizations multiplicity of values, experiences, and interpretations. (e.g., [11]). Such projects begin to reconceptualize biosensing, emphasizing emergent co-created meaning Interfaces for personal reflection with biosensory data between human and machine. tend toward two extremes: At one end, a time series graph of a physiological signal with no added My work focuses on personal reflection with biosensing annotation frames the data as passively observed fact while drawing from the reconceptualizations described and provides no guidance for human sensemaking. At above. I design for collaborative interpretive the other end, some devices impose their own relationships between human and machine, where each interpretation such as “happiness” or “stress” (e.g., has some initiative in meaning making. As part of this, [18,20]), leaving little space for human-driven I try to reveal the initiative, agency, or decisions emotional meaning making. So, interfaces tend to give already being made by biosensing machines. Although the machine too little or too much initiative in the often framed as “facts” passively observed from the process of personal meaning making with biosensing. natural environment, biosensory data are co- constructed by our bodies in contact with a sensor, I believe mixed initiative creative interfaces and algorithms in the hardware, and the manner in which personal reflection with biosensing have some they are displayed. The human efforts and inescapably interesting parallels, and that participating in this political decisions of engineers, scientists, and Figure 1: Color-changing fabric workshop could greatly inform my work. Creative designers went into creating an electrical signal, weaves and crochet to display practice can be part of meaning making, self deciding it was worthy of study, and presenting it. information. Fabric displays expression, exploration, or contribute to one’s sense of Revealing some of this embedded machine initiative evoke different associations and identity, and so too can personal reflection. Rather than may open up more of a dialogue in which humans feel interpretive relationships (from viewing personal reflection as some deterministic they can question, challenge, and more freely interpret prior work [1]). process of seeking one’s single true self, I see it more biosensory data. as an ongoing process of exploring and creating multiple possible aspects of one’s identity. These My work mixes machine, human, and material The next piece [7] explored the social meaning of initiatives for personal reflection, engaging the physical clothing-based biosensory representations. I developed properties of dynamic materials, such as Hint, a t-shirt whose thermochromic screenprint pattern thermochromic fabrics [2,9,10,12,13], to create responds to the wearer’s skin conductance (Fig. 2), and volatile, unstable biosensing displays. By evoking studied how pairs of friends, each wearing Hint shirts, associations with fashion and textiles, color-changing conversed and interpreted the display. I found that in fabric displays may seem less like objective “fact” than this context biosensing display became part of social a traditional time series graph on a screen, which may performance [5]. What if wearers crafted their display encourage users to take more initiative in forming to support their intended performance? What role might interpretations. The thermochromic fabric itself also the biosensing display play in social interaction? takes its own initiative, responding to environmental temperature as well as biosensors, and changing on its With Ripple, I adapted and extended the design of Hint own slower time scale of a few minutes rather than to be robust and comfortable enough for daily life, and milliseconds. This fosters interactions that are slower studied wearers’ interpretations with the display over Figure 2: Shirt that senses and displays skin conductance. Small [6], leveraging ambiguity [4] to foster reflection and the course of two days of going about their daily lives. white rectangles gradually appear open ended interpretation [15,16]. In the following From making music in the studio with friends, to work, when the wearer’s skin sections, I describe in more detail my work with color- to socializing at a bar, participants interpreted the conductance spikes, an indication changing fabric biosensing displays. display in a wide variety of contexts. Participants began of excitement such as stress or to build an interpretive relationship with their display, happiness (from prior work [7]). Personal Reflection with Clothing-Based relating it to aspects of their personality, events in their Overall, my designs foster Biosensing Displays day, or their emotions [currently under review]. interactions in which human, With Ebb [1] in collaboration with Project Jacquard machine, and material initiatives are significantly different than in [13], we explored associations around color-changing As a third year PhD student (pre-qualifying exams), I more typical consumer devices fabrics and information display in the context of fashion see my work thus far as one initial foray into personal for personal reflection with and personal style. Through material explorations reflection with biosensing, with still many open biosensing. Rather than a time [3,17], we developed thermochromic color-changing questions and improvements to be made. For future series graph or discrete fabrics (Fig. 1). We engaged fashion designers and work, I am exploring other material representations, emotional categories, the machine’s display is a more everyday wearers in envisioning what role these fabrics more socially situated meaning making, and rethinking ambiguous abstract pattern and might play in their design practice and sense of style. the relationship between human, biosensor, and more open to human initiative in Participants expressed appreciation for the slowness display. I think framing biosensory data, mediated by interpretation. Rather than the and low resolution of this material, likening it more to a its material display, as having its own initiative or voice hard fast-switching material of canvas than a screen, and envisioned slow, gentle, rather than just as passively observed facts, can foster electronic screens, the softer slower thermochromic fabrics subtle displays. We asked, how might these displays interactions that mix human and machine agency in evoke different associations. shift our interactions with information? creative and personal growth with biosensing. References 8. Teoma Jackson Naccarato and John MacCallum. 1. Laura Devendorf, Joanne Lo, Noura Howell, Lin Lee 2016. From representation to relationality: Bodies, Jung, Nan-Wei Gong, M. Emre Karagozler, Shiho biosensors and mediated environments. Journal of Fukuhara, Ivan Poupyrev, Eric Paulos, and Kimiko Dance & Somatic Practices 8, 1: 57–72. Ryokai. 2016. “I don’t want to wear a screen”: http://doi.org/10.1386/jdsp.8.1.57_1 Probing perceptions of and possibilities for dynamic 9. Linnéa Nilsson, Mika Satomi, Anna Vallgårda, and displays on clothing. Proceedings of the 34th Linda Worbin. 2011. Understanding the Complexity Annual ACM Conference on Human Factors in of Designing Dynamic Textile Patterns. Ambience. Computing Systems (CHI’16). 10. Maggie Orth. 2009. 100 Electronic Art Years. 2. Delia Dumitrescu, Marjan Kooroshnia, and Hanna Retrieved from Landin. 2014. Exploring the relation between time- http://www.maggieorth.com/art_100EAYears.html based textile patterns and digital environments. 11. Rakhi Parekh. 2014. The Ripple. Retrieved from Ambience. Retrieved March 12, 2016 from http://www.rakhiparekh.me/work-08.html http://hdl.handle.net/2320/14141 12. E. R. Post, M. Orth, P. R. Russo, and N. 3. Ylva Fernaeus and Petra Sundström. 2012. The Gershenfeld. 2000. E-broidery: Design and Material Move How Materials Matter in Interaction fabrication of textile-based computing. IBM Design Research. Proceedings of the Designing Systems Journal 39, 3.4: 840–860. Interactive Systems Conference, ACM, 486–495. http://doi.org/10.1147/sj.393.0840 http://doi.org/10.1145/2317956.2318029 13. Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, M. 4. William W. Gaver, Jacob Beaver, and Steve Emre Karagozler, Carsten Schwesig, and Karen Benford. 2003. Ambiguity As a Resource for Robinson. 2016. Project Jacquard: Manufacturing Design. Proceedings of the SIGCHI Conference on Digital Textiles at Scale. Proceedings of the 34th Human Factors in Computing Systems, ACM, 233– Annual ACM Conference on Human Factors in 240. http://doi.org/10.1145/642611.642653 Computing Systems. 5. Erving Goffman. 1959. The presentation of self in 14. Scott Rubin and Thin Edge New Music Collective. everyday life. Anchor Books, New York. 2016. Naked to the Sky. Retrieved from 6. Lars Hallnäs and Johan Redström. 2001. Slow https://www.youtube.com/watch?v=yeIptvATmE0 Technology – Designing for Reflection. Personal 15. Phoebe Sengers, Kirsten Boehner, Shay David, and Ubiquitous Comput. 5, 3: 201–212. Joseph “Jofish” Kaye. 2005. Reflective Design. http://doi.org/10.1007/PL00000019 Proceedings of the 4th Decennial Conference on 7. Noura Howell, Laura Devendorf, Rundong (Kevin) Critical Computing: Between Sense and Sensibility, Tian, Tomás Vega, Nan-Wei Gong, Ivan Poupyrev, ACM, 49–58. Eric Paulos, and Kimiko Ryokai. 2016. Biosignals as http://doi.org/10.1145/1094562.1094569 social cues: Ambiguity and emotional interpretation 16. Phoebe Sengers and Bill Gaver. 2006. Staying in social displays of skin conductance. Designing Open to Interpretation: Engaging Multiple Interactive Systems (DIS). Meanings in Design and Evaluation. Proceedings of the 6th Conference on Designing Interactive Systems (DIS’06), ACM, 99–108. http://doi.org/10.1145/1142405.1142422 17. Anna Vallgårda and Johan Redström. 2007. Computational Composites. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 513–522. http://doi.org/10.1145/1240624.1240706 18. Feel. Retrieved January 12, 2016 from http://www.myfeel.co/ 19. Affectiva. Retrieved from http://www.affectiva.com/ 20. Spire. Spire. Retrieved May 24, 2016 from http://www.spire.io