=Paper=
{{Paper
|id=Vol-1907/5_mici_howell
|storemode=property
|title=Personal Reflection as Creative Practice in Collaboration with Biosensing Machines
|pdfUrl=https://ceur-ws.org/Vol-1907/5_mici_howell.pdf
|volume=Vol-1907
|authors=Noura Howell
|dblpUrl=https://dblp.org/rec/conf/chi/Howell17
}}
==Personal Reflection as Creative Practice in Collaboration with Biosensing Machines==
https://ceur-ws.org/Vol-1907/5_mici_howell.pdf
Personal Reflection as Creative
Practice in Collaboration with
Biosensing Machines
Noura Howell Abstract
University of California, Berkeley Personal reflection with biosensing machines, such as
Berkeley, CA 94720, USA step counters or heart rate sensors, places human and
noura@berkeley.edu machine into a collaborative relationship of creating
and interpreting data. Biosensory data are at once
intimate and interpersonal, embodied and abstracted,
objective and subjective, posing unique challenges for
data analysis for personal meaning making. My work
explores collaborative human-machine interpretation
with biosensing. I design material representations of
biosensory data and study the interpretive relationships
humans develop with these in daily life, such as color-
changing clothing that responds to the skin
conductance of the wearer. Different materials suggest
different associations and interpretations, fostering
different kinds of “interpretive relationships” than
typical data analysis of a graph on a screen. Although
personal reflection may not typically be construed as
creative practice, I posit that important parallels exist
and would like to explore these further by participating
in this workshop.
Author Keywords
Biosensing; thermochromic; reflective design.
ACM Classification Keywords
Copyright © 2017 for this paper is held by the author(s).
Proceedings of MICI 2017: CHI Workshop on Mixed-Initiative Creative H.5.m. Information interfaces and presentation (e.g.,
Interfaces. HCI): Miscellaneous.
Biosensing & Personal Reflection explorations manifest in the mind, in social interactions,
Biosensing is on the rise in daily life. For example, the and through creative expression. Some core aspects of
Feel wristband monitors skin conductance, pulse, and personality or identity may be fixed, but personal
temperature to track mood and give wellness advice growth is a lifelong creative process.
[18], and Affectiva uses facial analysis to detect
emotions [19]. Prevailing trends in biosensing promote Beyond personal reflection, other work has explored
individualistic, algorithmically defined emotions, enrolling biosensing for creative expression, such as
wellness, and self-improvement. My stance is that this accelerometers or heart rate sensors worn by dancers
alone is far too limiting, and that biosensing designs that influence timbre or rhythm for musicians (e.g.,
should foreground the subjective, creative process of [8,14]), or installations that transform and combine
personal meaning making in order to support a participants’ heart rate into interactive visualizations
multiplicity of values, experiences, and interpretations. (e.g., [11]). Such projects begin to reconceptualize
biosensing, emphasizing emergent co-created meaning
Interfaces for personal reflection with biosensory data between human and machine.
tend toward two extremes: At one end, a time series
graph of a physiological signal with no added My work focuses on personal reflection with biosensing
annotation frames the data as passively observed fact while drawing from the reconceptualizations described
and provides no guidance for human sensemaking. At above. I design for collaborative interpretive
the other end, some devices impose their own relationships between human and machine, where each
interpretation such as “happiness” or “stress” (e.g., has some initiative in meaning making. As part of this,
[18,20]), leaving little space for human-driven I try to reveal the initiative, agency, or decisions
emotional meaning making. So, interfaces tend to give already being made by biosensing machines. Although
the machine too little or too much initiative in the often framed as “facts” passively observed from the
process of personal meaning making with biosensing. natural environment, biosensory data are co-
constructed by our bodies in contact with a sensor,
I believe mixed initiative creative interfaces and algorithms in the hardware, and the manner in which
personal reflection with biosensing have some they are displayed. The human efforts and inescapably
interesting parallels, and that participating in this political decisions of engineers, scientists, and
Figure 1: Color-changing fabric workshop could greatly inform my work. Creative designers went into creating an electrical signal,
weaves and crochet to display practice can be part of meaning making, self deciding it was worthy of study, and presenting it.
information. Fabric displays expression, exploration, or contribute to one’s sense of Revealing some of this embedded machine initiative
evoke different associations and identity, and so too can personal reflection. Rather than may open up more of a dialogue in which humans feel
interpretive relationships (from viewing personal reflection as some deterministic they can question, challenge, and more freely interpret
prior work [1]). process of seeking one’s single true self, I see it more biosensory data.
as an ongoing process of exploring and creating
multiple possible aspects of one’s identity. These
My work mixes machine, human, and material The next piece [7] explored the social meaning of
initiatives for personal reflection, engaging the physical clothing-based biosensory representations. I developed
properties of dynamic materials, such as Hint, a t-shirt whose thermochromic screenprint pattern
thermochromic fabrics [2,9,10,12,13], to create responds to the wearer’s skin conductance (Fig. 2), and
volatile, unstable biosensing displays. By evoking studied how pairs of friends, each wearing Hint shirts,
associations with fashion and textiles, color-changing conversed and interpreted the display. I found that in
fabric displays may seem less like objective “fact” than this context biosensing display became part of social
a traditional time series graph on a screen, which may performance [5]. What if wearers crafted their display
encourage users to take more initiative in forming to support their intended performance? What role might
interpretations. The thermochromic fabric itself also the biosensing display play in social interaction?
takes its own initiative, responding to environmental
temperature as well as biosensors, and changing on its With Ripple, I adapted and extended the design of Hint
own slower time scale of a few minutes rather than to be robust and comfortable enough for daily life, and
milliseconds. This fosters interactions that are slower studied wearers’ interpretations with the display over
Figure 2: Shirt that senses and
displays skin conductance. Small
[6], leveraging ambiguity [4] to foster reflection and the course of two days of going about their daily lives.
white rectangles gradually appear open ended interpretation [15,16]. In the following From making music in the studio with friends, to work,
when the wearer’s skin sections, I describe in more detail my work with color- to socializing at a bar, participants interpreted the
conductance spikes, an indication changing fabric biosensing displays. display in a wide variety of contexts. Participants began
of excitement such as stress or
to build an interpretive relationship with their display,
happiness (from prior work [7]).
Personal Reflection with Clothing-Based relating it to aspects of their personality, events in their
Overall, my designs foster Biosensing Displays day, or their emotions [currently under review].
interactions in which human,
With Ebb [1] in collaboration with Project Jacquard
machine, and material initiatives
are significantly different than in
[13], we explored associations around color-changing As a third year PhD student (pre-qualifying exams), I
more typical consumer devices fabrics and information display in the context of fashion see my work thus far as one initial foray into personal
for personal reflection with and personal style. Through material explorations reflection with biosensing, with still many open
biosensing. Rather than a time [3,17], we developed thermochromic color-changing questions and improvements to be made. For future
series graph or discrete
fabrics (Fig. 1). We engaged fashion designers and work, I am exploring other material representations,
emotional categories, the
machine’s display is a more
everyday wearers in envisioning what role these fabrics more socially situated meaning making, and rethinking
ambiguous abstract pattern and might play in their design practice and sense of style. the relationship between human, biosensor, and
more open to human initiative in Participants expressed appreciation for the slowness display. I think framing biosensory data, mediated by
interpretation. Rather than the and low resolution of this material, likening it more to a its material display, as having its own initiative or voice
hard fast-switching material of
canvas than a screen, and envisioned slow, gentle, rather than just as passively observed facts, can foster
electronic screens, the softer
slower thermochromic fabrics
subtle displays. We asked, how might these displays interactions that mix human and machine agency in
evoke different associations. shift our interactions with information? creative and personal growth with biosensing.
References 8. Teoma Jackson Naccarato and John MacCallum.
1. Laura Devendorf, Joanne Lo, Noura Howell, Lin Lee 2016. From representation to relationality: Bodies,
Jung, Nan-Wei Gong, M. Emre Karagozler, Shiho biosensors and mediated environments. Journal of
Fukuhara, Ivan Poupyrev, Eric Paulos, and Kimiko Dance & Somatic Practices 8, 1: 57–72.
Ryokai. 2016. “I don’t want to wear a screen”: http://doi.org/10.1386/jdsp.8.1.57_1
Probing perceptions of and possibilities for dynamic 9. Linnéa Nilsson, Mika Satomi, Anna Vallgårda, and
displays on clothing. Proceedings of the 34th Linda Worbin. 2011. Understanding the Complexity
Annual ACM Conference on Human Factors in of Designing Dynamic Textile Patterns. Ambience.
Computing Systems (CHI’16). 10. Maggie Orth. 2009. 100 Electronic Art Years.
2. Delia Dumitrescu, Marjan Kooroshnia, and Hanna Retrieved from
Landin. 2014. Exploring the relation between time- http://www.maggieorth.com/art_100EAYears.html
based textile patterns and digital environments. 11. Rakhi Parekh. 2014. The Ripple. Retrieved from
Ambience. Retrieved March 12, 2016 from http://www.rakhiparekh.me/work-08.html
http://hdl.handle.net/2320/14141 12. E. R. Post, M. Orth, P. R. Russo, and N.
3. Ylva Fernaeus and Petra Sundström. 2012. The Gershenfeld. 2000. E-broidery: Design and
Material Move How Materials Matter in Interaction fabrication of textile-based computing. IBM
Design Research. Proceedings of the Designing Systems Journal 39, 3.4: 840–860.
Interactive Systems Conference, ACM, 486–495. http://doi.org/10.1147/sj.393.0840
http://doi.org/10.1145/2317956.2318029 13. Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, M.
4. William W. Gaver, Jacob Beaver, and Steve Emre Karagozler, Carsten Schwesig, and Karen
Benford. 2003. Ambiguity As a Resource for Robinson. 2016. Project Jacquard: Manufacturing
Design. Proceedings of the SIGCHI Conference on Digital Textiles at Scale. Proceedings of the 34th
Human Factors in Computing Systems, ACM, 233– Annual ACM Conference on Human Factors in
240. http://doi.org/10.1145/642611.642653 Computing Systems.
5. Erving Goffman. 1959. The presentation of self in 14. Scott Rubin and Thin Edge New Music Collective.
everyday life. Anchor Books, New York. 2016. Naked to the Sky. Retrieved from
6. Lars Hallnäs and Johan Redström. 2001. Slow https://www.youtube.com/watch?v=yeIptvATmE0
Technology – Designing for Reflection. Personal 15. Phoebe Sengers, Kirsten Boehner, Shay David, and
Ubiquitous Comput. 5, 3: 201–212. Joseph “Jofish” Kaye. 2005. Reflective Design.
http://doi.org/10.1007/PL00000019 Proceedings of the 4th Decennial Conference on
7. Noura Howell, Laura Devendorf, Rundong (Kevin) Critical Computing: Between Sense and Sensibility,
Tian, Tomás Vega, Nan-Wei Gong, Ivan Poupyrev, ACM, 49–58.
Eric Paulos, and Kimiko Ryokai. 2016. Biosignals as http://doi.org/10.1145/1094562.1094569
social cues: Ambiguity and emotional interpretation 16. Phoebe Sengers and Bill Gaver. 2006. Staying
in social displays of skin conductance. Designing Open to Interpretation: Engaging Multiple
Interactive Systems (DIS). Meanings in Design and Evaluation. Proceedings of
the 6th Conference on Designing Interactive
Systems (DIS’06), ACM, 99–108.
http://doi.org/10.1145/1142405.1142422
17. Anna Vallgårda and Johan Redström. 2007.
Computational Composites. Proceedings of the
SIGCHI Conference on Human Factors in
Computing Systems, ACM, 513–522.
http://doi.org/10.1145/1240624.1240706
18. Feel. Retrieved January 12, 2016 from
http://www.myfeel.co/
19. Affectiva. Retrieved from
http://www.affectiva.com/
20. Spire. Spire. Retrieved May 24, 2016 from
http://www.spire.io