We present JogChalker, a system that allows recreational runners to capture their affective experience while running using touch gestures. Using a small set of simple gestures, a runner can record affect while running without looking at a screen or entering into a multi-step interaction. Gestures are recognized, but also recorded at high fidelity, as we believe how the gesture is made may itself be expressive and useful for runners to review. We present our initial prototype, describe the goals, structure, and outcomes of a four-week participatory design session, and discuss the consequent capture and visualization implications for JogChalker that we are currently exploring. JogChalker provides new opportunities for self-tracking affective experience during running and for helping runners recall and interpret their runs. Gesture; visualization; design; running; emotion.
Running is a physical activity enjoyed by many. It has low barriers to participation, and for most, running is an active pastime rather than a competitive sport. However not every run is as enjoyable as the next, and personal preferences for runners vary: weather conditions, location, terrain, music, time of day, etc. Popular mobile applications such as Runkeeper, Runtastic, and Endomondo track running data and let runners visualize and share their runs. Aside from freeform annotation at the end of a run, such applications don’t currently provide a means of capturing the affective experience of a run. Consequently, their visualization interfaces emphasize physical performance over the qualitative but critically important notion of enjoyment. Without a means of capturing emotion or affective experience during runs, runners don’t have a way of tracking and identifying patterns that correlate with a positive running experience. Such a feature would enable runners to better choose the time, place, and circumstances of their leisure runs. Manual tracking tools like Moodmap [1] and Emotion Map [2] allow users to tag locations and times with emotions, and present these on a map. Typical widget-based interfaces can be difficult or impossible to use when physically active, however [3]. In this paper we present JogChalker, a system that allows recreational runners to capture their affective experience while running using touch gestures.
The current prototype consists of a capture application and a visualization tool. The capture application is written in Java and runs on Android devices (Figure 1(a)), and provides a full screen gesture capture interface. A yellow trace line shows a gesture as it is being made, and it is recorded in real time. A standard Android gesture recognition library classifies the gesture once completed. In addition to the time and location in which a gesture is made, we record the traversal of the drawn gesture in terms of elapsed time and screen coordinates, as well as the width of the touch area and the device pressure (if supported by the hardware) throughout the gesture. Using this we generate an SVG animation so that the gesture can be replayed on the visualization interface. From the low level data a number of higher-level attributes of the gesture can be determined, including repetition, total area, average speed, and total time taken. After initial testing we identified five candidate running-related emotional states to support (bored, tired, mellow, euphoric, exhilarated), and developed simple candidate gestures for evaluation (Figure 1(b)). These were initial gesture sets, we chose gestures as we didn't want to focus on gesture elicitation during the design process, but rather the design of the mobile interface and visualization dashboard.
(a) (b) (c) JogChalker’s visualization tool is written in JavaScript, and was built using Mapbox Studio (Figure 1(c)). Currently the tool displays a single running route (obtained using a manual export from a running tracker on the mobile device). A list of the gestural annotations made during the run is provided, and these are also marked on the route itself using teardrop markers, above which are the SVG gesture images.
Informal testing of the initial JogChalker prototype generated many questions, including: Is it comfortable to make gestures while running? Is making a gesture emotionally expressive? Will runners use the candidate states and gestures? How should affective experience be visualized and queried after a run, or after many runs? How could JogChalker be integrated with existing running data capture tools? We wanted to further develop the prototype for a field study to explore some of these questions. We employed a participatory design approach with recreational runners. After pilot testing with two lab colleagues we arrived at the methodology summarized here. We recruited 4 recreational runners (one female and three male, age 25-35), who each participated in 4 design sessions distributed over a 4 week period. Each session was divided into two parts – capture interface design and visualization interface design. After first receiving training on making the 5 gestures (for bored, mellow, tired, euphoric, exhilarated), participants ran for 30-60 mins using the capture tool prior to each session. Participants were asked simply to run a familiar route. They used an Android smartphone with a pressure-sensitive screen worn on an armband, and a Mio heart rate wristband. Gestures were displayed on the side of the armband for quick reference (Figure 1(b)). Since participants were not used to recording emotions while running, the mobile device would vibrate if no gesture was recorded over a 10 minute interval; otherwise participants were not prompted to record gestures. The Runkeeper application was also launched on the phone, and we preloaded the phone with a personal playlist if they preferred to listen to music while running. Participants also wore a GoPro camera while running. This was to generate a video stream that we provided as a potential element to include in the visualization interface, and to get a record of whether they slowed down or stopped, and whether they looked at the screen when making a gesture.
The first two PD sessions were done individually. In session 1 participants sketched potential modifications to the capture application using pen, paper, and post-its (Figure 2). They were then shown the visualization prototype, Runkeeper’s visualization interface, and the GoPro video feed. They were provided with pen, paper, and a set of paper widgets (including elements from the two visualizations and others not presented on either visualization including video, music, and weather data) and sketched a single visualization interface that would integrate the captured gestures with other data they deemed relevant for visualizing their experience (Figure 2). In session 2, participants used the same tools, to envision how to integrate gesture capture into Runkeeper, and work on their visualization design after viewing those made by the other runners (Figure 3).
The last two sessions were conducted as a group. In Session 3, the group presented and discussed each member’s designs, then worked to create a single integrated gesture capture design, and a single affective experience visualization interface. We additionally gave them some scenarios to consider when refining their design (e.g., running on a rainy day, running in a crowded area). Session 4 was conducted in the style of a Future Technology Workshop [4]. The group brainstormed about alternative methods that recreational runners could use to capture and visualize affective experience.
Recorded emotions varied; 1 participant drew gestures for mellow, exhilarated and bored, 3 others used tired and mellow mainly. We annotated designs and identified themes that emerged in the designs and participant comments across the 4 design sessions. Due to space constraints we briefly discuss some highlights here. Participants liked that gesture capture was automatic. They found recording gestures tricky when in full run, but didn’t mind slowing down to do so. All participants wanted to define their own gestures, and found it difficult to distinguish between euphoric and exhilarated. They all wanted to be able to record voice annotations, instead of or in addition to gestures. Integration with Runkeeper was refined toward a simple interface to enter recording modes, and a screen for reviewing and deleting annotations (see Figure 4) on the mobile. The group also suggested that recording an emotion could immediately trigger a change in music playlist. Our participants did not mention discomfort with the armband but did discuss using a smart watch as an alternative.
Visualization interface designs maintained a simple map-based run plot; most debate centered around whether data other than route and gesture location should always be visible or only after a selection interaction. When a gesture location is selected in the group design, a synchronized video stream would play the corresponding segment, and biometric data, music, weather, and the gesture itself would be displayed in a popup. (see Figure 5) Despite prompting, the notion of visualizing long term data patterns was not explored in detail by the group.
Despite some difficulties, our participants were satisfied with gesture as a means of recording affective experience when running, although they all felt that options for audio and custom gesture should be available. It is important to note that participants did not use audio annotations during the runs, and it may have its own issues (background noise, feeling awkward, breathlessness). Our PD approach may have limited novelty and variety in the visualization interface; participants were primed by the initial prototype and Runkeeper’s visualization, and the final result was a fairly straightforward “mashup” of the 2 interfaces. Showing the sketches of other users did encourage participants to think about their decisions, however the designs were very similar to begin with.
Our PD methodology also did not allow us to explore more nuanced aspects of gestural affect capture and visualization, including whether animating gestures supports inference and recall of affective experience, and whether and how long-term use of the interface supports discovery of running patterns leading to enjoyment. Our future work will explore both questions.
We are further investigating how collecting and visualizing affective experience alongside traditional running biometric and geospatial data can be used to generate richer insights into what can influence a runner’s performance. One focus of this work is to determine effective visual representations of gesture data, including an assessment of whether an individual can interpret emotional intensity by viewing an animation showing how a gesture was made. We are also exploring how JogChalking might encourage richer, more subjective recollections of running experiences. Over the long term this may help runners to discover the running patterns which lead to enjoyment for them, and for supporting tools to provide recommendations based on this data.
[1] Angela Fessl, Verónica Rivera-Pelayo, Viktoria Pammer, and Simone Braun. 2012. Mood tracking in virtual meetings. In Proceedings of the 7th European conference on Technology Enhanced Learning (EC-TEL'12). Springer-Verlag, Berlin, Heidelberg, 377-382. [2] Yun Huang, Ying Tang, and Yang Wang. 2015. Emotion Map: A Location-based Mobile Social System for Improving Emotion Awareness and Regulation. In Proceedings of CSCW 2015. ACM, New York, NY, USA, 130-142. [4] Giasemi N. Vavoula, Mike Sharples, Paul D. Rudman. 2002. Developing the 'Future technology workshop' method. In Proceedings of the International Workshop on Interaction Design and Children (pp. 65–72). Eindhoven: The Netherlands