Presented at RepliCHI2013. Copyright c 2013 for the individual papers by the papers authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors.
We report on two of our own studies, each of which has built on a laboratory based nding and explored if and how the e ects played out in everyday settings. In each, we found e ects that in some ways validated the prior lab studies, but each also pointed to very di erent implications for HCI than those which were suggested by the initial lab work. By lab-based work we mean here empirical studies that are tightly controlled and aim to uncover causal relationships.
We would like to use these examples to open a discussion
within RepliCHI on whether or not the transferring of such
lab ndings into the eld is a speci c type of replication
that is especially important for HCI research, as (i) we
generally do want our systems/ ndings to transfer into
eld settings; and (ii) it is plausible to expect similar
results to the ones in our studies when transferring other
lab e ects into the real-world. We expect that further
discussion of this topic could well complement the existing
\into the wild" literature in HCI that now focuses more on
open-ended, in-situ exploration (e.g., see [
The two studies reported below come from di erent projects, but each builds on a nding that was previously rigorously analysed in the lab and seemed to be potentially useful in HCI.
The rst study picked up on a psychology experiment by
Janssen et. al. [
In another study, we explored work done by Marci et. al.
in psychotherapy and psychophysiology [
Such a link could be of interest for HCI, e.g., as a novel indicator to embed in various a ective computing systems, creating systems to support teaching of empathy for psychotherapy students, workplace etc. However, the original research was based on a very speci c setting (therapy session) and participants with speci c skills (therapist with many years of training to become highly empathetic). As such, we were interested to test how robustly the observed e ects appear in the types of real-world settings that are of interest of HCI, but which are also often full of distractions and potential intervening variables that could not be controlled in real-world deployment.
We designed a study [10] in which pairs of friends discussed a topic of their choice in a public house during normal opening hours. The rationale was to test the robustness of the link in a setting that is more extreme (in terms of potential disruptions and intervening variables) than those needed for the potential applications. In other words, we argued that if the e ect is robust enough to appear in a busy pub and for pairs of friends talking about any topics of importance, it is then more likely to appear also in a therapy students class, workplace setting or other potential application contexts.
Our results followed a similar pattern as in the rst study:
we have seen results that are in line with the original
work, but the implications for HCI application of these
have changed. For example, when we focused on
interactions where participants were instructed to discuss
their topics naturally, then thirty-seconds long video
snapshots chosen purely on the basis of high synchrony
showed also more empathy related non-verbal behaviour
(as judged by independent raters). This ts with the prior
lab results. However, we also found high synchrony in a
condition where we asked one of the participants to ignore
the other, i.e., where then little empathy could be
expected. Such inconsistencies led us to suggest a
re-interpretation of skin conductance synchrony { seeing it
not only as an indicator of empathy, but potentially as a
more general indicator of \mutual reactivity" (i.e., that
people emotionally react to each other). Such reactivity
then just happens to correspond well to empathy in the
right contexts, such as therapy session or a discussion of
two friends about an issue important for one of them. We
were able able to further support this hypothesis through
other psychological literature such as [
To summarise, each of the two studies have shown that the expected e ects can appear also in an uncontrolled, real-world setting, and are thus potentially robust enough for HCI applications. However, and maybe more importantly, each also clari ed and better scoped the potential implications of the original nding for HCI.
Stepping away from the two examples here, it does seem
that, at least for results in psychology, transfer of e ects
from the lab to the eld is far from an obvious claim. For
example, Mitchell [
Can this be expected also of lab-based research in HCI?
To our knowledge, there is little literature on this within
HCI so far. It is also not discussed in the recent \into the
wild" literature, e.g., [
We think it would be interesting to discuss in more depth how this focus on lab-to- eld transfer of e ects di ers and complements the existing work on research \in the wild". One immediate di erence is the focus, i.e., whether a well-understood lab e ect is robust enough to also appear in more realistic (and thus messy) conditions. Among other things, this will probably also raise methodological questions, as the main aim of such work is to test if an e ect appears (thus pointing to more quantitative, experimental work), but in a setting where one cannot control many of the potentially intervening variables. See Oulavirta [8] for an initial discussion of similar topic in the context of Pervasive computing.
We intended to demonstrate that examining whether the results of lab studies appear robustly `in the wild' may be a speci c kind of replication research, and one that could be of signi cant bene t to the CHI community. Drawing on our two studies, we saw that while the core e ect did translate, the implications about how it might be used within HCI were changed markedly. We referenced additional literature in psychology suggesting that such results might also be expected for other lab-based ndings.
Petr Slovak is a researcher and PhD student at the HCI Group at Vienna University of Technology. Drawing on his background in both psychology and computer science, his research focuses on support for teaching of empathy in medical and therapeutic settings, with speci c interest in the use of biosensors.
Paul Tennent is a researcher at the Mixed Reality Lab at the University of Nottingham. He has worked on a number of systems designed to support the transformation of complex system log data into accountable, queryable objects that can be used in qualitative analysis. More recently he has turned to the analysis and representation of biodata with a particular focus on television and the public understanding of science.
Geraldine Fitzpatrick is Professor at Vienna University of Technology in Austria and heads the Institute of Design and Assessment of Technology. She is interested in how we design pervasive, tangible and ubiquitous technologies to t in with everyday contexts, with a particular interest in supporting social interaction and collaboration, and health and well being. Laboratory. Perspectives on Psychological Science 7, 2 (Mar. 2012), 109{117. [8] Oulasvirta, A. Rethinking Experimental Designs for Field Evaluations. IEEE Pervasive Computing 11, 4 (Oct. 2012), 60{67. [9] Slovak, P., Janssen, J., and Fitzpatrick, G.
Understanding heart rate sharing: towards unpacking physiosocial space. In CHI '12 (2012), 859{868. [10] Slovak, P., Tennent, P., and Fitzpatrick, G. Exploring physiological synchrony in everyday meaningful interactions. Submitted for Interact'13 (2013).