=Paper=
{{Paper
|id=Vol-2903/IUI21WS-TExSS-3
|storemode=property
|title=Machine Learning Explanations as Boundary Objects: How AI Researchers Explain and Non-Experts Perceive Machine Learning
|pdfUrl=https://ceur-ws.org/Vol-2903/IUI21WS-TExSS-3.pdf
|volume=Vol-2903
|authors=Amid Ayobia,Katarzyna Stawarz,Dmitri Katz,Paul Marshall,Taku Yamagata,Raúl Santos-Rodríguez,Peter Flach,Aisling Ann O'Kane
|dblpUrl=https://dblp.org/rec/conf/iui/AyobiSKMYSFO21
}}
==Machine Learning Explanations as Boundary Objects: How AI Researchers Explain and Non-Experts Perceive Machine Learning==
https://ceur-ws.org/Vol-2903/IUI21WS-TExSS-3.pdf
Machine Learning Explanations as Boundary Objects: How AI
Researchers Explain and Non-Experts Perceive Machine Learning
Amid Ayobia, Katarzyna Stawarzb, Dmitri Katzc, Paul Marshalla, Taku Yamagataa, Raúl
Santos-Rodrígueza, Peter Flacha, and Aisling Ann O'Kanea
a
University of Bristol, Bristol, England
b
Cardiff University, Cardiff, Wales
c
The Open University, Milton Keynes, England
Abstract
Understanding artificial intelligence (AI) and machine learning (ML) approaches is becoming
increasingly important for people with a wide range of professional backgrounds. However, it
is unclear how ML concepts can be effectively explained as part of human-centred and
multidisciplinary design processes. We provide a qualitative account of how AI researchers
explained and non-experts perceived ML concepts as part of a co-design project that aimed to
inform the design of ML applications for diabetes self-care. We identify benefits and challenges
of explaining ML concepts with analogical narratives, information visualisations, and publicly
available videos. Co-design participants reported not only gaining an improved understanding
of ML concepts but also highlighted challenges of understanding ML explanations, including
misalignments between scientific models and their lived self-care experiences and individual
information needs. We frame our findings through the lens of Stars and Griesemer’s concept of
boundary objects to discuss how the presentation of user-centred ML explanations could strike
a balance between being plastic and robust enough to support design objectives and people’s
individual information needs.
Keywords 1
Explainable AI, AI literacy, Explanation, Diabetes, Boundary Objects
1. Introduction and Related Work technology [30], more recent work has sought
to integrate the approaches drawing not only on
human-centred but also participatory HCI
Understanding artificial intelligence (AI)
methodologies to understanding both how AI
approaches is becoming increasingly important
technology is being developed and how human-
for industry practitioners with a wide range of
AI interactions could be designed. “What I do
professional backgrounds and academic
know is that the future is not AI; it can only be
researchers working in interdisciplinary fields,
an AI enabled through HCI,” writes Harper
such as human-computer interaction (HCI).
[12], reflecting on the important role HCI could
While HCI and AI research have often been
play in the new age of AI. In particular, the HCI
characterised as having quite distinct views of
community has looked at practices of
the relationship between humans and
researchers, data scientists, user experiences
Joint Proceedings of the ACM IUI 2021 Workshops, April 13-17,
2021, College Station, USA
EMAIL: amid.ayobi@bristol.ac.uk (A. 1);
stawarzk@cardiff.ac.uk (A. 2); dmitrikatz23@gmail.com (A. 3);
p.marshall@bristol.ac.uk (A. 4); taku.yamagata@bristol.ac.uk
(A. 5), enrsr@bristol.ac.uk (A. 6); peter.flach@bristol.ac.uk (A.
7); a.okane@bristol.ac.uk (A. 8)
ORCID: 0000-0003-1104-0043 (A. 1); 0000-0001-9021-0615 (A.
2); 0000-0003-1345-7539 (A. 3); 0000-0003-2950-8310 (A. 4);
0000-0001-8624-7669 (A. 5); 0000-0001-9576-3905 (A. 6);
0000-0001-6857-5810 (A. 7); 0000-0001-8219-8126 (A. 8)
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
designers, and end-users to bridge gaps demonstrated the utility of interactive machine
between HCI and AI. learning to support people with disabilities in
Pointing out that the manual work and creating and customising gesturally controlled
human factors of ML research can be musical interfaces through a series of
overlooked, Gillies et al. [10] and Clarke et al. participatory design workshops. Although
[5] encourage researchers to draw on human- participants faced challenges in understanding
centred approaches to investigate the situated the training process to build instruments, they
and collaborative facets of ML practices and the managed to appropriate pre-trained instruments
design of usable ML support tools. Taking up according to their capabilities.
this call, Muller et al. [22] unpack how data
scientists develop an intuitive sense of their 2. Method
datasets and how they create ground truth
values as part of their data work. However, this
The objective of this study was to
perceived agency of working with data also has
its limits. For example, based on a contextual investigate how ML explanations were
inquiry, Kaur et al. [15] find that data scientists presented and perceived as part of a co-design
project that aimed to co-design ML-based
over-trust ML interpretability tools and face
challenges to accurately describe output data decision support concepts and co-create
suitable machine learning approaches. The
visualisations.
As ML plays an increasingly important role project involved HCI researchers, AI
in the design of products, not only data researchers, and industry practitioners, as well
scientists but also designers engage with ML as fifteen participants with T1 diabetes. This
[17]. However, designing human-AI paper focuses on one workshop that specifically
interactions entails major challenges [6, 7, 11, mediated ML concepts to workshop
31, 32]. For example, design professionals participants. We did not aim to evaluate the
report difficulties in understanding ML effectiveness or efficiency of the ML
explanations. Instead, we investigated the
capabilities, and recommend adopting data
following research questions:
science jargon, including the use of quantitative
evaluation methods, to be able to contribute to
a data-centric work culture [31]. Envisioning a • How did AI researchers explain ML
variety of feasible AI experiences and rapidly concepts to co-design workshop
prototyping realistic human-AI interactions are participants?
further challenges that designers are faced with, • How did co-design workshop participants
considering time extensive ML training perceive the presented ML explanations?
workflows and a lack of data to design with [6, • What are the transferable implications for
32, 33]. Furthermore, designers can find it designing user-centred ML explanations?
difficult to productively collaborate with AI
engineers because of a lack of a shared The first author conducted 18 interviews via
language and methodologies that help align phone and video conference systems.
human-cantered design and machine learning Interviews involved eight people with T1D who
work streams [11]. participated in the co-design project (referred to
Moving on from how data scientists and as P1, P2, etc.), three HCI researchers (e.g.
designers work with AI concepts and tools, HCI1), and three AI researchers (e.g. AI1). To
prior work has drawn on participatory support recollection before the interviews, a
approaches to investigate end-users’ slide deck was shared with participants
perceptions and the ethical implications of AI including ML explanations used throughout the
systems [9, 21, 23, 28]. In particular, Loi et al. workshop. Interview topics covered prior
[17, 18] have highlighted that participatory experiences with AI/ML and perceptions of ML
design approaches are suitable to address AI explanations. Interview questions were
challenges and inform AI futures: participatory adjusted for each group of interviewees and
design has been shown to be a powerful lasted approximately 30 minutes. The audio
methodology to explore the design space of recordings were transcribed verbatim. This
desirable technologies and foster mutual interview study received an ethical approval
learning between multidisciplinary actors [2, from the Faculty Ethics Committee.
24, 26, 29]. For example, Katan et al. [13] have
Data collection and analysis was conducted The concept of anomaly detection was
in a staggered way according to project roles. A explained with the help of two line graphs (see
qualitative data analysis software was used by Figure 1). The first line graph showed
the first author to thematically code data [3]. As continuous blood glucose measures over time
some participants were authors, each in milligrams per decilitre. Representing a
interviewee was sent the representative quotes binary machine interpretation, the second line
for the codes and explicitly agreed to their use graph highlighted four anomalies in the
before group analysis was conducted. The data continuous blood glucose data of the first line
corpus was iteratively analysed in an inductive graph. Participants reported being used to
fashion drawing on open coding by all the reflect on line graphs when using different
authors [3]. health and wellbeing applications [14].
However, they wished to hear narratives that
3. Findings described the real-word context and
experiences of the person who collected the
data to be able to relate and make sense of the
We first report on how AI researchers anomaly explanation. For example, P8 made it
explained ML concepts to participants as part
clear that it is important not only to understand
of a co-design workshop using different types the contributing factors of anomalies but also
of explanations, including analogical
how anomalies could be managed:
narratives, data visualisations, and publicly
available videos. We then describe how “What you’re not really seeing is why those
workshop participants, including HCI anomalies are happening. […] if we’re
researchers and people with diabetes, perceived talking about diabetes, I think the ‘why’ is
the presented ML explanations and what just as important in order to understand how
benefits and challenges they experienced. to tackle those anomalies.” (P8)
Moreover, participants highlighted that
3.1. ML Explanations binary representations of anomalies (see Figure
1, second line graph) may be useful to explain
Since the objective of the co-design project the concept of anomaly detection, however,
involved the design of ML based applications potentially not suitable to support sense-making
for diabetes self-management, AI researchers and decision-making in everyday life. They felt
used different methods to explain ML more comfortable with data visualisations that
approaches to workshop participants, including supported their agency in identifying and
data visualisations, analogies and videos of dismissing anomalies based on their lived
real-world AI applications. experience. For example, high blood glucose
values in daily life were not necessarily
anomalous if participants were able to make
educated guesses about contributing contextual
3.1.1. Data Visualisation: factors and manage these situations.
Anomaly Detection
3.1.2. Analogy: Reinforcement
Learning
Another ML concept that was explained as
part of the co-design workshops was
reinforcement learning. AI researchers
mediated the concept of reinforcement learning
with the help of the analogy of training a dog.
Figure 1: line graphs to explain anomaly detection
associated with dietary challenges people with
diabetes can experience:
Reinforcement Learning
• RL is like a dog learning good behaviours
• Give a cookie for good behaviour
“I’ve got dogs and I give them treats, little
• Initially, the dog behaves randomly, and dog treats. I think the use of the word cookie
accidentally does something good, and
receives a cookie.
I found amusing shall we say. Because
• Then it learns how to get a cookie cookies are not a reward for us diabetics. In
(situation + action)
fact, that’s a challenge.” (P10)
3.1.3. Video: Agent Behaviour
Figure 2: analogy to explain reinforcement learning
In addition, researchers used a seminal video
“At first, it was a bit like, ‘What!?’ and then, [8], that is widely cited in the machine learning
when it was explained, it was like, ‘Oh, yes, that community, to demonstrate how agents learn to
makes sense,’” P5 remembered, indicating that play the game of hide-and-seek. The video
understanding this analogy requires translating showed how agents developed strategies and
the act of training a dog to the act of training a counterstrategies over time, such as jumping on
software agent that aims to maximise reward in cubes and moving cubes to block doors. All
a given environment. Participants reused the participants described the video as a well-
analogy of training a dog in different contexts, produced, powerful and memorable exemplar
such as P8 who wished to be able to use a semi- that mediated machine learning driven multi-
automated self-tracking approach [4] that agent behaviour with advanced character
empowers people to manually stop false design and an entertaining narrative:
machine interpretations:
“The way the video showed how they sort of
“So, you could use the dog example again, developed and how they learned was really
where it might be learning something which clear, and the characters are quite cute, so I
necessarily isn’t correct, if that makes sense, think it was quite funny as well, at the same
like it might find a pattern which you don’t time. Again, that was a great example to
want it to learn. So, I think... I don’t think show how machine learning can work.”
it’s a question of like manually versus (P5).
automatic. I think they need to work together
in some shape or form. […] there needs to However, similar to the analogy of training
be some sort of manual input to tell the a dog, participants found it challenging to
machine learning aspect, ‘Please don’t transfer the hide-and-seek game to their
learn this.’” (P8) diabetes self-management practices,
highlighting that machine learning explanations
Participants also perceived limitations of need not only be abstracted but also transferred
using the analogy of training a dog with to a personally meaningful and research-
cookies. For example, P3’s account refers to the specific context:
challenges of transferring anticipated emotions,
such as the desire to learn, to machines and the “I’m not sure how to transfer that to a
challenges of translating the analogy to the diabetic situation in a way, that particular
design space of digital applications: format. I mean there must be one, I haven’t
really thought that one through. […] what
“If a machine has desire or it’s how you have you got to have? You’ve got to have
explain the one for a cookie. I think that’s something whereby you’re correlating
the bit I find it hard to get my head round eating or carb intake, exercise and taking
with a machine […] So, I don’t know how insulin. So, those three factors, I think.” (P6)
you reward an app like a machine” (P3).
Furthermore, P10 politely critiqued the use
of the term ‘cookie’ in the context of diabetes
management, considering that cookies can be
3.2. Understanding of ML Participants described AI research and AI
concepts, such as ML, as data driven algorithms
Explanations that are written by humans and run on
computers. “AI is computers that learn, that
HCI researchers and workshop participants once you set certain criteria up or whatever,
reported gaining an improved understanding of they can gain knowledge themselves without
the presented ML approaches. Participants being told to gain knowledge, yeah. I think that
explained that even though they might not fully is the simplest form,” explained P10,
understand the “inner workings” (P3) of ML referencing the learning capabilities of AI
approaches, it was important to gain some technologies. Participants with diabetes also
knowledge of ML concepts to develop trust in reflected on potential limitations of ML
the design process and potential ML approaches, including differences between
implementations, though some noted the manual and automatic data collection, roles of
importance of it being presented in data quality and potential limitations of
understandable terms: predictive functionalities:
“I don’t think you just blindly follow stuff, “If it’s showing information based on weeks
particularly when designs are being made in and weeks of data-gathering and it’s
the background […] it’s better to put it into basically giving you your average day, I
terms that we could understand, which is mean, I suppose that could be useful. But
quite difficult when it can be so complex, but then, if you suddenly change your physical
I do think it’s quite important to give us activity, or you’re eating something at a
some understanding of how and what’s time that you don’t usually eat something,
going on in the background.” (P5) then I guess that could disrupt it.” (P8).
HCI researchers and participants reported
that learning about ML approaches as part of
4. Discussion
the workshops changed their prior
understanding of the benefits and limitations of Understanding AI approaches is becoming
ML based technologies. “Before, it was kind of increasingly important for people with a wide
like, you know, computers being able to think range of professional backgrounds in industrial
for themselves or like have a sentience,” P8 and academic settings. We have provided a
explained, exemplifying that some participants’ qualitative account of how AI researchers
prior understanding of AI was based on science explained ML concepts to HCI researchers and
fiction narratives that typically portray AI people with diabetes as part of a co-design
technologies with potentially dangerous project that aimed to inform the design of ML
autonomous and emotional capacities. applications for diabetes self-care. Here we
Reflecting on their co-design workshop discuss our findings through the lens of Stars
experiences, participants demonstrated and Griesemer’s concept of boundary objects to
differing degrees of ML literacy in creative outline how the presentation of user-centred
ways. For example, participants used existing ML explanations could strike a balance
digital consumer services as examples to between being plastic and robust enough to
explain ML functionality, such as support design objectives and people’s
recommendations: individual information needs as part of
multidisciplinary projects.
“I think the term ‘artificial intelligence’ is a
bit more specific than that, I think. It’s more 4.1. Framing ML Explanations as
to do with machine learning, […] So it’s
things like, you know, how Netflix decides
Boundary Objects
what you watch, kind of thing, or how you
choose your recommendation. I think it’s Star and Griesemer’s [25] concept of
algorithms, really.” (P3) boundary objects has been used as a theoretical
lens to understand how various actors with
different backgrounds, roles, and interests
successfully collaborate as part of
multidisciplinary endeavours. Boundary boundary objects - robustness and plasticity -
objects are artefacts that facilitate imply for the design of ML explanations.
communication and collaboration between
multiple actors and are defined as: 4.2. Balancing Robustness and
“objects which are both plastic enough to Plasticity
adapt to local needs and the constraints of
the several parties employing them, yet While the robustness of a ML explanation
robust enough to maintain a common can be described with features, such as being
identity across sites” (ibid, p. 393). algorithmically correct and transferable to
different research settings, the plasticity of a
In their study of how amateurs, ML explanation can be associated with
professionals, and administrators collaborate in features, such as being adaptable to people’s
a museum setting, Star and Griesemer lived experiences, reflective capacities, and
distinguish between four types of boundary information needs. Design techniques, such as
objects: (1) repositories provide a central personalisation and customisation are
location where objects, such as samples, are particularly suitable to support people’s
systematically stored and are available for individual needs and experiences of agency,
people to be used; (2) ideal type is an object, such as sense of identify and ownership [1].
such as a diagram, that provides an abstracted A robust and plastic enough ML explanation
representation that can be adapted by others; (3) support actors, such as a co-designer, product
coincident boundaries are objects, such as manager, and end-user, in making sense of and
tailored maps: they are defined by common acting on a ML explanation.
(geographical) boundaries but can have In our study, we have observed that
different contents, purposes, and styles; (4) participants made sense of ML explanations
standardised forms are boundary objects that based on their prior knowledge of AI narratives
are used as formal methods of communication and technologies, reused ML explanations,
across different actors. While these four types such as the analogy of training a dog, as part of
of boundary objects can be used in different co-design activities, and co-created mockups
ways and can have different meanings for that visualised possible ML-based
different actors from different social worlds, functionalities, such as predicting blood
they typically support communication and glucose values.
facilitate collaborations. Although boundary An important contributing factor for
objects aim to resolve conflicts, they are not adopting a ML explanation was familiarity:
neutral. The creation of boundary objects participants particularly valued the analogical
requires carefully managing power narrative of training a dog, since it seemed to
relationships to avoid forced use of predefined help bridge the unknown concept of
representations that can cause systematic reinforcement learning and the known practice
exclusion, discrimination, and injustice. of training a dog. Barriers to adopting and using
In our case, AI researchers used different a ML explanation seemed to be a lack of
types of ML explanations to support HCI abstraction and associations with people’s lived
researchers and people with diabetes in co- self-care experiences.
designing possible ML systems. To foster a
shared understanding of ML concepts, they 4.3. Considering Sociocultural
used analogical narratives to explain
reinforcement learning, data visualisations to Contexts and Ethical Implications
explain anomaly detection, and publicly
available videos to explain multi-agent The sociocultural underpinning of boundary
behaviour. These explanations can be objects suggests that co-designing a plastic and
characterised as ideal types, based on Star and robust enough ML explanation involves not
Griesemer’s types of boundary objects. only representing a specific ML concept
Framing these ML explanations as boundary correctly and evaluating whether the ML
objects poses the question what the theory of explanation was correctly understood, but also
boundary objects and the key properties of gaining a holistic and non-judgemental
understanding of how the ML explanation was
appropriated and experienced within a certain Content could be presented in engaging
context. For example, our qualitive inquiry has ways, as demonstrated by the creative
revealed the importance of tailoring general presentation of AI as a monster metaphor [7],
ML explanations to specific cases, such as self- the use of tangible cards in the context of data
managing diabetes, to avoid misalignments protection regulations [20], and “inspirational
between people’s lived experience and bits” [27] that expose dynamic properties of
scientific concepts of ML. sensors to allow designers to understand and
Conceptualising ML explanations as experience the properties of technology that
boundary objects means to acknowledge that might be used in research and design projects.
abstraction and ambiguity can lead to divergent
viewpoints, misinterpretations, and 5. Conclusion
misunderstandings. Our findings suggest that
gaining a good enough understanding of ML
We have provided a qualitative account of
explanations can support participants in
developing trust in design processes, data how AI researchers explained and non-experts
collection and analysis technologies, and perceived ML concepts as part of a co-design
project that aimed to inform the design of ML
overarching research objectives. However,
what a good enough understanding is and applications for diabetes self-care.
We have identified benefits and challenges
whether a good enough understanding of ML
explanations and functionalities is ethically of explaining ML concepts with analogical
responsible depends on contextual factors, such narratives, information visualisations, and
as the sensitivity of a research setting. While publicly available videos. Co-design
participants with diabetes sketched predictive participants reported not only gaining an
functionalities during co-design activities, AI improved understanding of ML concepts but
researchers highlighted fundamental also gaining trust in the co-design process of
differences between the desirability and ML based technologies, data collection and
analysis technologies, and overarching research
feasibility of ML-driven systems considering
objectives. However, co-design participants
fatal implications of false predictions and
recommendations in the case of continuous also highlighted challenges of understanding
blood glucose monitoring and management. ML explanations, including misalignments
between scientific models of ML and their lived
self-care experiences and prior knowledge of
4.4. Applying User Experience AI and ML approaches.
Design Methods Based on this understanding, we have
framed our findings through the lens of Stars
Developing a plastic and robust enough ML and Griesemer’s concept of boundary objects to
explanation can require an iterative and discuss how the presentation of user-centred
multidisciplinary design process with a detailed ML explanations could maintain a delicate
understanding of ML approaches, user groups, balance between being plastic and robust
and the intended purpose of a ML explanation. enough to support design objectives and
Considering that design methods and tools people’s individual information needs as part of
to facilitate co-design are recognised multidisciplinary projects.
methodological contributions [2, 16], we
encourage researchers and practitioners to 6. Acknowledgements
explore the design space of “learner-centered”
[19] ML explanations specifically for human- This project was funded by an Innovate UK
centred technology projects. Such design-led Digital Catalyst Award - Digital Health. RSR is
inquiries could explore how scientific ML partially funded by the UKRI Turing AI
explanations could be intertwined with people’s Fellowship EP/V024817/1. Many thanks to all
lived self-care experiences and their study participants and reviewers.
information needs as co-designers. These
explanation instruments could represent AI/ML
at a layer of abstraction above specific
algorithms and communicate not just of what
AI/ML can do, but also what it cannot.
7. References DOI:https://doi.org/10.1007/s42486-020-
00028-0.
[1] Ayobi, A. et al. 2020. Trackly: A [10] Gillies, M. et al. 2016. Human-Centred
Customisable and Pictorial Self-Tracking Machine Learning. Proceedings of the
App to Support Agency in Multiple 2016 CHI Conference Extended Abstracts
Sclerosis Self-Care. Proceedings of the on Human Factors in Computing Systems -
2020 CHI Conference on Human Factors CHI EA ’16 (San Jose, California, USA,
in Computing Systems (CHI ’20) (2020). 2016), 3558–3565.
[2] Bødker, S. et al. 2000. Co-operative [11] Girardin, F. and Lathia, N. 2017. When
Design—perspectives on 20 years with ‘the User Experience Designers Partner with
Scandinavian IT Design Model.’ Data Scientists. AAAI Spring Symposia
proceedings of NordiCHI (2000), 22–24. (2017).
[3] Braun, V. and Clarke, V. 2006. Using [12] Harper, R.H.R. 2019. The Role of HCI in
thematic analysis in psychology. the Age of AI. International Journal of
Qualitative Research in Psychology. 3, 2 Human–Computer Interaction. 35, 15 (Sep.
(Jan. 2006), 77–101. 2019), 1331–1344.
DOI:https://doi.org/10.1191/1478088706qp DOI:https://doi.org/10.1080/10447318.201
063oa. 9.1631527.
[4] Choe, E.K. et al. 2017. Semi-Automated [13] Katan, S. et al. 2015. Using Interactive
Tracking: A Balanced Approach for Self- Machine Learning to Support Interface
Monitoring Applications. IEEE Pervasive Development Through Workshops with
Computing. 16, 1 (Jan. 2017), 74–84. Disabled People. Proceedings of the 33rd
DOI:https://doi.org/10.1109/MPRV.2017.1 Annual ACM Conference on Human
8. Factors in Computing Systems - CHI ’15
[5] Clarke, M.F. et al. 2019. Better Supporting (Seoul, Republic of Korea, 2015), 251–254.
Workers in ML Workplaces. Conference [14] Katz, D.S. et al. 2018. Data, Data
Companion Publication of the 2019 on Everywhere, and Still Too Hard to Link:
Computer Supported Cooperative Work Insights from User Interactions with
and Social Computing (Austin TX USA, Diabetes Apps. Proceedings of the 2018
Nov. 2019), 443–448. CHI Conference on Human Factors in
Computing Systems (New York, NY, USA,
[6] Dove, G. et al. 2017. UX Design
2018), 503:1-503:12.
Innovation: Challenges for Working with
Machine Learning as a Design Material. [15] Kaur, H. et al. 2020. Interpreting
Proceedings of the 2017 CHI Conference Interpretability: Understanding Data
on Human Factors in Computing Systems Scientists’ Use of Interpretability Tools for
(Denver Colorado USA, May 2017), 278– Machine Learning. Proceedings of the
288. 2020 CHI Conference on Human Factors
in Computing Systems (Honolulu HI USA,
[7] Dove, G. and Fayard, A.-L. 2020.
Apr. 2020), 1–14.
Monsters, Metaphors, and Machine
Learning. Proceedings of the 2020 CHI [16] Kensing, F. and Blomberg, J. 1998.
Conference on Human Factors in Participatory design: Issues and concerns.
Computing Systems (Honolulu HI USA, Computer Supported Cooperative Work
Apr. 2020), 1–17. (CSCW). 7, 3–4 (1998), 167–185.
[8] Emergent Tool Use from Multi-Agent [17] Loi, D. et al. 2019. Co-designing AI
Interaction: 2019. Futures: Integrating AI Ethics, Social
https://openai.com/blog/emergent-tool- Computing, and Design. Companion
use/. Accessed: 2020-09-17. Publication of the 2019 on Designing
Interactive Systems Conference 2019
[9] Fu, Z. and Zhou, Y. 2020. Research on
Companion - DIS ’19 Companion (San
human–AI co-creation based on reflective
Diego, CA, USA, 2019), 381–384.
design practice. CCF Transactions on
Pervasive Computing and Interaction. 2, 1 [18] Loi, D. et al. 2018. PD manifesto for AI
(Mar. 2020), 33–41. futures. Proceedings of the 15th
Participatory Design Conference on Short [26] Steen, M. 2013. Co-design as a process of
Papers, Situated Actions, Workshops and joint inquiry and imagination. Design
Tutorial - PDC ’18 (Hasselt and Genk, Issues. 29, 2 (2013), 16–28.
Belgium, 2018), 1–4. [27] Sundström, P. et al. 2011. Inspirational
[19] Long, D. and Magerko, B. 2020. What is bits: towards a shared understanding of the
AI Literacy? Competencies and Design digital material. Proceedings of the 2011
Considerations. Proceedings of the 2020 annual conference on Human factors in
CHI Conference on Human Factors in computing systems - CHI ’11 (Vancouver,
Computing Systems (Honolulu HI USA, BC, Canada, 2011), 1561.
Apr. 2020), 1–16. [28] Trewin, S. et al. 2019. Considerations for
[20] Luger, E. et al. 2015. Playing the Legal AI fairness for people with disabilities. AI
Card: Using Ideation Cards to Raise Data Matters. 5, 3 (Dec. 2019), 40–63.
Protection Issues within the Design DOI:https://doi.org/10.1145/3362077.3362
Process. Proceedings of the 33rd Annual 086.
ACM Conference on Human Factors in [29] Vartiainen, H. et al. 2020. Machine
Computing Systems - CHI ’15 (Seoul, learning for middle-schoolers: Children as
Republic of Korea, 2015), 457–466. designers of machine-learning apps. 2020
[21] Morrison, C. et al. 2017. Imagining IEEE Frontiers in Education Conference
Artificial Intelligence Applications with (FIE) (Uppsala, Sweden, Oct. 2020), 1–9.
People with Visual Disabilities using [30] Winograd, T. 2006. Shifting viewpoints:
Tactile Ideation. Proceedings of the 19th Artificial intelligence and human–computer
International ACM SIGACCESS interaction. Artificial Intelligence. 170, 18
Conference on Computers and Accessibility (Dec. 2006), 1256–1258.
(Baltimore Maryland USA, Oct. 2017), 81– DOI:https://doi.org/10.1016/j.artint.2006.1
90. 0.011.
[22] Muller, M. et al. 2019. How Data Science [31] Yang, Q. et al. 2018. Investigating How
Workers Work with Data: Discovery, Experienced UX Designers Effectively
Capture, Curation, Design, Creation. Work with Machine Learning. Proceedings
Proceedings of the 2019 CHI Conference of the 2018 on Designing Interactive
on Human Factors in Computing Systems - Systems Conference 2018 - DIS ’18 (Hong
CHI ’19 (Glasgow, Scotland Uk, 2019), 1– Kong, China, 2018), 585–596.
15.
[32] Yang, Q. et al. 2020. Re-examining
[23] Muller, M. and Liao, Q.V. 2017. Exploring Whether, Why, and How Human-AI
AI Ethics and Values through Participatory Interaction Is Uniquely Difficult to Design.
Design Fictions. Human Computer Proceedings of the 2020 CHI Conference
Interaction Consortium. (2017). on Human Factors in Computing Systems
[24] Slattery, P. et al. 2020. Research co-design (Honolulu HI USA, Apr. 2020), 1–13.
in health: a rapid overview of reviews. [33] Yang, Q. et al. 2019. Sketching NLP: A
Health Research Policy and Systems. 18, 1 Case Study of Exploring the Right Things
(Dec. 2020), 17. To Design with Language Intelligence.
DOI:https://doi.org/10.1186/s12961-020- Proceedings of the 2019 CHI Conference
0528-9. on Human Factors in Computing Systems -
[25] Star, S.L. and Griesemer, J.R. 1989. CHI ’19 (Glasgow, Scotland Uk, 2019), 1–
Institutional ecology,translations’ and 12.
boundary objects: Amateurs and
professionals in Berkeley’s Museum of
Vertebrate Zoology, 1907-39. Social
studies of science. 19, 3 (1989), 387–420.