=Paper=
{{Paper
|id=Vol-1907/7_mici_dove
|storemode=property
|title=Mixed-Initiative Creative Interfaces for Collaborative Early-Stage Design
|pdfUrl=https://ceur-ws.org/Vol-1907/7_mici_dove.pdf
|volume=Vol-1907
|authors=Graham Dove
|dblpUrl=https://dblp.org/rec/conf/chi/Dove17
}}
==Mixed-Initiative Creative Interfaces for Collaborative Early-Stage Design==
https://ceur-ws.org/Vol-1907/7_mici_dove.pdf
Mixed-Initiative Creative Interfaces for
Collaborative Early-Stage Design
Graham Dove Abstract
CAVI, Aarhus University This position paper outlines my ongoing research into
Aarhus, 8200, Denmark how creativity unfolds in early stage design activities,
graham.dove@cc.au.dk and how such creativity can be supported. It considers
the challenges posed by this context in terms of
possible mixed-initiative creative interfaces; and poses
questions for my own research, and for designers of
mixed-initiative creativity support tools.
Author Keywords
Mixed-initiative interaction; creativity support; co-
design;
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g.,
HCI):
Introduction
Copyright © 2017 for this paper is held by the author(s). In the Creativity in Blended Interaction Spaces project
Proceedings of MICI 2017: CHI Workshop on Mixed-Initiative Creative at Aarhus University in Denmark, we are investigating
Interfaces.
the potential for integrating multiple digital devices and
different analog materials into shared environments
that support individual and group creativity [5]. This
research typically studies creativity in early-stage
design. We start from the perspective that tools and
materials support the creative agency of human users,
and that creative activities take place in complex
situations.
However, A.I. is now a feature of commercial creativity dynamically adapt their initiative style, and use an
support packages, e.g. generative design CAD tools interaction mode that supports human-style problem
[12,3]; conversational agents are a commonly used solving. Allen [2] identifies four levels of mixed-
interaction method, e.g. in smartphones and social initiative interaction:
media [4]; and computing has become ubiquitous [1].
The way computational systems are used in creative 1. Unsolicited Reporting: The computer monitors work
practice changes. Understanding how this change and if it identifies a problem notifies the user; but
unfolds, and the opportunities it presents, is an does not take or coordinate further action.
important part of our research. In this position paper I 2. Subdialogue Initiative: The computer can initiate
use Lawson and Loke’s framework for understanding subdialogues, e.g. asking for clarification. Once
the role of computers in design creativity [13] to pose clarified, initiative reverts to the user.
some questions for my own research, which I hope are 3. Fixed Subtask Initiative: The computer is
also relevant to others. responsible for particular tasks. The user sets a
goal then the computer retains the initiative whilst
Mixed Initiative Creative Interfaces working on this task. On completion initiative
This workshop is focused on those computational reverts to the user.
systems that are considered mixed-initiative creative 4. Negotiated Mixed Initiative: The computer monitors
interfaces (MICIs). This seems to be a useful category the current subtask and assesses whether: it is
on the spectrum between tools that support human able to, has the resources to, and is best qualified
creativity and systems for autonomous computational to coordinate interaction.
creativity. To help understand how systems might be
positioned on this spectrum, I take guidance from HCI Horvitz [10] highlights the key decisions that mixed-
research into mixed-initiative interaction e.g. initiative systems must take to support collaboration,
[2,10,11]; and to help position them within my own which include:
area of study, I take guidance from design research
into the roles computers might play in creative design 1. When to engage users with a service
activities [13]. 2. How to best contribute to solving a problem
3. When to pass control of problem solving back to
Mixed Initiative Interaction users
Mixed-initiative interaction aims to develop methods 4. When to query a user for additional information
that enable computer systems to: “support an efficient,
natural interleaving of contributions by people and Roles for Computers in Creative Design Processes
computers, aimed at converging on solutions to Lawson and Loke [13] imagined a CAD tool in which
problems” [11], and “where each agent can contribute creativity support was provided through conversation
to the task what it does best” [2]. Commonly it has between designer and system. They identify five roles
been treated as a form of dialogue, in which agents that such a tool might adopt:
1. Computer as Learner: The computer absorbs and future potential in other areas? Can mixed-initiative
remembers. In conversation with a designer it interfaces help us overcome some the issues raised by
records associations, and asks for an explanation of our research into other creative practices?
things it does not understand.
2. Computer as Informer: The computer answers Early-stage Design Activities
queries, and provides information and examples in Many of the creative practices we study within CIBIS
response to specific requests from the designer. are at the early stages of design processes, where the
3. Computer as Critic: The computer checks and situation is not yet well understood and there is much
comments on the validity of ideas. It takes a ambiguity. These activities typically involve seeking and
critical stance, presents possible alternative views, sharing information and insight, finding sources of
perhaps warning about potential mistakes. inspiration, and framing inquiry.
4. Computer as Collaborator: The computer builds on
what others have said. It takes a positive and Designers often use Post-It Notes to record, share and
supportive stance, e.g. elaborating on ideas and organise ideas, and through their use of Post-It Notes
extending metaphors. also develop and extend these ideas. The Post-It Notes
5. Computer as Initiator: The computer develops new help them to think about and manipulate their ideas,
perspectives, suggests new directions for ideation and construct semantic relationships that support long-
when others have no more to say, and takes term memory [6].
initiative in generative activities.
What might a mixed initiative interface that contributes
MICIs in Early-Stage Design to these processes be like? It seems probable that
The call for participation in this workshop identifies machine learning and natural language processing can
procedural content generation for computer games as play a role in making semantic connections between
an example of how mixed-initiative interfaces are ideas, and machine vision might track individual Post-It
providing creativity support, e.g. [15]. Autodesk’s Notes as they are manipulated through a design
Dreamcatcher project [3] also seems to be an example activity. A system that embodied Lawson and Loke’s
of human and A.I. in creative collaboration. In [13] learner and informer roles might usefully augment
simplistic terms, both these examples are based on a designers’ Post It Note activities, but the question for a
human designer setting parameters and an A.I. mixed-initiative interface would remain how and when
generating and partially evaluating large numbers of to contribute appropriately. Perhaps this might be
digital alternatives before presenting these back to the facilitated by the conventions, rules and structures that
human user for further evaluation. In both cases, the human participants typically follow, e.g. when
model of creativity is based on searching a possible brainstorming. Might these provide initial guidelines for
solution space. Does this represent a limitation in the how a system would make Horvitz’s [10] key decisions,
scope of creative applications using mixed-initiative and for selecting which of Allen’s levels [2] is most
interfaces? Or do these systems offer an indication of appropriate?
When working with stakeholders during co-design contexts. For example, machine learning might be used
workshops, we have found that activities such as to extract key moments, or uncover patterns and make
making collages from photographs can help them connections between different concepts in design
interpret visualized data. These activities encourage conversations. A system that embodies each of Lawson
participants to share their experiences and insights, and Loke’s learner, informer and critic roles [13] might
and through this explore possible contexts in which be a useful addition to designers’ reflective practice.
data were generated [9]. This provides an important
source of inspiration to support collaborative ideation. A Engaging with MICIs
mixed-initiative system that could work with The tough question for mixed-initiative interaction
participants interactively as they explore data would be remains how and when computational systems should
extremely interesting to investigate, and search tools interject, engage users, and take initiative. Familiar
that utilise analogy or metaphor offer powerful sources instances, such as spellcheck and grammar checking in
inspiration e.g. [16]. However, the activities word processing software, struggle to solve this
undertaken during co-design workshops typically aim to satisfactorily; and conversational agents can be
explore participants’ subjective experiences, and so any frustrating [14]. This is likely to be further complicated
system should sensitively draw these out, and be aware in situations where groups of human collaborators
of the possibility of priming responses too strongly. interact with ecologies of interactive artifacts and
Could a mixed-initiative creative interface also play this intelligent agents.
type of role, i.e. computer as facilitator?
A survey of UX practitioners working with machine
Supporting Reflective Practice learning [7] surfaced a number of challenges designers
Our research group also develops tools and investigates face working on the type of systems likely to play a
methods to support designers’ reflective practice. For leading role in mixed-initiative creativity support for the
example we have investigated how revisiting projects areas discussed here. The danger of systems that
to reflect on the way a design space changes increases monitor activity appearing creepy was highlighted as an
awareness of the constraints introduced by particular important UX concern, and the probability that systems
design choices, qualifies understanding of how design require ground truth from large amounts of data
activities filter the design space, and prompts challenged typical approaches to prototyping. Other
reconsideration of disregarded opportunities [8]. This difficulties designers raise, which might be indicative of
requires detailed design documentation, which can some of the challenges MICIs will face, included: the
significantly add to overhead. implication that “learning” means the system and data
will change over time, and be dynamic at a large scale;
Systems that interactively record design activities, and that statistical correlations lack common-sense,
monitor them and learn about what might be can appear simplistic and stupid, and therefore false
important, and subsequently prompt designers’ critical negatives or false positives can be hard to assimilate.
reflection, could be of great benefit in this and similar The wider issues designers face working with intelligent
systems are likely to be increasingly prominent in Design Material. Accepted for 2017 CHI Conference
systems that aim for Negotiated Mixed Initiative [2], on Human Factors in Computing Systems. ACM.
and where the computer is the Initiator [13]. 8. Graham Dove, Nicolai Brodersen Hansen, & Kim
Halskov. 2016. An Argument For Design Space
Acknowledgements Reflection. In Proceedings of the 9th Nordic
Conference on Human-Computer Interaction, (pp.
This research is funded by The Danish Innovation 17-26). ACM.
Foundation grant 1311-00001B. CIBIS
9. Graham Dove, & Sara Jones. 2014. Using
Information Visualization to Support Creativity in
References Service Design Workshops. In ServDes.2014
1. Gregory D. Abowd. "What next, ubicomp?: Service Future, Proceedings of the fourth Service
celebrating an intellectual disappearing act." Design and Service Innovation Conference, (pp.
In Proceedings of the 2012 ACM Conference on 281-290), Linköping University Electronic Press
Ubiquitous Computing, pp. 31-40. ACM, 2012.
10. Eric Horvitz. 1999. Principles of Mixed-Initiative
2. James F Allen. 1999. Mixed Initiative Interaction. User Interfaces. In Proceedings of the SIGCHI
In IEEE Intelligent Systems and Their Applications. Conference on Human Factors in Computing
14,5: 14-16 Systems. (CHI 99): 159-166.
3. Autodesk. Project Dreamcatcher. Retrieved January 11. Eric Horvitz. 2007. Reflections on Challenges and
27th 2016 from promises of Mixed-Initiative Interaction. In AI
https://autodeskresearch.com/projects/dreamcatch Magazine. 28,2: 19-22
er
12. Paul Keskeys. 2016. How Generative Design Will
4. John Brownlee. 2015. Apple finaly learns AI is the Change Architecture Forever. Architizer. Retrieved
new UI. Retrieved January 27th, 2017 from January 26th from http://architizer.com/blog/how-
http://www.fastcodesign.com/3047199/apple- generative-design-will-change-architecture-forever
finally-learns-ai-is-the-new-ui
13. Bryan Lawson and Shee Ming Loke. 1997.
5. CIBIS. http://cavi.au.dk/projects/creativity-in- Computers, Words and Pictures. In Design Studies,
blended-interaction-spaces/ 18,2: 171-183
6. Graham Dove, Sille Julie Jøhnk Abildgaard, Michael 14. Ewa Luger, and Abigail Sellen. 2016. Like having a
Mose Biskjær, Nicolai Brodersen Hansen, Bo T. really bad PA. In Proceedings of the 2016 CHI
Christensen, & Kim Halskov. 2017. Grouping Notes Conference on Human Factors in Computing
Through Nodes: The Functions of Post-It Notes in Systems (CHI’16): 5286-5297
Design Team Cognition. To Appear In Design
Studies, Special Issue on Designing in the Wild 15. Gillian Smith, Jim Whitehead and Michael Mateas.
(Papers from Design Thinking Research Symposium 2011. Tangara: Reactive Planning and Constraint
DTRS11). Elsevier. Solving for Mixed-Initiative Level Design. In IEEE
Transactions on Computational Intelligence and AI
7. Graham Dove, Jodi Forlizzi, Kim Halskov, & John in Games 3,3: 201-215.
Zimmerman. 2017. UX Design Innovation:
Challenges for Working with Machine Learning as a 16. http:/yossarian.co