=Paper=
{{Paper
|id=Vol-1907/4_mici_kazi
|storemode=property
|title=Mixed-Initiative Creative Interfaces for Dynamic Illustration and Generative Design
|pdfUrl=https://ceur-ws.org/Vol-1907/4_mici_kazi.pdf
|volume=Vol-1907
|authors=Rubaiat Habib Kazi,Benjamin Lafreniere,Justin Matejka
|dblpUrl=https://dblp.org/rec/conf/chi/KaziLM17
}}
==Mixed-Initiative Creative Interfaces for Dynamic Illustration and Generative Design==
https://ceur-ws.org/Vol-1907/4_mici_kazi.pdf
Mixed-Initiative Creative Interfaces for
Dynamic Illustration and
Generative Design
Rubaiat Habib Kazi Ben Lafreniere Abstract
Autodesk Research Autodesk Research Modern software applications for design and drawing
Toronto, ON, Canada Toronto, ON, Canada provide users with rich sets of features, many of which
rubaiat.habib@autodesk.com ben.lafreniere@autodesk.com provide computational assistance or otherwise amplify
the user’s ability to express themselves creatively. In
Justin Matejka this paper, we present several research projects that
Autodesk Research we have undertaken in this area, with a focus on how
Toronto, ON, Canada they enable a collaboration between the user and a
justin.matejka@autodesk.com computational system, and thus reveal potential areas
for research on mixed-initiative creative interfaces, and
interaction with these kinds of system.
Author Keywords
Generative design; dynamic illustration; mixed-
initiative creative interfaces
Copyright © 2017 for this paper is held by the authors. ACM Classification Keywords
Proceedings of MICI 2017: CHI Workshop on Mixed-Initiative Creative
H.5.m. Information interfaces and presentation: Misc.
Interfaces.
Introduction
Software applications for creative tasks ranging from
computer-aided design to sketching have moved far
beyond simply emulating the analog tools that came
before them. Instead, modern design software provides
an environment and rich sets of features that assist the
user in myriad ways. Likewise, software tools for design
and drawing enable creative minds to amplify their dynamics in a way that is easy to learn, apply, and
thoughts and express themselves in ways that were experiment with.
simply not possible using analog tools. In this paper,
we present a number of recent and ongoing projects Our approach aims to break those dynamics into
that we have undertaken to push the limits of these simple, understandable chunks that are easy to
domains, with a focus on innovations that enable the comprehend and which amplify the user’s
user and the design software system to work in tight understanding of the dynamics. Below we present two
collaboration with one another to produce creative examples of this approach.
results.
The projects we will present suggest several models for
human-machine collaboration in mixed-initiative
creative interfaces that have not been explored in detail
in past work. First, we explore challenges in Figure 1: We devised the principles of animation as a set of
seven motion amplifiers. The amplifiers are independent from
representing complex algorithms, simulations, and
each other and can be arbitrarily mixed to design motion
dynamics to amplify a user’s ability to understand and effects of an animated object. These tools correspond to the
interact with those phenomena. Specifically, we discuss principles of animation that address the geometric
these ideas in the context of work we’ve undertaken on manipulation, clarity, and temporal aspects of animation.
dynamic drawings. Second, we discuss the broader
Motion amplifiers [1] bring the exaggerated dynamics
area of generative design, touching on the interaction
of classical 2D animation into dynamic drawings. We
challenges of specifying design constraints to a system,
prototyped a sketching tool that formulates the
and evaluating the creative outputs generated by a
principles of 2D animation as a set of motion amplifiers
computational system.
that can be combined at will. Users can sketch
drawings and record motions through direct
Dynamic Drawings
manipulation, and apply the animation principles by
The goal of this line of work is to make animation easy
simply tapping on the corresponding amplifier icons
and accessible to a wider audience, beyond professional
(Figure 1). Each amplifier imposes deformations to an
animators. In general, crafting compelling motion
underlying grid, which in turn updates the
graphics is a challenging task, requiring significant
corresponding strokes. The goal of our design is to
practice and mastery of complex skills. Artists require a
increase the user's ability to modularize the
deep understanding of the complex physical simulations
fundamental concepts of 2D animation and combine
and dynamics of the effects they are trying to recreate
them together in new and powerful ways. By
(e.g., the motion of hair blowing in the wind) to be able
representing the principles of animation in a simplified
to recreate them effectively. The key challenge for
manner, our system offers users, particularly those
these types of systems is to provide amateurs with a
with no prior experience in animation, the opportunity
set of tools that allow them make use of these complex
to rapidly explore animation effects and produce viewed as the system’s contribution to a collaborative
expressive animated illustrations. creative process with the user. To enable a tight
interactive loop in which both the user and the system
contribute, the complex dynamics are decomposed into
manageable chunks (energy brushes and motion
amplifiers, respectively) which can be understood by
the user and assimilated into their creative process.
There is a connection to what Seymour Papert refers to
Figure 2: Example of a dynamic drawing authored with our as breaking down a complex problem into "mind-size
system. Left: original user drawing. Middle: the energy brush bites," which makes knowledge more communicable,
gestures to specify the underlying forces and detail effects.
more assimilable, and more simply constructible [3].
Right: the resulting dynamic drawing.
We see this as a potentially valuable model for human-
Energy brushes [7]. Dynamic effects such as waves, machine collaboration in mixed-initiative creative
splashes, fire, smoke, and explosions are an integral interfaces. User interfaces for creative tasks that
part of stylized animations. However, they are leverage these “mind-size bites” as a natural
challenging to produce because they are time-varying vocabulary to understand and compose complex
phenomena. We developed an interactive system that phenomena and algorithms might have powerful
enables artists to design such dynamics by sketching influence on how creators think, learn, and explore.
the underlying forces with energy brushes to animate
drawings and textures. An energy brush creates a Generative Design
stable, repetitive velocity field pattern by moving flow In generative design systems, a designer provides a set
particles along the gesture. Complex dynamics, such as of design goals and constraints to the system – these
splash, fire, hair, and explosion can be achieved by could be material parameters, manufacturing methods,
combining these simple energy patterns. Our approach or cost constraints. The system then generates a range
provides artistic controls over coarse and fine scales, of design alternatives, in a process that could
and preserves the fluidity of freeform sketching. Most potentially produce hundreds or thousands of solutions
importantly, it enables skilled artists, as well as that meet the specified constraints. From these, the
novices, to integrate complex dynamics into their designer can select a solution, or iterate on their
animations, by composing them out of simple energy specified set of parameters to get a new set of design
patterns in an interactive and iterative manner. alternatives from the system.
Dynamic Drawing as a Mixed-Initiative Creative Though generative design is not itself a new concept, it
Interface is only recently that generative design capabilities have
In the projects discussed above, the simulations of become available in commercial 3D design software,
complex dynamics provided by the system can be and researchers have begun to examine the
interactions between human designers and generative generating designs may be on the order of minutes or
design systems. hours, which can impede a tight interaction loop
between the designer and the user.
Viewed through the lens of mixed-initiative creative
interfaces, the generative design process presents Conclusions
several interesting challenges. In this paper, we have outlined how recent innovations
in design and drawing are creating systems in which
First, there is the question of how to enable the human users collaborate actively with computational systems
designer to communicate their goals and constraints to to produce creative results. We are interested in
the generative design system. Current solutions ask the contributing to the workshop and helping to define a
user to specify physical constraints and forces in a research agenda for mixed-initiative creative interfaces
manner that can be rigid and is often dictated by the in this domain and more broadly.
specific generative design algorithms used by the
system. Investigating more natural ways for a user to References
express their goals and constraints, or ambiguities in 1. Rubaiat Habib Kazi, Tovi Grossman, Nobuyuki
their goals, is an interesting area for investigation. In Umetani, and George Fitzmaurice. 2016. Motion
particular, we believe there is an opportunity to learn Amplifiers: Sketching Dynamic Illustrations Using
from and adapt approaches that have been used in the Principles of 2D Animation. Proceedings of the
mixed-initiative interfaces for game-level design [2,4,6] 2016 CHI Conference on Human Factors in
and generating procedural 3D worlds [5]. Computing Systems, ACM, 4599–4609.
Second, there is the question of how best to present 2. Antonios Liapis, Georgios N. Yannakakis, and Julian
the user with the set designs generated by the system, Togelius. 2013. Sentient Sketchbook: Computer-
which could number in the hundreds or thousands. Aided Game Level Authoring. Proceedings of the 8th
Solutions could include interfaces for filtering, ranking, Conference on the Foundations of Digital Games,
and visualizing the produced designs. One interesting 213–220.
possibility in this space is for the system to use implicit
indicators of the user’s interest in produced designs 3. Seymour Papert. 1980. Mindstorms: Children,
(e.g., using machine learning techniques), to build a Computers, and Powerful Ideas. Basic Books, Inc.,
model of the user’s interests. This could be used to New York, NY, USA.
narrow the space of generated designs, or feed back
into further iterations of the generative design process. 4. Mohammad Shaker, Noor Shaker, and Julian
Togelius. 2014. Ropossum: An Authoring Tool for
Finally, there are unique interaction challenges raised Designing, Optimizing and Solving Cut the Rope
by generative design with current technologies. For Levels. Proceedings of the Ninth AAAI Conference on
example, in 3D shape synthesis, the latency for
Artificial Intelligence and Interactive Digital
Entertainment, AAAI Press, 215–216.
5. R. M. Smelik, T. Tutenel, K. J. De Kraker, and R.
Bidarra. 2011. Semantic 3D Media and Content: A
Declarative Approach to Procedural Modeling of
Virtual Worlds. Comput. Graph. 35, 2: 352–363.
6. Gillian Smith, Jim Whitehead, and Michael Mateas.
2010. Tanagra: A Mixed-initiative Level Design Tool.
Proceedings of the Fifth International Conference on
the Foundations of Digital Games, ACM, 209–216.
7. Jun Xing, Rubaiat Habib Kazi, Tovi Grossman, Li-Yi
Wei, Jos Stam, and George Fitzmaurice. 2016.
Energy-Brushes: Interactive Tools for Illustrating
Stylized Elemental Dynamics. Proceedings of the
29th Annual Symposium on User Interface Software
and Technology, ACM, 755–766.