Modern software applications for design and drawing provide users with rich sets of features, many of which provide computational assistance or otherwise amplify the user's ability to express themselves creatively. In this paper, we present several research projects that we have undertaken in this area, with a focus on how they enable a collaboration between the user and a computational system, and thus reveal potential areas for research on mixed-initiative creative interfaces, and interaction with these kinds of system.
Generative design; dynamic illustration; mixedinitiative creative interfaces
Software applications for creative tasks ranging from computer-aided design to sketching have moved far beyond simply emulating the analog tools that came before them. Instead, modern design software provides an environment and rich sets of features that assist the user in myriad ways. Likewise, software tools for design and drawing enable creative minds to amplify their thoughts and express themselves in ways that were simply not possible using analog tools. In this paper, we present a number of recent and ongoing projects that we have undertaken to push the limits of these domains, with a focus on innovations that enable the user and the design software system to work in tight collaboration with one another to produce creative results.
The projects we will present suggest several models for human-machine collaboration in mixed-initiative creative interfaces that have not been explored in detail in past work. First, we explore challenges in representing complex algorithms, simulations, and dynamics to amplify a user’s ability to understand and interact with those phenomena. Specifically, we discuss these ideas in the context of work we’ve undertaken on dynamic drawings. Second, we discuss the broader area of generative design, touching on the interaction challenges of specifying design constraints to a system, and evaluating the creative outputs generated by a computational system.
The goal of this line of work is to make animation easy and accessible to a wider audience, beyond professional animators. In general, crafting compelling motion graphics is a challenging task, requiring significant practice and mastery of complex skills. Artists require a deep understanding of the complex physical simulations and dynamics of the effects they are trying to recreate (e.g., the motion of hair blowing in the wind) to be able to recreate them effectively. The key challenge for these types of systems is to provide amateurs with a set of tools that allow them make use of these complex dynamics in a way that is easy to learn, apply, and experiment with.
Our approach aims to break those dynamics into
simple, understandable chunks that are easy to
comprehend and which amplify the user’s
understanding of the dynamics. Below we present two
examples of this approach.
Motion amplifiers [
Energy brushes [
In generative design systems, a designer provides a set of design goals and constraints to the system – these could be material parameters, manufacturing methods, or cost constraints. The system then generates a range of design alternatives, in a process that could potentially produce hundreds or thousands of solutions that meet the specified constraints. From these, the designer can select a solution, or iterate on their specified set of parameters to get a new set of design alternatives from the system.
Though generative design is not itself a new concept, it is only recently that generative design capabilities have become available in commercial 3D design software, and researchers have begun to examine the interactions between human designers and generative design systems.
Viewed through the lens of mixed-initiative creative interfaces, the generative design process presents several interesting challenges.
First, there is the question of how to enable the human
designer to communicate their goals and constraints to
the generative design system. Current solutions ask the
user to specify physical constraints and forces in a
manner that can be rigid and is often dictated by the
specific generative design algorithms used by the
system. Investigating more natural ways for a user to
express their goals and constraints, or ambiguities in
their goals, is an interesting area for investigation. In
particular, we believe there is an opportunity to learn
from and adapt approaches that have been used in
mixed-initiative interfaces for game-level design [
Second, there is the question of how best to present the user with the set designs generated by the system, which could number in the hundreds or thousands. Solutions could include interfaces for filtering, ranking, and visualizing the produced designs. One interesting possibility in this space is for the system to use implicit indicators of the user’s interest in produced designs (e.g., using machine learning techniques), to build a model of the user’s interests. This could be used to narrow the space of generated designs, or feed back into further iterations of the generative design process. Finally, there are unique interaction challenges raised by generative design with current technologies. For example, in 3D shape synthesis, the latency for generating designs may be on the order of minutes or hours, which can impede a tight interaction loop between the designer and the user.
In this paper, we have outlined how recent innovations in design and drawing are creating systems in which users collaborate actively with computational systems to produce creative results. We are interested in contributing to the workshop and helping to define a research agenda for mixed-initiative creative interfaces in this domain and more broadly.