=Paper=
{{Paper
|id=Vol-2903/IUI21WS-HAIGEN-6
|storemode=property
|title=CreativeSVG: A Creativity Support Tool for Abstract Background Design with Generative Vector Graphics
|pdfUrl=https://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-6.pdf
|volume=Vol-2903
|authors=Yucheng Jin,Ge Gao,Tiffany Ong,Xi Su
|dblpUrl=https://dblp.org/rec/conf/iui/JinGOS21
}}
==CreativeSVG: A Creativity Support Tool for Abstract Background Design with Generative Vector Graphics==
https://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-6.pdf
CreativeSVG: A Creativity Support Tool for
Abstract Background Design with Generative
Vector Graphics
Yucheng Jina , Ge Gaoa , Tiffany Ongb and Xi Sua
a Lenovo Research, Beijing, China
b Stanford University, San Jose, CA
Abstract
Great graphic designs do not just come out overnight; they take many iterations to polish the design.
Designers create different versions of their work and compare the versions in iterations. However, the
exploration of different design is time-consuming and tedious. This paper presents a design tool, Cre-
ativeSVG, that aims to facilitate the ideation and iteration of abstract background design by two features:
1) the variations of a specific graphic for the selected design features 2) the transition between two cho-
sen graphics. Preliminary user feedback shows that our tool augments user creativity and increases
design efficiency to a certain extend.
Keywords
Augmented human, graphic design, creativity support tools, automated design
1. INTRODUCTION Computer-based design programs have been
shown to augment human creativity. Koch
We believe that creativity is an important cri- et al. [2] present an interface that supports
terion to assess the graphic design. To in- ideation in creating mood boards. Kazi et
crease design creativity, designers usually ex- al. [9] present SandCanvas, a multi-touch art
plore and compare different design possibili- medium that enhances users’ creativity in cre-
ties, try different paths and styles, and iterate ating sand animations. Color Builder [7] al-
the design [1]. Creating multiple versions of lows designers to choose and experiment with
design allows designers to compare different different color schemes. SketchPlore [5] infers
versions and ultimately choose the best one. a designer’s task and suggests improvements
However, this process is time-consuming and without overriding the designer. These sys-
tedious. A number of tools have been devel- tems fall under the umbrella of Creativity Sup-
oped to facilitate the specific tasks of design port Tools [3], interfaces designed to enhance
process such as ideation [2, 3], layout choos- creativity by including elements such as sup-
ing [4, 5, 6], and color picking [7, 8]. port for exploration [1]. Previous work has
automated different aspects of the design pro-
Joint Proceedings of the ACM IUI 2021 Workshops, April cess. However, there is little work automat-
13-17, 2021, College Station, USA ing the graphic design and even less work
" jinyc2@lenovo.com (Y. Jin); gaoge1@lenovo.com (G. that discusses increasing designers’ creativ-
Gao); tiffanyo@stanford.edu (T. Ong);
suxi2@lenovo.com (X. Su)
ity through automation. We try to facilitate
two design processes, ideation and iteration
© 2020 Copyright for this paper by its authors. Use permit-
ted under Creative Commons License Attribution 4.0 Inter- while allowing users to be creative and pro-
CEUR
national (CC BY 4.0).
CEUR Workshop Proceedings duce novel designs through automated design
http://ceur-ws.org
(CEUR-WS.org)
Workshop ISSN 1613-0073
Proceedings
techniques. tion that allows designers to simultaneously
Creating various design alternatives is a explore design alternatives based on four de-
crucial part of the design iteration. This itera- sign features such as color, layout, and shape.
tive process enhances creativity by allowing The second part is a panel that presents transi-
designers to “exhaust the obvious and explore tionary designs between two selected designs.
new ideas” [10]. Dow et al. [11] showed that
creating multiple design alternatives in par-
allel produces higher-quality, more-diverse 2. SYSTEM DESIGN AND
work than sequentially. It has been demon- IMPLEMENTATION
strated that an automated presentation of de-
sign alternatives improves design quality. Some 2.1. Working Flow of Generating
systems [12, 4] suggest layouts for placing Graphics
text and images to designers. Users who were
given suggestions produced higher quality de- Figure 2 illustrates the working flow of Cre-
signs than those without suggestions. Lee et ativeSVG. First, to leverage a designer’s prior
al. [13] showed that when designers were pre- knowledge and experience, we asked experi-
sented with layout examples of a webpage’s enced graphic designers to define three key
design, they created higher-quality designs. graphic design elements: color, shapes, and
Additionally, we have seen the automated de- layout. As a result, we have 26 good color
sign for small graphics, such as thumbnails for schemes, 243 elementary shapes (Figure 1),
videos [14] and automated layouts for visual- and six layout principles such as balance, align-
textual presentations, such as for magazine ment, proximity. The whole working flow
covers [6]. However, the ease of the existing consists of seven steps: 1) following the prede-
automated design often comes at the cost of fined source materials and principles, the gen-
lacking creativity support. These programs erator randomly produces design data sets; 2)
often eliminate the tedious aspects of design a translation tool generates SVG based graph-
but do not encourage creative exploration. ics according to the data sets produced in the
The design examples play a critical role in first step; 3) designers annotate good graphics
the process of creative design [15]. We have and add four pairs of tags such as bright ver-
seen several design tools [16, 17] that sup- sus dark, soft versus sharp; 4-5) then we em-
port exploration of various generative design ploy generative adversarial networks (GANs) [19]
examples by tweaking the system parameters to train a model that fits the positive samples;
in the tools. Before designing CreativeSVG, 6) based on user-specified tags and system-
we had conducted a preliminary user study generated random noise, the model generates
to understand the pain points in graphic de- feature data for rendering graphics; 7) a trans-
sign. Based on the study results, we find lation tool translates feature data to resulting
that designers expect these tools to help graphics.
them in the phases of ideation and iter-
ation rather than automatically generat- 2.2. Algorithm of Generating
ing a final work.
Graphics
This tool employs Generative Adversarial
Networks (GANs) [18] to generate abstract In general, GANs [18, 20] are a kind of neural
graphics according to user specifications. The network that mimics data distribution. When
tool consists of two parts for exploring design we feed data sets to GANs, after training, GANs
alternatives; the first part is a novel visualiza- generate similar (but different) data to fit the
Figure 1: The elementary shapes are categorized into three types of shape: rectangle, triangle, and
circle. Each type of shape contains shapes which are positioned in a 9*9 matrix with smooth changes
by small gradual steps along two dimensions.
Figure 2: The work flow of automatic generation of design.
data set we provided. If we describe a good generate rasterized graphics take RGB data of
graphic by structural data, then GANs can each pixel as feature data, which leads to very
generate similar good graphics. high dimension data and may fail to fit data
The existing approaches of using GANs to distribution when the variety of graphics is
great. Therefore, instead of generating raster-
ized graphics, we propose a new approach us-
ing GAN to generate Scalable Vector Graphics
(SVG) based on our predefined features crucial
to graphic design, such as shape type, layout,
color, scale, rotation, etc. Previous work have
shown the potential of using GAN to gener-
ate simple SVGs, however, these generated
SVG having relatively simple patterns such as
fonts [21] and sketches [22].
Furthermore, we want to generate graph-
ics that are similar to the graphics we feed, Figure 3: The configuration page of specifying
but we also want to iterate the design as in conditions for graphic generation.
real design scenarios. To this end, we use
conditional GANs [19], which allows users to
specify conditions to adjust the generation of
graphics. These conditions can be indicated tion (high, low, both), and value (bright, dark,
by the tags of graphics, for instance, “warm”, both). Users can also select up to three types
“cold”, “sharp”,“soft”. of shape types (circular, square-like, triangu-
After passing tags to conditional GANs to lar). Each time a user changes a preference,
train the model, we can control the outputs. the system automatically presents 20 sample
Here we use a vector with 39 numbers as an graphics according to the new chosen pref-
input of the model. The first nine numbers erences. These samples are displayed on the
are conditions controlling visual style, and right side of the screen. Once the user is sat-
the rest 30 numbers are random numbers to isfied with these samples, she can click “See
keep a variety of outputs (Figure 2(6)). To More Designs.”
enable design iteration, we can maintain the
random numbers the same and change a spe- 2.3.2. Exploration Page
cific condition value, which generates similar After selecting a graphic from the configura-
graphic designs but with smooth nuance in tion page, the Exploration page (Figure 4) is
that condition. meant to encourage creative exploration and
ease of iteration. The left side of the screen
2.3. User Interface Design displays 100 graphics. These graphics have
the design constraints specified from the first
We next present a high-level view of the de-
page. Users can click “More” to load 100 more
sign interface. The user interface aims to pro-
designs with the same constraints or any of
vide users with a tool to discover and select
the designs that interest them. Clicked de-
various generated designs.
signs will show up on the right side of the
screen, surrounded by different iterations of
2.3.1. Configuration Page this design. These iterations are ordered in
In Figure 3, users can see samples of generated different axes, from left to right, top to bot-
designs based on their chosen preferences. tom, top-left to bottom-right, and top-right
Users can toggle three variables. These vari- to bottom-left. The axes titles include “Bright
ables include hue (cool, warm, both), satura- to Dark,” “Disorderly to Orderly,” “Sharper to
Figure 4: The page of exploring different graphics and their variation for different design features.
Softer,” and “Simple to Complex,” according resenting the gradual transition from the first
to the different types of iteration presented. clicked design to the second will show up on
These iterations represent slightly changed the right side. Users can click any of these
designs and mimic the various ways in which transitionary designs to enlarge it and click
a designer would change an original design “Add to Favorites” to add the design to the
during the iterative process. Users can click same “Favorites” section as in the Exploration
on any of these iterated designs to enlarge Page. Users can click “Cancel” to deselect any
them. They can then click “Add to Favorites” enlarged designs and select two new designs
to add the design to the “Favorites” section to start the process over again. Users can click
located at the bottom left of the screen. Each “Back” to be taken back to the Exploration
graphic in the favorites section can be down- Page.
loaded as an SVG, ready to be edited in any
applicable editing software. Users can also
click “Back” to be taken back to the first page 3. PRELIMINARY
to change graphic preferences. FEEDBACK
2.3.3. Transition Mode Page We recruited 15 designers in our company to
try this design tool and interview them with
Clicking “Transition Mode” takes the user to two questions regarding creativity support
another page for exploration. In this page, and the opinions on the presented features.
users click on two designs from the left side Thirteen out of fifteen designers stated that
of the screen (Figure 5). Several graphics rep-
Figure 5: The page of showing transitionary designs between the two selected graphics.
they found the auto-generation of designs ef- most enjoyed the download function as it al-
fectively helped their design process. Eight lowed them to edit the design in their soft-
of them noted the convenience of having de- ware.” One liked the transition tool because
signs generated for them as it “saved time, es- of “I can see even more backgrounds, on top of
pecially given the limited time considerations” the ones that I already like” (P9). One enjoyed
(P4). Many of them felt that in addition to the agency of choosing colors and shapes:
helping saving time, CreativeSVG helped in- “I liked choosing the shape preferences, and I
spired new ideas: “It improved my design qual- would like to see more shapes” (P11). Although
ity and stimulated ideation” (P5). Users also many participants were pleased with the final
saw the value in having time to focus their generated posters, many felt that the quality
efforts on creating rather than ideating: “It of generated designs could be improved. One
saved me time, and so I had more time to try was happily surprised by the random arrange-
more ideas” (P6). However, one person noted ments, but another noted that the designs
that it was a complicated process to “choose seemed random, without meaning. Many felt
through thousands of pictures” (P14). that the toggling of preferences did not fully
Five enjoyed the Exploration Page because generate different designs: “There is no clear
it allowed them to see many different designs difference in graphics when I choose different
at once: “I like the explore function which can color preference and shape preferences.” (P11).
explore different types of patterns” (P1). Three Some of them expressed a desire for greater
control over generated designs. [3] B. Shneiderman, Creativity support
tools: Accelerating discovery and inno-
vation, Communications of the ACM 50
4. CONCLUSION (2007) 20–32.
[4] P. O’Donovan, A. Agarwala, A. Hertz-
In this paper, we first present a novel system
mann, Designscape: Design with inter-
that aims to help designers create creative
active layout suggestions, in: Proceed-
graphics with less effort. Compared with the
ings of the 33rd annual ACM conference
traditional approach of applying GANs on
on human factors in computing systems,
pixel-based data, we proposed a new way to
ACM, 2015, pp. 1221–1224.
leverage GANs to generate graphics based
[5] K. Todi, D. Weir, A. Oulasvirta, Sketch-
on graphics’ low dimension feature data. We
plore: Sketch and explore with a layout
then evaluated our system CreativeSVG with
optimiser, in: Proceedings of the 2016
15 professional designers. The results indicate
ACM Conference on Designing Interac-
that our tool can increase design efficiency
tive Systems, ACM, 2016, pp. 543–555.
and augment creativity to some extends. How-
[6] X. Yang, T. Mei, Y.-Q. Xu, Y. Rui, S. Li,
ever, the current system can only generate
Automatic generation of visual-textual
graphics using our predefined shapes, which
presentation layout, ACM Transactions
may influence the variety of designs. We will
on Multimedia Computing, Communi-
improve our algorithm to increase the variety
cations, and Applications (TOMM) 12
of shapes.
(2016) 33.
[7] M. Shugrina, W. Zhang, F. Chevalier,
Acknowledgments S. Fidler, K. Singh, Color builder: A di-
rect manipulation interface for versatile
This research was partially supported by Spe- color theme authoring, in: Proceedings
cial Fund of Beijing Industrial Design Cen- of the 2019 CHI Conference on Human
ter. We thank our colleagues from Lenovo Factors in Computing Systems, ACM,
Research (Xu Zhao, Jie Yin, Xi Wan, Zhi Yang, 2019, p. 456.
etc.) who provided insight and expertise that [8] S. Lin, P. Hanrahan, Modeling how peo-
greatly assisted the research. ple extract color themes from images,
in: Proceedings of the SIGCHI Confer-
ence on Human Factors in Computing
References Systems, ACM, 2013, pp. 3101–3110.
[9] R. H. Kazi, K. C. Chua, S. Zhao, R. Davis,
[1] M. Resnick, B. Myers, K. Nakakoji, K.-L. Low, Sandcanvas: a multi-touch
B. Shneiderman, R. Pausch, T. Selker, art medium inspired by sand animation,
M. Eisenberg, Design principles for tools in: Proceedings of the SIGCHI Confer-
to support creative thinking (2005). ence on Human Factors in Computing
[2] J. Koch, A. Lucero, L. Hegemann, Systems, ACM, 2011, pp. 1283–1292.
A. Oulasvirta, May ai?: Design ideation [10] A. E. Arntson, Graphic design basics,
with cooperative contextual bandits, in: Cengage Learning, 2011.
Proceedings of the 2019 CHI Conference [11] S. P. Dow, A. Glassco, J. Kass,
on Human Factors in Computing Sys- M. Schwarz, D. L. Schwartz, S. R.
tems, ACM, 2019, p. 633. Klemmer, Parallel prototyping leads to
better design results, more divergence,
and increased self-efficacy, ACM neural information processing systems,
Transactions on Computer-Human 2014, pp. 2672–2680.
Interaction (TOCHI) 17 (2010) 18. [19] M. Mirza, S. Osindero, Conditional gen-
[12] P. O’Donovan, A. Agarwala, A. Hertz- erative adversarial nets, arXiv preprint
mann, Learning layouts for single- arXiv:1411.1784 (2014).
pagegraphic designs, IEEE transactions [20] A. Radford, L. Metz, S. Chintala, Unsu-
on visualization and computer graphics pervised representation learning with
20 (2014) 1200–1213. deep convolutional generative adver-
[13] B. Lee, S. Srivastava, R. Kumar, R. Braf- sarial networks, arXiv preprint
man, S. R. Klemmer, Designing with arXiv:1511.06434 (2015).
interactive example galleries, in: Pro- [21] R. G. Lopes, D. Ha, D. Eck, J. Shlens, A
ceedings of the SIGCHI conference on learned representation for scalable vec-
human factors in computing systems, tor graphics, in: Proceedings of the IEEE
ACM, 2010, pp. 2257–2266. International Conference on Computer
[14] B. Zhao, S. Lin, X. Qi, Z. Zhang, X. Luo, Vision (ICCV), 2019.
R. Wang, Automatic generation of [22] A. Das, Y. Yang, T. Hospedales, T. Xiang,
visual-textual web video thumbnail, in: Y.-Z. Song, Béziersketch: A generative
SIGGRAPH Asia 2017 Posters, ACM, model for scalable vector sketches, in:
2017, p. 41. The European Conference on Computer
[15] S. R. Herring, C.-C. Chang, J. Krantzler, Vision (ECCV), 2020.
B. P. Bailey, Getting inspired!: under-
standing how and why examples are
used in creative design practice, in: Pro-
ceedings of the SIGCHI Conference on
Human Factors in Computing Systems,
ACM, 2009, pp. 87–96.
[16] J. Matejka, M. Glueck, E. Bradner,
A. Hashemi, T. Grossman, G. Fitzmau-
rice, Dream lens: Exploration and visual-
ization of large-scale generative design
datasets, in: Proceedings of the 2018
CHI Conference on Human Factors in
Computing Systems, 2018, pp. 1–12.
[17] L. Zaman, W. Stuerzlinger, C. Neuge-
bauer, R. Woodbury, M. Elkhaldi,
N. Shireen, M. Terry, Gem-ni: A system
for creating and managing alternatives
in generative design, in: Proceedings of
the 33rd Annual ACM Conference on
Human Factors in Computing Systems,
2015, pp. 1201–1210.
[18] I. Goodfellow, J. Pouget-Abadie,
M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, Y. Bengio, Gener-
ative adversarial nets, in: Advances in