Ongoing advances in generative AI systems have sparked great interest in using them interactively in creative contexts and for digital content creation and manipulation: Some examples include (1) generating or modifying images with generative adversarial networks (GANs) [1,2,3], (2) generating controllable movements for virtual characters with recurrent neural networks, deep reinforcement learning and physics simulations [4], and (3) controllable machine capabilities for generating or summarizing when working with text [5,6]. Such computational methods have also entered specifically artistic domains, including visual art [7], creative writing and poetry [8,9]. More examples can be found in a curated "ML x Art" list 1 .
A common vision, also present in the call for this workshop, paints a picture of creative human use of such AI as tools. In this view, these new interactive systems are hoped to realise key ideas from creativity support tools (CST, [10]) by leveraging AI capabilities. More specifically, this support could cast humans and AI in many different roles (for a recent overview see [11]). This includes, for example, using AI as a divergent or convergent agent, as described by Hoffmann [12], that is, to generate or evaluate (human) ideas. Related, Kantosalo and Toivonen [13] highlight alternating cocreation, with the AI "pleasing" and "provoking" the user. Moreover, Negrete-Yankelevich and Morales-Zaragoza [14] describe a related set of roles, including AI as an "apprentice", whose work is judged and selectively chosen by humans, or a leader-like role, which only leaves final configurations to the user.
Within this range of roles, the workshop call emphasises the generative capabilities of AI. In this paper, we thus focus on the role of AI as a generator, and the underlying goal of freeing its users to focus on a larger creative vision, while the AI takes care of more tedious steps.
With this goal in mind, this paper examines potential pitfalls on the way towards achieving it in practice. Our research approach is related to work on dark patterns in UI/UX design [15], which also examines -sometimes speculatively [16], sometimes empirically [17] -what "could go wrong", in order to ultimately inspire directions for interaction design that are beneficial to the users' interests. In doing so, we thus hope to contribute to a critical and constructive discussion on the roles of humans and AI in co-creative interactions, with an eye on related assumptions and potential side-effects for creative practices and beyond.
Our interest in collecting pitfalls is inspired by work on dark patterns [16,17,15]: Both pitfalls and dark patterns identify issues with user interfaces and interactions that result in experiences or outcomes which might not be in the user's best interest. However, in contrast to what is often assumed in dark patterns, pitfalls do not imply bad intention, rather oversight or lack of information 2 .
Concretely, related work collected speculative dark patterns for explainability, transparency and control in intelligent interactive systems [16] by transferring dark patterns previously described for UI/UX de-sign [15]. Other work collected dark UI/UX patterns empirically by reviewing a large set of existing mobile applications [17]. Both approaches seem challenging to directly transfer to collecting pitfalls in the context of cocreative generative AI, since there are no previously defined pitfalls and no easily accessible collections (or "app stores") of many usable applications for review.
Therefore, we followed a qualitative, speculative approach and brainstormed on potential pitfalls, or "what could go wrong" (cf. [18]) in interactions with co-creative AI. Here we are loosely inspired by aspects of speculative design [19], although that area typically aims to address broader issues than what we focus on here. Further inspiring "speculative futures" for human-AI co-creative systems, along with a conceptual framework, can be found in the work by Bown and Brown [20].
We particularly explore issues grounded in today's interactions and UIs, which can be reasonably well imagined to potentially occur with the current state of the art of generative AI models. In particular, our brainstorming started from three prompts: (1) Issues arising from currently limited capabilities of AI, and (2) from exploring what might happen with too much AI involvement; plus (3) thinking beyond use and usage situations. Considering this approach, we see the pitfalls presented here not as a comprehensive and "definitive" list but rather as a stimulus for discussion in the research community -at the workshop and beyond.
Table 1 shows the pitfalls we collected. In particular, we present nine pitfalls, three for each of our starting prompts, that is, for limited AI (pitfalls 1-3), too much AI involvement (pitfalls 4-6), and for aspects beyond use (pitfalls 7-9).
Overview of the collected pitfalls. Additionally, Figure 2 visualises one example for each of the categories "Limited AI", "Too much AI" and "Beyond use".
The table characterises each pitfall with a name, affected aspects (categories), a description of the problem, and a concise pitfall "vignette": This includes an example scenario describing a system in which this issue arises, along with an illustrating diagnosis of how this might have happened in the design and development of said system, plus corresponding ideas for potential solutions or open questions. For each category of pitfalls ("limited AI", "too much AI", and "beyond use") we picked one example for further illustration in Figure 2.
As an additional overview, Figure 1 locates these pitfalls within an interaction loop in human-AI co-creative systems; the loop is taken from a framework by Guzdial and Riedl [27]. In this figure we illustrate our underlying mental model of human-AI interaction. It consists of the user and the AI as potential actors collaborating on a shared artifact. The AI can get involved in the creation process in one of two ways: It can either be prompted to contribute through the user interface (e.g. using a predefined function to achieve an image manipulation) or it can act without a (user) prompt, e.g. to suggest edits or flag errors. We further include the training data in this model, as it provides the basis for the AI's actions and decisions. While we located the pitfalls within this model, these locations are by no means the only possible ones. They represent our interpretations of which point in the interaction loop is most likely affected by each pitfall. As an example, a lack of expressive interaction may not only be rooted in the user interface, but can also be caused by insufficient training data to support more meaningful options.
While Table 1 lists concrete example problems, here we reflect more broadly on the consequences of such pitfalls for co-creative generative systems. In particular, we see two broad directions -overt and covert consequences.
First, users might be annoyed, distracted, or otherwise put off by bad user experiences due to these pitfalls. For example, cases where the AI directly overwrites the user (pitfall 4), or distracts the user from their productive task (pitfall 6) might be particularly harmful in this regard. Observing AI failures might lead to algorithm aversion, as described by Dietvorst et al. [29]. In these cases, users might avoid future use of such systems.
In contrast, users might also be affected negatively without noticing it. For example, this might be the case if the AI implies invisible boundaries (pitfall 1) that hinder creative exploration. Similarly, "silent" issues might result from the generative AI introducing incorrect information (pitfall 3), distractions (pitfall 6), or biases and legal issues (pitfalls 7-9). Users might only (much later) stumble across issues in downstream processes, evaluations or reflections. If such issues then affect evaluation of the user's creative work (e.g. due to false information, pitfall 3), this might result in algorithm anxiety, described by Jhaver et al. [30].
Overall, the pitfalls might thus result in a range of possible consequences, from bad user experiences, negative impacts on creative work, abandonment of tools, to broader issues, including privacy related and legal ones. Figure 1: Visualisation of our underlying mental model of the interaction loop in human-AI co-creative systems. We place our identified pitfalls (see Table 1) in this loop based on the position where they most likely occur.
Research and Design of Co-Creative Generative Systems?
Put briefly, this position paper describes what could go wrong in order to stimulate discussions of how to get it right. More concretely, here we describe three potential uses.
The described pitfalls can help researchers and designers to think about a wide range of concrete aspects of interaction and UI design for co-creative generative systems (e.g. temporal and spatial integration of AI actions in UIs). In this way, they may raise awareness for making design choices explicit that might have otherwise not been prominently considered. These design choices could then also be considered in light of relevant frameworks, such as Horvitz' mixed initiative principles [31] or the co-creative framework described by Guzdial and Riedl [27] (cf. Figure 1).
Moreover, the problematic systems described in the pitfalls in Table 1 might inspire informative baseline systems for comparison with (hopefully) better solutions. For example, a typical HCI user study on an AI photo editor might compare an AI vs non-AI version. However, as illustrated with the example for pitfall 5 (Agony of Choice), another insightful evaluation might further use a baseline that involves AI "even more" than the intended design solution to be evaluated.
Evaluating technical systems for creative use is challenging [32], for example, since creativity and quality criteria are often hard to operationalise, and may require interdisciplinary knowledge. Additionally involving AI can be expected to complicate evaluations further. Here, our pitfalls and examples may provide helpful concrete starting points, as a thinking prompt towards developing evaluations that satisfy both HCI and AI inter- nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
My social security number is 078 AI Suggestions:
- -----------------------------------------------------------------05-1120 (b) Example for an AI leaking sensitive information from the training dataset (based on [25]), either as a suggestion or as a response to a primer (enabling active attacks). Such an attack has been demonstrated by Carlini et al. [26].
@ai.draft: First article of an article series about the largest cities in the world.
In this article we will start with the cities Tokyo, Mexico City, and Istanbul.
In our article series, we are going to visit the largest cities in the world. This time we will focus Tokyo, Mexico City, and Istanbul, three very different cultural centers. We start right away with the largest city of these: Tokyo. The Japanese city is considered the largest city in the world in terms of its population. 37.468.000 people are living there.
(c) A text editing tool could offer intelligent features, e.g. drafting paragraphs or completing a sentence. Yet, the AI might not have the capability to refer to sources -to the human it remains unclear if the claims in a text are true. This leads to a false sense of proficiency. Here, the AI drafted a paragraph with claims about Tokyo's approximate population (bold). However, it refers to the metropolitan area, not the city proper. The interface in the figure is inspired by Yang et al. [28]. ests. For instance, readers and workshop participants (with different backgrounds) could think about how they would improve the design -and evaluate it -for a concrete problematic example system in Table 1; and in particular how they might then make explicit and formulate their criteria in these cases.
One may ask if the illustrated issues might simply vanish in future systems that can build on better AI capabilities. Based on our considerations here, we do not expect this to be case: Co-creative systems involving both human and AI actions are not only limited by AI capabilities. We also have to expect problems arising from interaction and UI design as well as from integration into creative human practices. For example, a lack of expressiveness in interactions (pitfall 2) can still cause problems for creative human use, even in a system with a powerful, "perfect" generative model under the hood. In summary, the pitfalls highlight that human-AI co-creative systems sit at the intersection of HCI and AI, and that successful designs need to consider human-centred aspects in the process. Our pitfalls reflect this in their mix of issues relating to interaction, UI and AI. We thus aim to motivate interdisciplinary work on such systems, also regarding research and design methodology.
One vision of interactive use of AI tools in co-creative settings focuses on the role of the AI as a generator that augments what people can achieve in creative tasks. This paper examined potential pitfalls on the way towards achieving this vision in practice, starting from three speculation prompts: Issues arising from (1) limited AI, (2) too much AI involvement, and (3) thinking beyond use and usage situations.
Concretely, we collected a set of nine potential pitfalls (Table 1) and discussed possible consequences and takeaways for researchers and designers along with illustrating examples. With this collection, we hope to contribute to a critical and constructive discussion on the roles of humans and AI in co-creative interactions, with an eye on related assumptions and potential side-effects for creative practices and beyond.
This project is funded by the Bavarian State Ministry of Science and the Arts and coordinated by the Bavarian Research Institute for Digital Transformation (bidt).