AI agents are becoming a part of our everyday life, thanks to artificial intelligence technologies. Human-AI co creativity involves a human and an AI collaborating on creative tasks as partners [1]. Rather than being perceived as a support tool, AI agents in co-creative systems should be regarded as a co-equal partner. This field has the potential to transform how people perceive and interact with AI. A study showed that AI ability alone does not ensure a positive collaborative experience of users with the AI [2]. In recent years, researchers have designed many co-creative systems with powerful AI ability, yet sometimes users fail to maintain their interest and engagement while collaborating with the AI due to the quality of the collaboration and interaction. The literature asserts that user engagement is associated with the way users interact with a system [3]. Interaction design is often an untended topic in the co-creativity literature despite being a fundamental property of co-creative systems. Bown asserted that the success of a creative system's collaborative role should be further investigated through interaction design as interaction plays a key role in the creative process of co-creative systems [4]. Therefore, as a young field, there are potential areas of interaction design to be explored for designing effective co-creative systems that engage users and provide a better collaborative experience.
With the intention of investigating the trends in the interaction design of the existing co-creative systems, we utilized the archival website, "Library of Mixed-Initiative Creative Interfaces" (LMICI), which archives 74 existing co-creative systems [5]. Angie Spoto and Natalia Oleynik created this archive after a workshop on mixedinitiative interfaces led by Deterding et al. in 2017 [6,5]. The archive provides the corresponding literature and other relevant information for each of the systems. We analyzed the interaction designs of the co-creative systems present in the archive. Apparently, most of the cocreative systems use instructing interaction [7], where users provide instructions to the system using buttons, sliders, or text to communicate directly with the AI (other than communicating through the creative product). However, using buttons and sliders, users can communicate with the AI in a very constrained and minimal way in most of the systems. Very few systems use text, voice, or embodied communication for user to AI direct communication during a co-creation to provide information to the AI, give feedback to the AI, etc. For example, Image to Image [8] is a co-creative system that converts a line drawing of a particular object from the user into a photo-realistic image. The user interface has only one button that users click to tell the AI to convert the drawing. Other than the button, there is no way of communicating with the AI to provide information, suggestion or feedback. In a human collaboration, collaborators communicate to provide feedback and convey important information to each other and is a major component of the mechanics of co-creation [9]. The literature about human-AI co-creation says that embodied communication improves coordination between the human and the AI [10]. Additionally, literature asserts that a communication channel for conversation between co-creators other than communicating through the shared creative product improves user engagement in a human creative collaboration [11]. These literatures led us to investigate the impact of embodied communication from the AI and a conversation between the human and AI on user engagement and collaborative experience in human-AI co-creativity. Our research questions emerged from the issue that most existing co-creative systems use instructing interaction type, which uses one-way communication, human to AI. For this work, we will investigate the impact of conversing interaction and AI embodiment on user engagement, user perception about the AI and the overall collaborative experience. The two research questions we have are-• How does AI embodiment and conversing interaction influence user engagement? • How does AI embodiment and conversing interaction influence user perception about the AI agent as the collaborative partner and the overall collaborative experience?
For investigating the research questions, we have developed a prototype of a co-creative system named Creative PenPal where the user and the AI collaborate on a design ideation task. Users can generate ideas for designing a particular object by sketching on a canvas, and the AI will also contribute to the design ideation by showing different inspirational sketches. Creative PenPal utilizes a conversing interaction for the communication between the human and the AI. Additionally, a virtual embodied character for the AI agent is utilized. For investigating the research questions, we describe the study design in the paper. By the time of the workshop, we will have the data and insights from the study.
Louie et al. identified that AI ability alone does not ensure a positive collaborative experience of users with the AI [2]. Bown asserted that the success of a creative system's collaborative role should be further investigated in terms of interaction design as interaction plays a key role in the creative process of co-creative systems [4]. Later Yee-King and d'Inverno argued for a stronger focus on the user experience, suggesting a need for further integration of interaction design practice into human-AI co-creativity research [12].
Interaction types are ways a user interacts with a product or application [13]. Instructing interaction is where users issue instructions to a system. This can be done in many ways, including typing in commands, selecting options from menus or on a multitouch screen, pressing buttons or using function keys, etc. In contrast, the conversing interaction type is where users have a dialogue with a system. Users can speak via an interface or type in questions or answers to which the system replies via text or speech output [13]. Conversational agents have transitioned into multiple industries with increased ability for user engagement in intelligent conversation.
The literature asserts that embodied communication aids synchronization and coordination in improvisational human-computer co-creativity [10]. Being able to converse with each other shows an increased engagement level in a human creative collaboration [11]. A user's confidence in an AI agent's ability to perform tasks is improved when imbuing the agent with embodiment and social behaviors compared to the agent solely depending on conversation [14]. Bente et al. reported that embodied telepresent communication improved both social presence and interpersonal trust in remote collaboration settings with a high level of nonverbal activity [15].
User engagement with virtual embodied conversational agents can be measured via user self-reports; by monitoring the user's responses, tracking the user's body postures, head movements and facial expressions during the interaction, or by manually logging behavioral responses of user experience [16]. Carrol and Latulipe proposed a quantitative and psychometric survey, called Creativity Support Index (CSI), to assess a tool's creativity support by measuring six dimensions of creativity via self reports: Exploration, Expressiveness, Immersion, Enjoyment, Results Worth Effort and Collaboration [17].
Creative PenPal is an interactive prototype, created with Javascript, which has all the interaction components except the back-end AI model. We have selected a collection of sketches as the database for creating a seamless experience that mimics an actual implementation of the AI model. The sketch generation is automated where the system selects sketches from the collection. We have two versions of the Creative PenPal prototype to investigate and compare the user engagement and collaborative experience between the two versions. The original version uses a conversing interaction and a virtual embodied AI (see Figure 1). The virtual embodied AI character, a pencil, is shown in section A of Figure 1. We will address the AI character as PenPal in the rest of the paper. Section B is where the conversation happens between the PenPal and the user via text and buttons. We can see the design task displayed in section C. Both the user and the AI collaborate in a design ideation task where both collaborators generate ideas for the design of an object as sketches. Users will design the specified object in the task by sketching on the canvas shown in section F. Users can undo a stroke using the "Undo Previous Sketch" button and start the design ideation over by using the "Clear the canvas" button. When users hit the "Inspire me" button shown in section B, the virtual AI character will show an inspirational sketch of a conceptually similar object, an object that have similar working mechanism or usage as the design task object, on its canvas shown in section G. Previous work on co-creative design ideation [18] showed that users were more inspired by conceptually similar objects than visually similar objects that share structural similarity as the design task object. Users can also ask for visually similar objects or sketches of the design task object to get inspiration by saying they didn't like the conceptually similar object (described in the next section). Section E shows the name of the object located in the PenPal generated sketch. The other version uses an instructing interaction where users can instruct the AI using buttons without AI embodiment (Figure 2). We will use both of these two versions to compare the impact of two different interaction designs on user engagement and collaborative experience with an AI.
For the interaction model, we choose a conversing interaction. The conversation with the virtual embodied AI is simple so that the user will be able to go deeper into the ideation process without any interruption in the design flow. The embodied virtual agent will show some affective characteristics, for example, when the user likes its contribution, it will be seen as happy and when the user does not like the contribution from the AI, it will be sad. The conversation is divided into five different situational phases demonstrated in Figure 2. Each phase includes the embodied state of the AI and conversational interaction between the user and the PenPal. The text without a comment bubble represents the embodied state of the AI in Figure 2. The texts with comment bubbles represent dialogues of the user and the AI, and the icon indicates which dialogue belongs to whom. Different responses from the user initiate another phase, which is shown using arrows in Figure 2. If the user can respond with different options, "/" sign is used in the Figure.
This phase will start when the user starts the design task. PenPal will introduce itself and ask the users if they want to see an inspirational sketch from the AI by saying "Hi! I am your Creative PenPal. Do you want me to inspire you?". Users can respond immediately by pressing the button "Inspire me" or they can keep ideating by sketching and respond later.
When the user hits the button "Inspire me" indicating the desire to see an inspirational sketch, the PenPal will move to the canvas and generate a sketch. The PenPal will ask the user whether they liked the sketch or not. The user can reply with the "Yes" button or the "No" button. This phase is for collecting user preferences.
When users select the "Yes" button in response to Pen-Pal's question to determine whether the user liked the sketch or not, it means the sketch inspired the user in
When users click the "No" button, indicating that Pen-Pal's generated sketch did not inspire them, PenPal arrives with a sad face and says, "Sorry that I could not inspire you!" (left side of Figure 4, the gree arrow indicates transition). Then it suggests the user ask for specific types of objects as inspiration by saying, "Let's try to be more specific about what you want me to inspire with" (Right side of Figure 4). The user can respond witree options, "Design Task Objects" (as our design task object is shopping cart, the button says "Shopping Carts"), "visually similar objects", or "conceptually similar objects". Visually similar objects have visual structural similarity as the design task object and conceptually similar objects have similar working mechanism or usage as the design task object. When the user clicks any of these three buttons, the PenPal will generate a sketch accordingly.
When the user finishes the design ideation sketching, they let the virtual agent know by clicking the "Finish ideation" button. The virtual agent arrives and greets the user for completing the design ideation task by saying, "Well done! You did a great job! ".
The user experiment will take place virtually. We will use Google Meet to connect with the study participants.
The target sample size for the study is 50 participants including 25 males and 25 females as participants. This study will use a between-subject study where one group of participants will test the version with instructing interaction and without any embodied AI character. The other group will test the version with the conversing interaction and a virtual embodied AI agent. The study will start with a short pre-study survey to collect some demographic information about the participants, for example, gender, age-range, drawing/ sketching skills, etc. Then, the participant will carry out the design task using either one version of the Creative PenPal. The task for this study is-"Ideate the design of a shopping cart for the elderly within 20 minutes. You must include three design inspirations from the AI in the design". The whole task will be screen recorded. After the task, the participants will fill out Creativity Support Index (CSI), which is a well known psycho metric survey, for measuring six dimensions of creativity: Exploration, Expressiveness, Immersion, Enjoyment, Results Worth Effort and Collaboration [17] to evaluate user engagement, collaboration and immersion. After that, a retrospective think-aloud will be conducted as the participants watch the screenrecording video of the task to understand the rationale behind the user interaction process and user experience. The study will end with a follow-up semi-structured interview to determine in depth qualitative data about the user engagement and overall experience with the AI.
In the young yet fast-growing field of human-AI cocreativity, attention is needed to design human-centered co-creative systems where users are engaged in a successful collaborative experience. Interaction design where users can communicate with the AI for providing user preference improves the collaborative experience and user attitude towards the AI [2]. Conversing virtual agents have transitioned into services such as ecommerce, leading to an increased ability for user engagement. In a conversing interaction, users can provide their feedback on the AI's contribution, which provides more information to the AI about user preferences. Conversing interaction also helps users perceive the AI as a partner rather than a tool. A user's confidence in an AI agent's ability to perform tasks is improved when imbuing the agent with embodiment and social behaviors compared to the agent solely depending on conversation [14]. The embodiment improves the user perception of an AI agent in terms of a collaborative partner, an entity. Users also tend to trust an AI in terms of their ability when they can see their presence. The embodiment also helps design affective AI where an AI's feelings are visible in its expression or gesture. As a young field and new research area, interaction design is rarely discussed in the existing literature despite being a fundamental property of an adequate co-creative system. An adequate interaction model dramatically improves the quality of the collaboration and engages users.
Investigating the impact of conversing interaction and AI embodiment for designing effective co-creative systems that engage users is essential. We develop the prototype of Creative PenPal as an effort to explore the impact of a conversing embodied co-creative AI agent on user engagement, user perception and overall collaborative experience. We describe the study design that will provide insights for designing effective co-creative systems that engage users and improve their collaborative experience with the AI agent. With the insights and results from the study, we will improve the interaction design of Creative PenPal and implement the AI model in the improved prototype. At the time of the workshop, we will have the data and insights in our hands from the study, revealing the impact of the interaction model we used. During the workshop, we will be able to demonstrate the results and insights.