Agent to Improve User Engagement and Collaborative Experience in Human-AI Co-Creative Design Ideation

Abstract

In recent years, researchers have designed many co-creative systems that are very promising with a powerful AI, yet some fail to engage the users due to the unimpressive quality of the collaboration and interaction. Most of the existing co-creative systems use instructing interaction where users only communicate with the AI by providing instructions for contribution. In this paper, we demonstrate the prototype of a co-creative system for design ideation, Creative PenPal that utilizes an interaction model that includes human-AI conversing interaction using text and a virtual embodiment of the AI character. We hypothesize that this interaction model will improve user engagement, user perception about the AI, and the collaborative experience. We describe the study design to investigate the impact of this particular interaction model on user engagement and the overall collaborative experience. By the time of the workshop, we will have the data and insights from the study.

Keywords

• Collaborative
• Design Ideation
• or feedback
• In a human collaboration
• collaborators

1. Introduction

AI agents are becoming a part of our everyday life, thanks

to artificial intelligence technologies. Human-AI co creative tasks as partners [ 1 ]. Rather than being perceived as a support tool, AI agents in co-creative systems should be regarded as a co-equal partner. This field has the potential to transform how people perceive and interact with AI. A study showed that AI ability alone does not ensure a positive collaborative experience of users with the AI [ 2 ]. In recent years, researchers have designed many co-creative systems with powerful AI ability, yet sometimes users fail to maintain their interest and engagement while collaborating with the AI due to the quality of the collaboration and interaction. The literature asserts that user engagement is associated with the way users interact with a system [ 3 ]. Interaction design is often an untended mental property of co-creative systems. Bown asserted that the success of a creative system’s collaborative role should be further investigated through interaction design as interaction plays a key role in the creative process of co-creative systems [ 4 ]. Therefore, as a young field, there are potential areas of interaction design to be explored for designing efective co-creative systems that engage users and provide a better collaborative experience. USA the mechanics of co-creation [ 9 ]. The literature about human-AI co-creation says that embodied communication improves coordination between the human and the topic in the co-creativity literature despite being a funda- or embodied communication for user to AI direct comAI [ 10 ]. Additionally, literature asserts that a commu- buttons or using function keys, etc. In contrast, the connication channel for conversation between co-creators versing interaction type is where users have a dialogue other than communicating through the shared creative with a system. Users can speak via an interface or type in product improves user engagement in a human creative questions or answers to which the system replies via text collaboration [ 11 ]. These literatures led us to investi- or speech output [ 13 ]. Conversational agents have trangate the impact of embodied communication from the AI sitioned into multiple industries with increased ability and a conversation between the human and AI on user for user engagement in intelligent conversation. engagement and collaborative experience in human-AI The literature asserts that embodied communication co-creativity. Our research questions emerged from the aids synchronization and coordination in improvisational issue that most existing co-creative systems use instruct- human-computer co-creativity [ 10 ]. Being able to coning interaction type, which uses one-way communica- verse with each other shows an increased engagement tion, human to AI. For this work, we will investigate level in a human creative collaboration [ 11 ]. A user’s the impact of conversing interaction and AI embodiment confidence in an AI agent’s ability to perform tasks is on user engagement, user perception about the AI and improved when imbuing the agent with embodiment and the overall collaborative experience. The two research social behaviors compared to the agent solely depending questions we have are- on conversation [ 14 ]. Bente et al. reported that embodied telepresent communication improved both social • How does AI embodiment and conversing inter- presence and interpersonal trust in remote collaboration action influence user engagement? settings with a high level of nonverbal activity [ 15 ]. • How does AI embodiment and conversing interac- User engagement with virtual embodied conversational tion influence user perception about the AI agent agents can be measured via user self-reports; by monas the collaborative partner and the overall col- itoring the user’s responses, tracking the user’s body laborative experience? postures, head movements and facial expressions during the interaction, or by manually logging behavioral

For investigating the research questions, we have de- responses of user experience [ 16 ]. Carrol and Latulipe veloped a prototype of a co-creative system named Cre- proposed a quantitative and psychometric survey, called ative PenPal where the user and the AI collaborate on a Creativity Support Index (CSI), to assess a tool’s creativdesign ideation task. Users can generate ideas for design- ity support by measuring six dimensions of creativity ing a particular object by sketching on a canvas, and the via self reports: Exploration, Expressiveness, Immersion, AI will also contribute to the design ideation by showing Enjoyment, Results Worth Efort and Collaboration [ 17 ]. diferent inspirational sketches. Creative PenPal utilizes a conversing interaction for the communication between the human and the AI. Additionally, a virtual embodied character for the AI agent is utilized. For investigating 3. Interface the research questions, we describe the study design in the paper. By the time of the workshop, we will have the data and insights from the study.

2. Related Work

Louie et al. identified that AI ability alone does not ensure

a positive collaborative experience of users with the AI [ 2 ]. Bown asserted that the success of a creative system’s collaborative role should be further investigated in terms of interaction design as interaction plays a key role in the creative process of co-creative systems [ 4 ]. Later YeeKing and d’Inverno argued for a stronger focus on the user experience, suggesting a need for further integration of interaction design practice into human-AI co-creativity research [ 12 ].

Interaction types are ways a user interacts with a product or application [ 13 ]. Instructing interaction is where users issue instructions to a system. This can be done in many ways, including typing in commands, selecting options from menus or on a multitouch screen, pressing

Creative PenPal is an interactive prototype, created with

Javascript, which has all the interaction components except the back-end AI model. We have selected a collection of sketches as the database for creating a seamless experience that mimics an actual implementation of the AI model. The sketch generation is automated where the system selects sketches from the collection. We have two versions of the Creative PenPal prototype to investigate and compare the user engagement and collaborative experience between the two versions. The original version uses a conversing interaction and a virtual embodied AI (see Figure 1). The virtual embodied AI character, a pencil, is shown in section A of Figure 1. We will address the AI character as PenPal in the rest of the paper. Section B is where the conversation happens between the PenPal and the user via text and buttons. We can see the design task displayed in section C. Both the user and the AI collaborate in a design ideation task where both collaborators generate ideas for the design of an object as sketches. Users will design the specified object in the task C

D by sketching on the canvas shown in section F. Users can undo a stroke using the ”Undo Previous Sketch” button and start the design ideation over by using the ”Clear the canvas” button. When users hit the ”Inspire me” button shown in section B, the virtual AI character will show an inspirational sketch of a conceptually similar object, an object that have similar working mechanism or usage as the design task object, on its canvas shown in section G. Previous work on co-creative design ideation [ 18 ] showed that users were more inspired by conceptually similar objects than visually similar objects that share structural similarity as the design task object. Users can also ask for visually similar objects or sketches of the design task object to get inspiration by saying they didn’t like the conceptually similar object (described in the next section). Section E shows the name of the object located in the PenPal generated sketch. The other version uses an instructing interaction where users can instruct the AI using buttons without AI embodiment (Figure 2). We will use both of these two versions to compare the impact of two diferent interaction designs on user engagement and collaborative experience with an AI.

4. Interaction Model

For the interaction model, we choose a conversing in

teraction. The conversation with the virtual embodied AI is simple so that the user will be able to go deeper into the ideation process without any interruption in the design flow. The embodied virtual agent will show some afective characteristics, for example, when the user likes its contribution, it will be seen as happy and when the user does not like the contribution from the AI, it will be sad. The conversation is divided into five diferent situational phases demonstrated in Figure 2. Each phase includes the embodied state of the AI and conversational interaction between the user and the PenPal. The text without a comment bubble represents the embodied state of the AI in Figure 2. The texts with comment bubbles represent dialogues of the user and the AI, and the icon indicates which dialogue belongs to whom. Diferent responses from the user initiate another phase, which is shown using arrows in Figure 2. If the user can respond with diferent options, “/” sign is used in the Figure.

4.1. PenPal Introduction

This phase will start when the user starts the design task. PenPal will introduce itself and ask the users if they want to see an inspirational sketch from the AI by saying ”Hi! I am your Creative PenPal. Do you want me to inspire you?”. Users can respond immediately by pressing the button ”Inspire me” or they can keep ideating by sketching and respond later.

4.2. PenPal Generating Sketch and

Collecting User Preferences

When the user hits the button ”Inspire me” indicating the desire to see an inspirational sketch, the PenPal will move to the canvas and generate a sketch. The PenPal will ask the user whether they liked the sketch or not. The user

can reply with the ”Yes” button or the ”No” button. This phase is for collecting user preferences.

4.3. User Liked PenPals Sketch

When users select the ”Yes” button in response to PenPal’s question to determine whether the user liked the sketch or not, it means the sketch inspired the user in

their design ideation. The PenPal will arrive with a happy including 25 males and 25 females as participants. This face and say, ”I am glad that you liked the sketch! Let me study will use a between-subject study where one group know if you want to see another inspirational sketch as of participants will test the version with instructing inan idea”. If The user wants to see an inspiration again, teraction and without any embodied AI character. The they will select the ”Inspire me” button. other group will test the version with the conversing interaction and a virtual embodied AI agent. The study 4.4. User Did not Like PenPals Sketch will start with a short pre-study survey to collect some When users click the ”No” button, indicating that Pen- demographic information about the participants, for exPal’s generated sketch did not inspire them, PenPal ar- ample, gender, age-range, drawing/ sketching skills, etc. rives with a sad face and says, ”Sorry that I could not Then, the participant will carry out the design task using inspire you!” (left side of Figure 4, the gree arrow indi- either one version of the Creative PenPal. The task for cates transition). Then it suggests the user ask for specific this study is- “Ideate the design of a shopping cart for types of objects as inspiration by saying, ”Let’s try to be the elderly within 20 minutes. You must include three more specific about what you want me to inspire with” design inspirations from the AI in the design”. The whole (Right side of Figure 4). The user can respond witree task will be screen recorded. After the task, the particioptions, ”Design Task Objects” (as our design task object pants will fill out Creativity Support Index (CSI), which is shopping cart, the button says ”Shopping Carts”), ”vi- is a well known psycho metric survey, for measuring six sually similar objects”, or ”conceptually similar objects”. dimensions of creativity: Exploration, Expressiveness, Visually similar objects have visual structural similarity Immersion, Enjoyment, Results Worth Efort and Collabas the design task object and conceptually similar objects oration [ 17 ] to evaluate user engagement, collaboration have similar working mechanism or usage as the design and immersion. After that, a retrospective think-aloud task object. When the user clicks any of these three will be conducted as the participants watch the screenbuttons, the PenPal will generate a sketch accordingly. recording video of the task to understand the rationale behind the user interaction process and user experience. 4.5. User Finished Sketching The study will end with a follow-up semi-structured interview to determine in depth qualitative data about the user engagement and overall experience with the AI.

When the user finishes the design ideation sketching,

they let the virtual agent know by clicking the ”Finish ideation” button. The virtual agent arrives and greets the user for completing the design ideation task by saying, ”Well done! You did a great job! ”.

5. Study Protocol

The user experiment will take place virtually. We will use Google Meet to connect with the study participants. The target sample size for the study is 50 participants

6. Discussion

In the young yet fast-growing field of human-AI cocreativity, attention is needed to design human-centered co-creative systems where users are engaged in a successful collaborative experience. Interaction design where users can communicate with the AI for providing user preference improves the collaborative experience and user attitude towards the AI [ 2 ]. Conversing virtual agents have transitioned into services such as ecommerce, leading to an increased ability for user engagement. In a conversing interaction, users can provide their feedback on the AI’s contribution, which provides more information to the AI about user preferences. Conversing interaction also helps users perceive the AI as a partner rather than a tool. A user’s confidence in an AI agent’s ability to perform tasks is improved when imbuing the agent with embodiment and social behaviors compared to the agent solely depending on conversation [ 14 ]. The embodiment improves the user perception of an AI agent in terms of a collaborative partner, an entity. Users also tend to trust an AI in terms of their ability when they can see their presence. The embodiment also helps design afective AI where an AI’s feelings are visible in its expression or gesture. As a young field and new research area, interaction design is rarely discussed in the existing literature despite being a fundamental property of an adequate co-creative system. An adequate interaction model dramatically improves the quality of the collaboration and engages users. Investigating the impact of conversing interaction and AI embodiment for designing efective co-creative systems that engage users is essential.

We develop the prototype of Creative PenPal as an efort to explore the impact of a conversing embodied co-creative AI agent on user engagement, user perception and overall collaborative experience. We describe the study design that will provide insights for designing efective co-creative systems that engage users and improve their collaborative experience with the AI agent. With the insights and results from the study, we will improve the interaction design of Creative PenPal and implement the AI model in the improved prototype. At the time of the workshop, we will have the data and insights in our hands from the study, revealing the impact of the interaction model we used. During the workshop, we will be able to demonstrate the results and insights.

References

[1] N. Davis , Human-computer co-creativity: Blending human and computational creativity , in: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment , volume 9 , 2013 .

[2] R. Louie , A. Coenen , C. Z. Huang , M. Terry , C. J. Cai , Novice-ai music co-creation via ai-steering tools for deep generative models , in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 2020 , pp. 1 - 13 .

[3] A. Sutclife , Designing for user engagement: Aesthetic and attractive user interfaces , Synthesis lectures on human-centered informatics 2 ( 2009 ) 1 - 55 .

[4] O. Bown , Player responses to a live algorithm: Conceptualising computational creativity without recourse to human comparisons? , in: ICCC, 2015 , pp. 126 - 133 .

[5] Library of mixed-initiative creative interfaces , http://mici.codingconduct.cc/, ???? (Accessed on 05/31/ 2020 ).

[6] S. Deterding , J. Hook , R. Fiebrink , M. Gillies , J. Gow , M. Akten , G. Smith , A. Liapis , K. Compton , Mixedinitiative creative interfaces , in: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems , 2017 , pp. 628 - 635 .

[7] Y. Rogers , H. Sharp , J. Preece , Interaction design: beyond human-computer interaction , John Wiley & Sons, 2011 .

[8] P. Isola , J.-Y. Zhu, T. Zhou , A. A. Efros , Imageto-image translation with conditional adversarial networks , in: Proceedings of the IEEE conference on computer vision and pattern recognition , 2017 , pp. 1125 - 1134 .

[9] C. Gutwin , S. Greenberg , M. Roseman , Workspace awareness in real-time distributed groupware: Framework, widgets, and evaluation , in: People and Computers XI , Springer, 1996 , pp. 281 - 298 .

[10] G. Hofman , G. Weinberg, Interactive improvisation with a robotic marimba player , Autonomous Robots 31 ( 2011 ) 133 - 153 .

[11] N. Bryan-Kinns , F. Hamilton , Identifying mutual engagement, Behaviour & Information Technology 31 ( 2012 ) 101 - 125 .

[12] M. Yee-King , M. d'Inverno, Experience driven design of creative systems ( 2016 ).

[13] J. Preece , H. Sharp , Y. Rogers , Interaction design: beyond human-computer interaction , John Wiley & Sons, 2015 .

[14] K. Kim , L. Boelling , S. Haesler , J. Bailenson , G. Bruder, G. F. Welch , Does a digital assistant need a body? the influence of visual embodiment and social behavior on the perception of intelligent virtual agents in ar , in: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) , IEEE, 2018 , pp. 105 - 114 .

[15] G. Bente , S. Rüggenberg , N. C. Krämer , Social presence and interpersonal trust in avatar-based, collaborative net-communications , in: Proceedings of the Seventh Annual International Workshop on Presence , 2004 , pp. 54 - 61 .

[16] C. Clavel , A. Cafaro , S. Campano , C. Pelachaud , Fostering user engagement in face-to-face humanagent interactions: a survey , in: Toward Robotic Socially Believable Behaving Systems-Volume II , Springer, 2016 , pp. 93 - 120 .

[17] E. Cherry , C. Latulipe, Quantifying the creativity support of digital tools through the creativity support index , ACM Transactions on ComputerHuman Interaction (TOCHI) 21 ( 2014 ) 1 - 25 .

[18] P. Karimi , J. Rezwana , S. Siddiqui , M. L. Maher , N. Dehbozorgi , Creative sketching partner: an analysis of human-ai co-creativity , in: Proceedings of the 25th International Conference on Intelligent User Interfaces , 2020 , pp. 221 - 230 .