We present the Collaborative Canvas, a prototype tool for exploring ways for groups to interact with large language models (LLMs) in ideation tasks. Collaborative Canvas provides a shared, graphical canvas in which multiple parties - human and LLM - can share ideas in the form of virtual “sticky notes” that can be moved around the canvas. The development of Collaborative Canvas raised numerous issues about the role of an LLM in group interactions: is it useful, what role does it play within the group's workflow, and how do people interact with generated content? A preliminary examination of the Collaborative Canvas shows that users found the generative capabilities to be useful, although they preferred to review and filter generated content before sharing it with the group. Users also speculated that the role of the AI could extend into facilitating group brainstorming rather than being confined to idea generation. Our work motivates the study of human-AI co-creation in group settings beyond dyadic interactions.
Researchers have recently begun exploring the role of generative AI for various tasks within
design, including persona generation [
Many studies of human-AI co-creativity and human use of generative AI focuses on the interactions between a single human user with a generative AI application. However, given the collaborative and real-time nature of shared canvas tools, new research questions arise around the role of generative AI in group work: do people in a group find it useful to have an integrated generative “assistant,” what role does it play within the group’s workflow, and how do people interact with generated content?
We developed the Collaborative Canvas as a way to prototype diferent forms of interaction with generative AI, and more specifically, with large language models. In developing our prototype, we uncovered important design considerations regarding the ownership and visibility of generated content and the UX mechanisms used to invoke the generative functionality. Our work seeks to identify new research avenues for studying the impact that generative AI has in group settings, pushing research in human-AI co-creativity beyond the examination of dyadic interactions.
Wang et al. [5] recently described the challenges of human-AI collaboration by reminding the HCI community that, “Interaction is not the same as Collaboration” ([5, p.1]. Other researchers, including Aragon et al. [6], Inkpen et al. [7], and Shneiderman [8] have made similar arguments. The concept of AI agents acting as “teammates” has recently gained interest (e.g., [9]), but research in human-AI co-creation often focuses on the “team” as being composed of one human and one AI agent (e.g. [10, 11, 12, 13, 14, 15, 16]).
Despite the prevailing focus on human-AI dyads, some researchers have begun exploring the use of generative AI in larger team settings. Gan et al. [17] introduced an AI-Mediated Group Ideation tool that acts as a creative mediator in an architectural ideation process. In their study, they found that the AI mediator did have utility, but they also identified challenges in integrating it within the workflow in ways that were not distracting. Cvetkovic et al. [18] derived a number of design principles for a conversational assistant that collects and summarizes ideas from a group ideation session. Their study also found utility in having the assistant participate in group ideation, but identified challenges with knowing when it should make a contribution and how much to contribute. Lavrič and Škraba [19] examined the brainstorming capabilities of OpenAI’s GPT-3.5-turbo model and found that the ideas it produced were useful. Although their examination was of ideas that were solely generated by the LLM, the authors concluded, “the hybrid process of generating ideas should be tested by real committees, where the OpenAI GPT API would be applied in combination with human agents. This might contribute to a better set of innovative ideas...” [19, p.1297].
We designed and implemented Collaborative Canvas to explore ideas around group collaboration with generative AI. Motivated by the emerging use of generative AI within the design profession, we desired an environment that would be useful to designers for conducting various activities within the practice of design thinking. As the designers within our own organization make heavy use of Mural, we initially explored whether we could incorporate generative capabilities using its API3. However, we found that we were unable to prototype many types of interactions 3Our explorations began before the introduction of Mural AI: https://www.mural.co/blog/announcing-mural-ai with Mural’s API4 and pivoted to building our own tool.
We show a screenshot of the Collaborative Canvas in Figure 1. As we aimed not to replicate a commercial product, it possesses a minimally-useful set of functionality commonly found in shared canvas tools: • Multiple users are able to join the shared canvas environment and interact with it in real time. A list of connected users shows who is currently in the environment. • Users are able to add, select, and remove sticky notes. • Users are able to change the size, color, and position of sticky notes. • Users are able to zoom in and out within the canvas.
In addition to providing core shared canvas functionality, Collaborative Canvas also ofers several diferent ways to invoke a large language model to generate new content: 4At the time, Mural’s API did not support their embedded chat feature, eliminating the possibility of examining conversational interactions. In addition, the API did not provide support for reading the selection state of a user, eliminating the possibility of selecting a group of sticky notes and invoking a generative model to, for example, summarize their contents.
• New sticky notes can be generated and added to the main canvas from a textual prompt in a modal (Figure 2). These notes are visible to all users when they are generated. • New sticky notes can be generated within a private scratchpad (Figure 3A & B). Users are then able to drag sticky notes from the scratchpad into the shared canvas space (Figure 3C). • Users can progressively refine the content of generated sticky notes through follow-up requests. The sequence of requests are preserved so that notes can be regenerated in the context of previous requests (Figure 3D).
A
B
C C
We implemented the Collaborative Canvas using the Svelte5 framework for the front-end UI and Python with FastAPI6 for the back end. The back end was responsible for communicating with an internal version of the IBM watsonx.ai platform where the LLM was hosted. While this platform provides API access to a number of state-of-the-art LLMs, we chose the Llama 2 [20] model for our prototyping and experiments. We developed a prompt for this model to set the context for users’ actions, including generating sticky notes and progressively refining generated responses. This prompt also instructed the model to format its responses in a way that was easy to parse into separable chucks by the front end (e.g. to create multiple sticky notes from a single LLM response). We list the LLM prompt used by Collaborative Canvas in Appendix A. In Figure 4, we illustrate the flow that occurs from the moment a user requests the LLM to generate a group of sticky notes until they are generated. 5Svelte. https://svelte.dev 6FastAPI. https://fastapi.tiangolo.com
We conducted a preliminary evaluation of the Collaborative Canvas to understand the extent to which its diferent generative capabilities could be used within a group brainstorming session. We recruited 8 design professionals within our organization who facilitate design thinking workshops as part of their work. Our goal was to learn how these domain experts, who currently make extensive use of shared canvas tools for collaborative work, might use the integrated generative AI capabilities of the Collaborative Canvas. We hosted hour-long sessions with groups of 2-3 participants in which we demonstrated how to interact with the canvas and invoke its generative AI capabilities. We then asked participants to use the tools to complete a short ideation task in which they each simultaneously wrote sticky notes manually on the canvas or used the AI tools to generate notes with as many ideas as possible to identify possible benefits of using LLM’s in design thinking sessions. Afterwards they worked together to cluster ideas by theme.
Several important considerations arose from the feedback from our participants: • Participants engaged with AI in varied ways. Some participants explored their own ideas ifrst before incorporating AI-generated content; others leveraged AI support from the outset to either augment their understanding of the task domain or generate ideas. • Most participants preferred to use the personal scratchpad to generate sticky notes rather d a p h c tr a c s l a n o s r e P s a v n a C d n e k c a B
User enters query and desired sticky note count Original request
Prompt history Backend document
store Session config
Prompt template
Generate individual sticky note widgets with result contents
User iterates over the result Refinement request
User selects one or more sticky notes and drags them into
the Canvas Place selected sticky notes from personal scratchpad in the position the user dropped them
The development of the Collaborative Canvas, as well as our preliminary study, revealed several interesting new directions. We group these directions into specific improvements for the Collaborative Canvas tool and new research directions for human-AI co-creation.
Diferentiate AI-generated vs. human-generated outputs . Participants found it extremely
important to distinguish between AI-generated and human-generated content in a group setting.
Visual cues, such as icons and outlines, are one way to clearly represent which sticky notes were
generated by the AI. Assigning credit for AI outputs may also involve capturing the prompt
that resulted in the output, acknowledging the user who wrote the prompt and anyone who
revised a sticky note, and displaying a record of both user and AI contributions for each sticky
note. Rezwana and Maher [
Provide control for the novelty and diversity of generated ideas. Participants recognized
the need to sometimes produce unexpected (or provocative) ideas to push groups along their
ideation process [
Find better ways of hiding inference latency. In our preliminary study, we sometimes
found that inference latencies were high, leading to situations in which a group of sticky notes
were “pending” for long periods of time. These periods frustrated our users, especially when
these sticky notes occluded other content. Future work is needed to design new mechanisms
for hiding inference latency in a shared canvas environment.
5.2. New research directions for human-AI co-creation
Extend beyond content generation. Participants saw significant value in using AI to support
their unique roles as facilitators of design thinking workshops. They envisioned helping groups
get “unstuck” by generating new ideas. Facilitators also manage the brainstorming process
by denoting diferent phases (e.g. idea generation vs. clustering vs. filtering & selection), and
they envisioned that the AI could provide them with individualized support for organizing and
executing these phases. Work by McComb et al. [
Explore new interactive techniques. We explored invoking the generative capabilities of an LLM through direct manipulation interactions by generating sticky notes directly on the canvas or in the scratchpad. It is also possible to incorporate conversational interactions in the interface, such as by using a dedicated pane for Q&A (similar to the technique used by Ross et al. [10] in the Programmer’s Assistant). Additional forms of interaction may also be possible. For example, could we generate a blending of two ideas by dropping one sticky on top of another? Is there a way to “embed” conversation within the canvas? Given the rise of multi-modal models and the visual nature of the canvas, how can we incorporate images (or audio, or video) as outputs from (or inputs to) the generative model?
Evaluate human-AI team efectiveness . How does the use of generative AI in a group brainstorming activity influence the quality of the team’s output? Does it make the process more eficient or require less mental efort? Further research is required to determine the precise quantitative impact of generative AI on group productivity.
AI proactivity. When should an AI assistant make a contribution to a group? Our tool
must be explicitly invoked by a user in the group, suggesting a completely reactive mode of
operation. However, mixed initiative interactions [
We developed the Collaborative Canvas tool to explore group interactions with an LLM in the context of ideation tasks. The tool allows users to generate content by adding it to the canvas itself or by placing it in a private scratchpad that can be reviewed before it is added to the canvas. In addition, users can progressively refine content through context-sensitive re-generation requests. In a preliminary evaluation of the Collaborative Canvas, participants found utility in generated content by using it outright, modifying it, or using it to stimulate their own thinking. They also identified important considerations regarding the role of AI in an ideation session and made various recommendations for improving the tool. Our work motivates further study of AI assistance within group settings. [5] D. Wang, E. Churchill, P. Maes, X. Fan, B. Shneiderman, Y. Shi, Q. Wang, From humanhuman collaboration to human-ai collaboration: Designing ai systems that can work together with people, in: Extended abstracts of the 2020 CHI conference on human factors in computing systems, 2020, pp. 1–6. [6] C. Aragon, S. Guha, M. Kogan, M. Muller, G. Nef, Human-centered data science: an introduction, MIT Press, 2022. [7] K. Inkpen, S. Chancellor, M. De Choudhury, M. Veale, E. P. Baumer, Where is the human? bridging the gap between ai and hci, in: Extended abstracts of the 2019 chi conference on human factors in computing systems, 2019, pp. 1–9. [8] B. Shneiderman, Human-centered AI, Oxford University Press, 2022. [9] I. Seeber, E. Bittner, R. O. Briggs, T. De Vreede, G.-J. De Vreede, A. Elkins, R. Maier, A. B.
Merz, S. Oeste-Reiß, N. Randrup, et al., Machines as teammates: A research agenda on ai in team collaboration, Information & management 57 (2020) 103174. [10] S. I. Ross, F. Martinez, S. Houde, M. Muller, J. D. Weisz, The programmer’s assistant: Conversational interaction with a large language model for software development, in: Proceedings of the 28th International Conference on Intelligent User Interfaces, 2023, pp. 491–514. [11] L.-Y. Chiou, P.-K. Hung, R.-H. Liang, C.-T. Wang, Designing with ai: An exploration of co-ideation with image generators, in: Proceedings of the 2023 ACM Designing Interactive Systems Conference, 2023, pp. 1941–1954. [12] J. Geerts, J. de Wit, A. de Rooij, Brainstorming with a social robot facilitator: Better than human facilitation due to reduced evaluation apprehension?, Frontiers in Robotics and AI 8 (2021) 657291. [13] J. Koch, A. Lucero, L. Hegemann, A. Oulasvirta, May ai? design ideation with cooperative contextual bandits, in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12. [14] A. Millwood, C.-C. Dias-Taguatinga, Ai image generation tools as an aid in brainstorming architectural visual designs, thesis, stockholm university, 2023. [15] C. Oh, J. Song, J. Choi, S. Kim, S. Lee, B. Suh, I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–13. [16] B. Wieland, J. de Wit, A. de Rooij, Electronic brainstorming with a chatbot partner: A good idea due to increased productivity and idea diversity, Frontiers in Artificial Intelligence 5 (2022) 880673. [17] A. W. J. Gan, Q. Dang, B. Western, J. L. G. del Castillo, Ai-mediated group ideation, eCAADe proceedings (2023). URL: https://api.semanticscholar.org/CorpusID:262061776. [18] I. Cvetkovic, M. Gierlich-Joas, N. Tavanapour, N. Debowski-Weimann, E. A. Bittner, Augmented facilitation: Designing a multi-modal conversational agent for group ideation (2023). [19] F. Lavrič, A. Škraba, Brainstorming will never be the same again—a human group supported by artificial intelligence, Machine Learning and Knowledge Extraction 5 (2023) 1282–1301. [20] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini,
A. Collaborative Canvas Prompt Template 1 This is a conversation with an automatic collaborative assistant that is expert, 2 eager, helpful, and humble. Here are some considerations the assistant must have 3 when generating the response to the user’s query, which is located within the 4 <USERS_QUERY> tags: 5 - The assistant will respond in a way that the user can generate actions based on 6 assistant’s response. 7 - The actions presented by the assistant should be in a numbered list format. 8 - The assistant will always indicate the end of the generated actions with the 9 string <END> at the conclusion of the list. 10 - The assistant should not use line breaks between each item in the numbered list. 11 - If the assistant receives context actions within the <CONTEXT_ACTIONS> tags, then 12 it must utilize those actions as context to generate the response actions. 13 - The assistant will respond with a maximum of 10 actions, but it can also respond 14 with fewer if the assistant deems it necessary or if the user asks it to generate a 15 few actions or a specific amount, such as a couple. 16 - The assistant must not generate actions with identical content. 17 - An example of the format that the assistant should use when responding is: 18 1. Response generated by the assistant for the first action. 19 ... 20 n. Response generated by the assistant for action number n. 21 <END> 22 <USERS_QUERY></USERS_QUERY> 23 <CONTEXT_ACTIONS></CONTEXT_ACTIONS> Listing 1: Collaborative Canvas prompt template - The content within the <USERS_QUERY> and <CONTEXT_ACTIONS> tags is populated at runtime in the backend of the prototype once the user enters the query and the desired sticky note count, as shown in Figure 4.