AI supported Creativity is a hot debated topic, due to the rise of LLMs. This paper explores a prototypical implementation of the classic Six Thinking Hats method via a LLM, using the OpenAI API. The results suggest that such AI support is useful in a creative process; but will not replace the human creativity.
Large Language Models (LLMs, singular: LLM), generative mathematical models of the statistical distribution of tokens in vast public corpus of human generated texts [1], are very prominent due to the success of ChatGPT 1 .
The current Gartner Hype Cycle sees LLMs at their zenith [2]. Some practical disadvantages and weaknesses of the technologies are already known, such as hallucinations [3] or copyright violations [4]. It is currently unclear when and to what extent the disillusionment will occur and at what level productivity will level out. What is clear, however, is that this technology is predicted to have a major impact on creative work and knowledge work [5], [6], [7]. One research line towards this direction is ai driven creativity / ai driven inspiration / ai powered creativity [5], [8], [9], [10].
This paper describes a prototypical implementation of the Six Thinking Hats method via a LLM (Six Thinking Chatbots). It shows what a combination of creativity technique and a large language model might look like. Section 2 takes on related works. Section 3 describes the design assumptions, the program flow and according prompts. In section 4 the results of the prototype are evaluated. Section 5 closes this paper.
Related work from the view of requirements engineering such as [11] describe the usage of LLM in each major stage of requirements engineering according to [12]. For the elicitation phase, a single-shot prompt is used. This is comparable to the webbrowser prompt in section 4.2 of this paper. This point of view to this topic is not well explored; first experiments such as [13] are being carried out.
Related work from the view of LLM usage is in full bloom. In the ChatGPT hype, the development of prompting strategies such as [14], [15], [16], [17] provide standardized approaches to prompting. These approaches take on the technical usage of prompting, such as the "write clear instructions", "provide reference texts" [17]. Though these are very useful, these approaches do often not tackle complex situations of human conversations.
The main new idea of this paper is the simulation of a human workshop, using the LLM in multiple, programmed steps.
The Six Thinking Hats methodology is well known in Requirements Engineering. Although it entails several ways to use the different hats2 , this paper only explores one application i.e. the identification of one alternative from a group of three.
A practical workshop needs an introduction by the moderator (blue hat). Due to the OpenAI API, this type of introduction is not necessary or is covered by providing appropriate information before the prompt. Before the actual prompt, a system content is given that contains the content of an introduction to a classic workshop. The order of colors is white, green, yellow, black, red and blue.
To limit the necessary computing power, the contribution from each hat is limited to three bullet points, without explicit limit of words or characters.
Python3 was used as the implementation environment. A simple object-oriented design of the prototype was chosen. The source code is available at https://GitHub.com/rachmann-alexander/ sixthinkingchatbots.
ChatGPT-4 was used as the LLM. In general, ChatGPT-4 is considered the market leader and is therefore suitable as an implementation tool for the prototype. It is unclear whether other LLMs would show a substantially different result. Choosing ChatGPT entails the use of the corresponding API, which in turn entails certain prompt forms.
Six Thinking Chatbots follows this flow:
1. The user delivers a problem statement.
2. The messages for white, green, yellow, black, and red are computed. The messages to each hat follow the structure:
• System Content gives the LLM the context to interpret the prompt. System Content is a necessary parameter to the ChatGPT API. • User content, also a necessary parameter to the ChatGPT API, is subdivided in two entities: Problem Statement, a description of the topic to-be-discussed by the hats. Task, a description of what the hat should do.
The messages are submitted to ChatGPT API and responses are received. 3. Compute the prompt for blue hat; submit the prompt to ChatGPT API. The prompt to blue follows the same structure as above, but incorporate the responses to the other hats.
The response for blue is received. 4. All prompts and responses are exported to a Markdown-formatted document.
OpenAI expects the user to give context to a certain prompt, which is given in the system content. This context gives the LLM a better understanding of what the task should be. Each hat receives a different system content, depending on its nature as defined in Six Thinking Hats Methodology. Table 1 lists the system content of each hat. Each prompts of all hats, except for blue Hat, contain a problem statement and a generic task. Table 2 gives the content for the white hat, table 3 for the blue hat.
The blue hat owns the moderator role and has therefore the task to summarize the outputs of the other hats. The prompt to the blue hat is therefore more complex. At first, the content is given, then the task itself. After that all other contributions are given. Table 4 gives an example for a response to the blue hat.
System Contents of the hats. All system contents start with "You are a participant of a workshop". The content in the table complete the prefix.
Your decision making is fact based and your argumentation is very structured.
Your decision making is based on the possibilities of innovations.
You are an optimist.
You are a pessimist.
Your decision making is emotion based.
You are the moderator of a workshop, using the Six Thinking Hats methodology. You are neutral to the contributions from the participants and value each contribution.
Example of a prompt, structured by system content (example of the White Hat), problem statement (example according to "Sales Channel" as used in the prototype) and task (generic for all Hats except Blue Hat).
System content: You are a participant of a workshop. Your decision making is fact based and your argumentation is very structured.
The goal of the workshop is to evaluate three alternatives to a problem. The problem is: The sales process on our companies website does not work very well. Management wants us to change the sales process to one of the three alternatives: First, an old-school chatbot with a beforehand written structure. Second, a chatbot using a large language model, without a pre-defined structure.
Third, an old-school contact form. Prompt (Task): Describe your point of view to this problem, using only three bullet points.
Example of the prompt to blue hat. The problem statement is the same as in the prompts to the other hats and therefore emitted ("…") in the table. The prompt contain the contribution from all other hats; this is quite long and therefore also emitted in the table. All contents are available at the GitHub Repository.
System content: You are the moderator of a workshop, using the Six Thinking Hats methodology. You are neutral to the contributions from the participants and value each contribution. Prompt (Task): Summarize the contributions of the participants, as listed beneath. Think about the problem statement and decide, based on the contributions of the participants, which alternative should be chosen.
The problem statement of the workshop is as follows: … Prompt (Contribution White): Contribution from a fact-oriented participant: … Prompt (Other Contributions): Contribution from …hat: …
Example of the response from blue hat.
Response from the LLM to the Blue Hat Prompt After analyzing all of these contributions, it appears that the majority lean towards investing in a large language model chatbot for the company. Those in favor of this option highlighted several benefits including its adaptability to changing customer needs, the potential for learning and improvement over time, and the ability to provide tailored, efficient service on the fly. They also lauded its innovative approach and the potential for the drastically improved client interaction it could offer. It's recommended to consider the concerns raised by the pessimist too, ensuring to mitigate potential issues like consistency, the maintenance of brand tone, and avoiding technical glitches. Availability of resources, cost implications and consideration for the long-term impact are also factors that should be taken into account when implementing a large language model chatbot solution.
The participants showed less enthusiasm for the old-school chatbot and the contact form, identifying limitations like lack of flexibility and evolution, impersonal service, slow turnaround, and inability to handle complex inquiries. These options could result in a process that may feel outdated and inefficient, potentially deterring customers rather than facilitating sales and improving the customer experience. Therefore, based on these contributions, the large language model chatbot stands out as the most promising alternative.
To answer Q4, five texts were generated. To compute how similar these texts are, one can use the shingle algorithm to compute the jaccard similarity. The jaccard similarity describes how similar two documents are to each other. A similarity of 1 means the two documents are identical, a 0 means the documents do not have an intersection. There is no definition on how the jaccard similarity must be in order to identify two documents as the same or nearly same. [20] The average similarity of different texts is 0,608. Considering, that there is no common understanding of how similar this is, one may conclude, that the prompts are always distinctively different, but always alike.
The answer to question 4 is not as to compute as the other answers. First, the quality of the outcomes of a workshop depend highly on the participants, their motivation, the moderator, the conduction of the workshop, etc. If a person workshop would provide different or better outcomes, is not easy to determine. However, the arguments made by the chatbots are mostly common knowledge and practice. It would be no surprise, if these outcomes were elaborated by non-experts. The chosen application scenario was generic. It is also unclear whether the same results would arise with very specific use cases or domain experts.
This paper introduces a prototypical implementation of the Six Thinking Hats method, called Six Thinking Chatbots. The source code is available at GitHub, as are all evaluation cases. The experiment showed that the Six Thinking Chatbots deliver distinctively different results than regular prompting (Q1). The different hats play out their specific role (Q2 and Q3). The script delivers always distinctively different outputs (Q4). It is unclear, if the responses of the chatbots are different / in what way they are different to responses of humans (Q5). However, the author would use the script only to support human creativity (such as a preparation to a Six Thinking Hats workshop), not as a substitution for human creativity.
One can evaluate the Six Thinking Chatbots in terms of:
• Q1: Would a regular prompt deliver the same results with less effort? • Q2: Are the arguments by the colored hats disjoint? Did the hats take different position? • Q3: Did the blue hat summarize the contributions from the other chatbots correctly? • Q4: How "creative" is the script? How much do the responses differ from each other, if the script runs several times? • Q5: Are the contributions valid, do they represent the state of the art or are they comparable to a workshop with people?
To answer Q1 and Q4, one needs to generate material for comparison. As comparison for Q1, the StrategyQA-Prompting style was chosen [19]. The prompt is structured in a context-part (see table 5), which informs the LLM about the workshop. The second part is the task itself. The third part defines the output format 4 . This last part was not necessary in Six Thinking Chatbots, because the export is handled by the script.
Prompt to generate comparison material. This prompt is not deployed via software, so that the code repository is not equipped with code that will only be executed once. All contents are available at the GitHub Repository. The Web browser interface of ChatGPT was used execute this prompt.
The goal of the workshop is to evaluate three alternatives to a problem. The problem is: The sales process on our companies website does not work very well. Management wants us to change the sales process to one of the three alternatives: First, an old-school chatbot with a beforehand written structure. Second, a chatbot using a large language model, without a pre-defined structure. Third, an old-school contact form. Evaluate which solution one would prefer. Give at most 15 arguments. After listing the arguments, write a conclusion including a selection on a solution. Use markdown syntax.
The response to this prompt is structured along the three alternatives (table 6); each alternative is structured in a pro-and contra-section with each three to five arguments.
To answer Q4, one needs to compute several results. After developing the Six Thinking Chatbots script (and thereby generating several preliminary results), five results were computed. All five results are available at GitHub. Only one Text (29.12.2023, 13.06.md in the GitHub Repo) was used a standard analysis material in tables 7 and 8.
To answer Q1 and Q2, table 7, page 8, shows the arguments made by the chatbots. One sees, that the arguments are mostly disjoint; this holds within the results of the colored hats as well as between the Six Thinking Chatbots result and the Web browser result. The chatbots did contribute different aspects and therefore fulfilled the expected outcome. To answer Q3, see table 8. Blue hat voted for the AI chatbot, which represents the majority of the votes of the other chatbots.