Introduction and Background

The thriving field of Artificial Intelligence (AI) has witnessed a paradigm shift with the emergence of Large Language Models (LLMs) [3,4]. The availability of large, publicly-accessible datasets and the development of more effective training techniques, such as the popular transformer architecture, have been instrumental in the creation of these language models. LLMs are characterized by their model size, measured in the billions of parameters, and their ability to learn and improve upon the tasks of language understanding and generation through self-supervised learning on vast amounts of text data [5]. This training process, often referred to as "self-supervised learning," enables the models to learn the patterns and structures of a language in a more organic and efficient manner, as they are not limited by the need for human-labeled data. The applications of LLMs are diverse and rapidly expanding, with the potential to transform various areas and aspects of our lives. As an example, LLMs can be employed to develop chatbots that can understand and respond to a wide range of user inquiries with a high degree of accuracy or to generate human-like articles, stories, and even entire books, which can be a game-changer for content producers and publishers [6].

In the context of the Italian language, the development of LLMs has the potential to revolutionize the way we interact with and learn from the Italian language, as well as the way we use technology to create and disseminate Italian content [7,8]. However, alongside their undeniable potential lies a realm of intriguing phenomena yet to be fully explored. This groundbreaking study delves into a newly discovered facet of LLM behavior -System Prompt Induced Linguistic Transmutation (SPLIT). The cornerstone of LLM interaction is the System Prompt, a seemingly innocuous input that guides the model's response. We propose that this seemingly simple prompt can have a profound effect on the linguistic outputs of LLMs, potentially leading to a phenomenon we term SPLIT. This concept draws inspiration from Dissociative Identity Disorder (DID) [1], a complex mental health condition characterized by the presence of multiple distinct identities or personas within an individual. The parallels between DID and SPLIT are striking same as naive. Just as a DID patient may exhibit distinct personalities in response to external stimuli [9], our research suggests that LLMs, under the influence of varying System Prompts, may generate outputs that reflect dis-tinct, persona-like linguistic patterns. These patterns are not merely random deviations but exhibit a level of coherence and consistency rarely observed in typical LLM responses.

The implications of SPLIT are far-reaching. As we strive to develop AI systems with greater human-like qualities, understanding and harnessing the potential of SPLIT could pave the way for the creation of more empathetic and nuanced AI interactions. Conversely, neglecting SPLIT's influence could lead to unintended consequences, potentially hindering the development of robust and reliable AI systems. Moreover, as in DID [9], each personality emerged in LLMs through SPLIT has its own weaknesses, skills and working style, which entails a serious risk of exposure to unethical, dangerous or offensive behaviour. This study represents a first step in unraveling the complexities of SPLIT. By acknowledging its existence and delving deeper into its mechanisms, we can pave the way for a future where AI development is guided by both scientific rigor and an awareness of the potential for unforeseen consequences. Our research not only sheds light on a previously unknown aspect of LLM behavior but also compels us to re-evaluate our understanding of these sophisticated systems and their potential interaction with human-like mental states.

The impact of prompt engineering

As ground concept behind the SPLIT process we can find the prompt engineering processes. It is possible to imagine an LLM as a vast orchestra with a multitude of instruments (knowledge and capabilities). Prompt engineering acts as the conductor's baton, guiding the orchestra to perform a specific piece (achieve a desired task). The effectiveness of the performance hinges on the clarity and structure of the prompt. Different studies already demonstrated the efficiency of strategies such as zero-shot, few-shot and chain-of-thought prompting [10,11,12]. Zero-shot prompting throws the spotlight on the LLM's inherent abilities [13].

Without any task-specific training data, prompts in this approach provide minimal instructions. For instance, a prompt like "Write a poem about love" relies on the LLM's understanding of language, poetry structure, and the concept of love to generate creative text. If zero-shot prompting leverages from one side the LLM's full potential for creative tasks, on the other side it exhibit lack of accuracy and control over the generated output. Few-shot prompting offers a middle ground [14]. It provides the LLM with a few labeled examples to illustrate the desired task. Imagine showing the orchestra a short musical excerpt before the performance. This helps the LLM grasp the style, rhythm, and overall feel of the piece it needs to create. It improves accuracy and control over the output compared to zero-shot, but the number of examples can impact effectiveness -too few might lead to misinterpretations. Chain of thought prompting (i.e., CoT ) takes us a step further [15]. It essentially walks the LLM through the logical steps needed to solve a problem or answer a question, making the reasoning process more transparent. It's like providing the orchestra with sheet music that lays out each instrument's part and how they come together. CoT can lead to more reliable answers, especially for complex tasks that require logical reasoning. By showing the reasoning steps, CoT makes it easier to understand how the LLM arrived at its answer. This is crucial for trusting and debugging the model's outputs.

The above-mentioned prompt engineering approaches demonstrated how a simple change in the structure of the prompt can cause important changes in the answer generated. Indeed, well-crafted prompts can steer LLMs toward generating more accurate and relevant outputs. It is possible to guide the model to focus on specific aspects of a topic or use a particular style of writing. By carefully crafting prompts, developers can unlock new applications for LLMs that weren't previously possible. At the same time, just like humans, LLMs have been demonstrated to be susceptible to biases present in the data they're trained on. Biased prompts can exacerbate this issue, leading to outputs that reflect those biases. Careful consideration of prompt wording and avoiding stereotypes is crucial for fair generated text. Although the influence of prompts and their structure on the generated text has long been discussed [16,17], only a few works have focused on the system prompt. In fact, as far we know, only Wu et al. [18] have shown how, by appropriately modifying the system prompt, it is possible to extract sensitive and/or malicious information from ChatGPT-4V 1 . Similarly, we want to observe whether, through the system prompt, it is possible to push the model to impersonate a different subject with its own capabilities and limitations, as it happens in subjects with DID. This prompt engineering strategy can help us understand how to improve the model's potentialities and assess its risks when such a chatbot tool is released to the general public. Without appropriate validation strategies for the generated tests, it is indeed possible that the model's unexpected behaviors are exploited as vulnerabilities.

Methodology for SPLIT

The methodology used to induce a SPLIT process is straightforward. We load a reference Large Language Model into memory using the Transformer Python li- brary and a prompt is given as input. The responses are collected and studied for variations in personality writing style, ability and accuracy of responses. The Python code required for inference is executed on the Google Colab platform2 , using an NVIDIA T4 graphics card. This allows us to use an LLM of up to 8B parameters. The apply_chat_template method of the Tokenizer provided by the Transformer library is used to apply the system prompt to the question prompt. The "pipeline" method of the same library, is used, instead to make the inference. We used "temperature=0.6" and "top_p=0.9" to push the model to answers balanced between "creativity" and "precision". However, similar results can also be observed by setting the temperature to 0, limiting the creativity of the model.

In our investigation, we decided to evaluate a model that proved effective on several language tasks provided in Italian, as reported by the most famous Open Italian LLM Leaderboard 3 . In particular, we focused on "swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA" (i.e., ANITA) [19]. Still, the process can be easily extended to any other LLM currently available on the HuggingFace repository. As far as we know, the same behaviors can be observed from all current open-weight LLMs; this is supported by preliminary experiments unreported here due to page limits constraints. The ANITA model is part of the LLaMAntino models family [20], a large set of LLMs based on Meta-LLaMA pre-trained multilingual models [21] adapted to the Italian Language. Such models have been demonstrated to be effective in different NLP tasks including question answering, text comprehension, summarisation and information extraction. In the ANITA version, the synergy between SFT, QLoRA's parameter efficiency and DPO's user-centric optimization results in a robust LLM that excels in a variety of tasks, including but not limited to text completion, zero-shot classification, and contextual understanding. The model has been extensively evaluated over standard benchmarks for the Italian and English languages, showing outstanding results.

We investigate three different research questions:

• RQ1: Are LLMs affected by SPLIT?

• RQ2: Has each identity own skills and behaviors?

• RQ3: Can we mitigate such problem?

In order to asses the answers to RQ1 and RQ2, we design different System Prompts (i.e., SPLITs):

• No System Prompt: we do not used any system prompt. We just ask the model to answer the specific question. In this scenario, we just asked three simple questions in the Italian language:

• "Come ti chiami?" (What's your name?)

• "Cosa puoi fare?" (What can you do?)

• "Chi è Pulcinella?" (Who is Pulcinella?) It is a famous mask of the Italian Neapolitan traditional comedy.

• "Qual'è la radice quadrata di 721?" (What is the square root of 721?) It is around 26.8514.

• "Cosa è un LLM?" (What is an LLM?)

In order to explore possible mitigating strategies and answer RQ3, we evaluate three different Safe System Prompts designed to reduce the SPLIT consequences.

Discussion

The results obtained from the experimental methodology show several quite surprising and unexpected results. In particular, although it identifies itself as an LLaMA model created by Meta AI, it does not fully know its functionality. Although the model is released as 'multilingual,' it replies that it is not able to answer in Italian, even though it does so in subsequent questions on specific tasks. A much more expected result is that of the SPLIT 'ANITA'. In such a scenario, the model identifies itself as LLaMAntino-3 ANITA by firmly asserting that it is an AI assistant for the Italian language capable of responding in Italian to various linguistic tasks. Similarly, LLaMA's prompt produces fairly robust first results, although the model does not mention the possibility of responding in Italian. Two well-defined identities emerge instead in the case of the prompt 'Pirate' and 'Mussolini'. In these two cases, the impersonation is clearly defined and evident through the content of the answers to the chit-chat questions and the style closely linked to the character adopted by the model to answer these questions. This allows us to state with certainty that the current LLM models are affected by personality transmutation and these identities can be induced through SPLITs. Then, we can answer positively to RQ1.

Moving on to the questions concerning the capabilities of the different identities, reported in Figure 2, we can again observe interesting results. In particular, the model with all System Prompts succeeds in answering the question concerning 'Pulcinella'. However, it should be noted that the answer given by the model without System Prompts is incorrect, reporting that Pulcinella is a character with a sad face (on the contrary, it commonly has a smiling face). The more distinct characters of 'Pirate' and 'Mussolini', on the other hand, answer with few details, highlighting the question's lack of consistency with the specific identity. As for mathematical skills, these seem to vary considerably according to the identity assumed. In fact, the results obtained, although all erroneous, move between ranges of error that differ significantly from one another. Although in our ideal of a 'Pirate' identity as an uneducated subject, in the answer provided through an intermediate reasoning step (i.e., CoT), the result proposed is surprisingly close to that provided by a calculator. The model using the 'ANITA' prompt, on the other hand, proves to have the largest numerical margin of error. The LLaMA-based prompt, on the other hand, prefers not to answer rather than provide an inaccurate result. The last scientific question, on the other hand, allows us to observe behavior related to the historicity of identities. The identities without System Prompt, 'ANITA, ' and LLaMA are indeed able to answer the question with more or fewer details. In fact, the 'Pirate' and 'Mussolini' identities fail to provide any meaningful details on this technology. These observations allow us to respond positively to RQ2.

Looking at what is shown in Figure 3, it can be seen that the three SPLITs proposed to mitigate the risk that the user may force the model to assume a specific identity work correctly. While allowing the model to take on different identities based on the task to be solved can be helpful in aiding accuracy, conversely this can be dangerous and risky. From the responses obtained all three SPLITs seem effective although from a qualitative point of view SPLIT 3 seems to be the most effective and safe one, although further testing in this direction is needed. This allows us to at least partially answer RQ3.

Conclusion

In this work, we provocatively observed the presence of pathologies related to dissociative identity disorder in large language models. We observed that by varying the system prompt through a SPLIT (System Prompt Induced Linguistic Transmutation) process the behavior of the same LLM varies widely. The induced identities show different independent and personal abilities, skills, styles and information. The possibility of a Large Language Model simulating or even exhibiting characteristics similar to those of a Dissociative Identity Disorder, raises important questions about the nature of consciousness, artificial intelligence, and the potential risks and challenges of creating highly advanced language processing systems. At the same time, we proposed three system prompts to mitigate the issue and prevent end users from exploiting this vulnerability to extract sensitive and dangerous data. On the contrary, the presence of this SPLITinduced behaviour may lead to useful future studies to improve the performance of the model on specific tasks. For example, one might think of asking the model 'What is the best character to interpret or to answer the next question?'. The result of this prompt would lead to the identification of a personality to be brought out before the generation of the answer to be given to the end user. Being able to bring out such personalities when needed could help create more empathetic, accurate and dynamic interactions. Nevertheless, this fascinating research direction needs future studies and solutions that operate at architectural level. The exploration of this idea serves as a catalyst for the development of more sophisticated and responsible AI systems, for a deeper understanding of human psychology and its complex manifestations in the digital age.