In recent decades, the rapid advancement of Artificial Intelligence (AI) has profoundly transformed various aspects of society. Specifically, within the workplace, AI has proven to excel in tasks involving extensive data analysis, high precision, and sustained cognitive effort. Nevertheless, research consistently emphasizes the effectiveness of human-AI collaboration, often referred to as hybrid intelligence, in achieving optimal results [5,11]. This has sparked a growing interest in comprehending the dynamics of Human-AI teams (HATs) to implement AI effectively within the workforce.

Trust emerges as a pivotal factor in shaping the dynamics of HATs, as it underlies critical team interactions. Striking the right balance of trust is essential within HATs, where excessive trust can lead to an over reliance on AI systems, causing users to overlook mistakes and errors [10]. Conversely, insufficient trust may result in team members underutilizing the capabilities of AI, ultimately leading to reduced team performance [5]. Calibrating trust within HATs entails transitioning from black box AI methods to explainable AI, which is essential for successful trust calibration. Presenting AI outputs in a more human-friendly manner, integrating elements of Social Intelligence (SI) [9,11], proves to be a valuable approach for explaining AI and facilitating trust calibration [11].

Previous research has indicated emojis can play a significant role in enhancing SI within professional settings. Emojis offer a valuable means for AI to convey emotional states, thereby allowing AI systems to better interpret and respond to users' emotional cues and potentially calibrate trust successfully. Building on prior research, which has effectively employed emojis on platforms like Twitter to develop models for inferring affect from emoji usage patterns [1], the use of emojis can be extended to foster SI in HATs and could allow for mutual understanding of affective state from both the human teammate and AI teammate, Furthermore, in the domain of health-related applications, particularly those involving chatbots inquiring about participants' mental well-being, studies have demonstrated the positive impact of emojis. Chatbots that incorporate emojis have received higher ratings in terms of user enjoyment, attitude, and confidence [4]. Research has also indicated messages from chatbots featuring emojis were rated on par with those from human senders [3]. Additionally, both human and AI senders who utilized emojis were perceived as significantly more socially appealing, competent computer-mediated communication, and credible compared to senders who relied solely on verbal messages [3]. The incorporation of emojis into AI-mediated communication not only enhances the ability to understand and express affective states but also fosters positive user experiences and perceptions, aligning with the goals of social intelligence within work environments. From the current literature we pose the following hypotheses:

H 1 : Use of Emojis in AI responses will influence the decision-making process when determining which teammate to trust.

H 2 : Type of Emojis in AI responses will influence the decision-making process when determining which teammate to trust.

We used a mixed between-within subjects' design (2x3 configuration) where participants interacted with an AI teammate of either high (90%) or low (60%) reliability and a human teammate with 30% reliability. Within these separate groups participants then interacted with three different emoji uses, either face emojis, icon emojis or no emojis. We determined a sample size of N = 44 for 85% power to detect a medium effect in a two-way ANOVA (α = .05).

The study employed a Wizard of Oz experimental method to facilitate development convenience and ensure optimal control. Participants were led to believe they were collaborating with an AI and a human teammate when instead they were actually interacting with responses produced by ChatGPT. The task involved presenting participants with random locations extracted from Google Earth. Participants were tasked with determining the continent, country, and city associated with each location, with the final decision resting on the participant, who assumed the role of the 'team leader'. A time constraint of 120 seconds per location was enforced, meaning participants had to rely on their teammates' responses to submit the location in time. Notably, the AI and human team-mates provided conflicting answers 90% of the time, necessitating participants to discern which teammate they trusted more.

A total of 30 locations were identified by each participant across three blocks, comprising 10 trials per block. Each block either used Face Emojis (☹,

), Icon Emojis ( , ) or No Emojis. Following each trial, the correct answer was revealed, enabling participants to assess the performance of the human and AI teammates. On each trial participants rated which teammate influenced them most and which teammate they trusted most, at the end of each block participants completed the Trust in Automation Questionnaire [6], which is has six sub-sections (Trust, Familiarity, Understanding, Intentions of developers, Reliability of AI and Propensity to trust) to measure different elements of trust of the AI interacted with in the previous block, questionnaires were slightly altered to fit the zero embodiment scenario being explored. We collected the location responses from ChatGPT, a large language model, by inputting location descriptions and requesting versions with emojis distributed throughout them, minimal editing was needed to make the responses suitable. The AI's writing style mimicked humans, following previous successful approaches [2]. We conducted the experiment using PsychoPy and hosted it on Pavlovia.

A total of 42 participants from the University of Glasgow were recruited. The group consisted of 24 males and 18 females and had a mix of students (n = 27) and professionals (n = 17).

We conducted a two-way ANOVA with interactions to compare trust ratings of the AI. The analysis indicated that neither the type of emojis used in AI responses nor the reliability level of AI teammates had a significant impact on trust. Specifically, the main effects of emoji types (F(2, 4) = 0.647, p = 0.524) and reliability (F(1, 4) = 1.363, p = 0.243) were non-significant, as well as the interaction effect between emoji type and reliability (F(2, 4) = 0.554, p = 0.575). We also conducted a two-way ANOVA with interactions to compare influence ratings of the AI. The analysis indicated that neither emoji type nor reliability had a statistically significant impact on influence. Specifically, both the main effects of within-condition (F(2, 4) = 0.368, p = 0.692) and reliability (F(1, 4) = 0.010, p = 0.921), along with the interaction effect between emoji type and reliability (F(2, 4) = 1.493, p = 0.225), were found to be non-significant.

We also analyzed the Trust in Automation Questionnaire [6]by completing two ANOVAs on the various subsections assessing different dimensions of trust. For the subsection Trust, the analysis demonstrated a statistically significant effect of Reliability (F(1, 2) = 69.133, p = 0.0142) while emoji type showed no significant impact. Post hoc comparisons via Tukey method revealed significant differences for all emoji type, but only by reliability. For the Familiarity subsection, Reliability demonstrated a significant impact (F (1, 2) = 141.187, p = 0.007), while emoji type had no significant effect. Post hoc tests indicated that there were differences in emoji type but only by reliability, not between emoji types. In the Propensity to trust subsection, Reliability showed a significant effect (F(1, 2) = 30.990, p = 0.0308), emoji type did not significantly influence trust. Tukey post hoc tests did not identify specific trust differences based on emoji type and reliability. In the Reliability of AI, Understanding and Intentions of developers' subsections, neither Reliability nor emoji type had a significant effect on trust, with both showing p-values above 0.05.

The aim of this study was to investigate how emojis influence trust calibration within Human-AI teams (HATs) and what this means for team dynamics. While emojis have shown potential in improving human-computer interactions [3,4], our research revealed that their impact on trust calibration within HATs was not as significant as anticipated and neither of our research hypotheses were fully supported.

Contrary to our expectations, integrating emojis into AI-mediated communication did not enhance trust calibration between human team members and AI. Despite emojis offering a more human-friendly and emotionally expressive interface, their effect on trust calibration in HATs seemed limited. Several factors may explain these outcomes. Trust in HATs appears to be influenced by multifaceted dynamics that go beyond emotional cues. Transparency of AI systems [7], their past performance [12], and the unique traits of human team members [8] likely play crucial roles in trust development. Emojis, while enhancing emotional expressiveness, might not address these fundamental trust determinants in HATs.

Additionally, the specific nature of tasks in HATs, demanding precision, data analysis, and cognitive effort, might overshadow the emotional cues conveyed by emojis. the experimental task does not require any emotions, in other situations where emojis are found to be useful there is often a need for emotion, such as health care [4].

Our research did find significant results concerning participants' trust and familiarity with AI, these results were only for reliability. Participants rated highly reliable AI as significantly less familiar compared to less reliable AI. This suggests that proficient AI using humanized behavior effectively, might not be frequently encountered. These findings could explain why the high reliability AI received lower trust scores on the TIA, previous research has shown highly reliable AI with high humanness is less trustworthy than humanized low reliability AI [2], possibly influenced by anthropomorphic priming [12]. Although limited because these trust results were not replicated in experimental data but only in the questionnaire, they support past research indicating that users find AI more trustworthy when it appears more human-like, especially when the AI's performance is not perfect [13].

In conclusion, our research highlights the intricate nature of trust calibration within HATs and indicates that while emojis have potential in enhancing human-computer interactions, their impact on trust might be more restrained in this specific context. Future studies should delve deeper into the complexities of trust in HATs and explore strategies beyond emojis that can effectively foster trust in the evolving realm of human-AI collaboration.