Negotiation in Intent-Based Interactions: Bridging the User Interface Accessibility and Usability Gap with LLMs

Keywords

• Usability

1. Background

Due to limitations in cognitive, linguistic, and perceptual abilities, individuals with Intellectual Disabilities (ID) face substantial barriers when interacting with digital systems [ 1 ]. These challenges include dificulties in understanding abstract symbols, executing multi-step tasks, interpreting dynamic interfaces, and handling error feedback [ 2 ]. While the W3C Web Content Accessibility Guidelines (WCAG) recommend principles such as simplified language, consistent navigation, and error prevention [ 3 ], they mainly address static Web content and generalized user models, often overlooking the diverse abilities and evolving needs of users with ID. As a result, despite the increasing ubiquity of digital technologies, many systems remain inaccessible or inefective for this population [ 4 ].

Large Language Models (LLMs), with their advanced capabilities in natural language understanding, generation, and reasoning, have the potential to revolutionize user interfaces and lay the foundation for intent-based interaction paradigms [ 5 ]. In such paradigms, users express their goals or needs through natural language, while the system interprets and acts on these inputs, abstracting away technical complexity. This shifts the user focus from

how to perform a task to articulating what they want to achieve [ 5, 6 ]. This evolution creates unprecedented opportunities for individuals with ID to engage meaningfully with intelligent agents, digital services, and low-code platforms. It empowers them to perform daily tasks independently and manage their smart environments, thereby enhancing autonomy and reducing reliance on caregivers [ 4, 7 ].

Nevertheless, relying solely on natural language is often insuficient to bridge the gap between users’ interpret ambiguous commands (e.g., ”make the room warmer”) due to the inherent vagueness of natural language [ 8, 9 ]. Another concern is that LLMs may produce non-deterministic, flawed, or hallucinated outputs that non-experts often struggle to evaluate and correct, thereby posing significant risks to user

CEUR Workshop

ISSN1613-0073 experience, privacy, and security [ 10, 11 ]. These risks may be further exacerbated when user knowledge or preferences conflict with system safety requirements [ 12 ].

In recent years, negotiation mechanisms have gained attention as a promising means of fostering mutual understanding in human–AI interactions, highlighting their potential to address the aforementioned challenges with innovative and practical solutions. Studies show that LLMs such as ChatGPT-4, with appropriate prompting, can efectively identify ambiguities and errors [ 13 ]. Through iterative user feedback, the model can progressively refine its understanding, enhance self-repair capabilities, and improve output quality over successive rounds [14]. Consequently, interactive negotiation for clarifying ambiguities, resolving conflicts, and facilitating collaborative decision-making is regarded as more reliable and beneficial for enhancing user satisfaction than direct generation approaches [ 13, 15 ].

2. Related Works

2.1. Intent Disambiguation and Multimodal Interaction

Existing research has proposed disambiguation strategies for conversations with AI chatbots, such as follow-up clarifications, rephrasing, and suggestive questions [ 16]. Other approaches aim to elicit user mental models through compositional paradigms, such as the Rule_5W framework, which helps users define articulated rules [ 17]. However, these methods often rely on specific communication skills, including grammatical, sociolinguistic, discourse, and strategic abilities [18, 19], which may not match the profiles of individuals with ID. In addition, limited abstract reasoning and dificulties translating intentions into structured commands further hinder their ability to design prompts and interact with systems efectively [ 1 ], potentially exacerbating their marginalization in digital technologies [ 4 ].

A growing body of research highlights the value of multimodal interfaces in supporting both the interpretation and expression of user intents [ 8 ]. For example, platforms such as IFTTT, Atooma, and Locale use icon-based visual languages to represent rules, enabling end-users to manage and customize their IoT-enabled smart environments more intuitively [17, 20] (e.g., see Figure 1). Expanding on this approach, Calò and De Russis combined LLMs with visual cues to clarify commands and capture nuanced user intents [ 8 ] (see Figure 2).

For individuals with neurodevelopmental disorders, research indicates that incorporating embodied and multimodal interaction into technology design can reduce cognitive load and enhance information comprehensibility [21]. Intelligent personal assistants that integrate voice commands with adaptive interfaces have proven efective in supporting daily tasks and independence in this population [ 22]. Spitale et al. further demonstrated that physically socially-assistive robots outperform virtual ones in boosting language skills and engagement during speech therapy for children with language impairments [23]. Additionally, Morra et al. [24] developed a tangible toolkit that enables users with ID to engage in technology creation through physical manipulation and visual–tactile afordances, thereby improving system usability and fostering skill development, self-confidence, and autonomy [ 25] (see Figure 3).

Creating Rule Events

JustAwake from 07 a.m. to 09 a.m. or

AlarmRinging from 07 a.m. to 09 a.m.

Add a new Event

2.2. Negotiation in Human-Agent Dialogues

Recent breakthroughs in Generative AI have spurred advancements in human-machine negotiation dialogue systems [26]. These systems typically involve goal-oriented, multi-turn interactions between humans and dialogue agents [27] (see Figure 4). Their core mechanisms integrate logical reasoning, dynamic strategies (e.g., argumentation, persuasion, confrontation, and compromise), and psychological factors to reach mutually acceptable solutions through strategic information exchange [28] (see Figure 5).

Negotiation strategies for dialogue agents are commonly categorized into integrative, distributive, and multi-party types [28]. Integrative negotiation promotes mutual gain through eliciting preferences, empathy, and coordinated proposals [26]. Distributive negotiation aims to maximize unilateral interests, employing tactics like contesting, empowerment, biased processing, and avoidance [29]. Multi-party negotiation requires modeling group dynamics, often addressed through reinforcement learning [ 30] or graph neural networks [31] to analyze complex subgroup interactions.

Beyond strategic considerations, the personality traits of negotiators also play a critical role in the negotiation process [28]. This involves mind modeling, which includes assessing psychological preferences, inferring intent, and predicting responses by mapping utterances to dialogue behaviors. It also involves understanding the emotional dynamics between negotiators, often subjectively measured by outcome satisfaction and partner perception [26].

Negotiation Cycle

Preference Negotiator

A Strategy

Potential Outcome Negotiations Outcome Interaction

Preference Negotiator

B Strategy

Deal Accepted Information Exchange

Not Accepted Conversational Agent

Human

3. Research Objectives

Although LLMs can understand complex instructions and generate coherent and contextually relevant responses, current LLM-based systems still struggle with ambiguous intents, conflicting user preferences, and consistent reliability [ 10 ]. Moreover, existing prompt and interface designs often fail to account for the communication, cognitive, and operational limitations of individuals with ID, resulting in reduced usability and compromised safety [ 1, 4 ]. This underscores the urgent need for innovative approaches that enable more inclusive technology experiences.

This research will explore the synergy between iterative negotiation strategies and multimodal interaction paradigms, and investigate how they can be efectively integrated into LLM-driven conversational agents. The goal is to enable collaborative interactions that harmonize user intent with agent capabilities, allowing both parties to jointly shape outcomes, thereby enhancing the user’s sense of control and trust. Ultimately, the research seeks to improve the accessibility and usability of digital systems and end-user development platforms [33], with a specific focus on supporting individuals with ID as a key beneficiary group by delivering more reliable, accurate, and user-centered interaction outcomes. To achieve these objectives, this research will address the following questions: • What core barriers do individuals with ID face in intent-based human–agent interactions? What types of tasks and preferences do they commonly exhibit? • Which multi-turn negotiation strategies efectively clarify ambiguous intents and resolve conflicts between user preferences and safety needs? • How can query prompts in human-agent negotiation dialogues be categorized to align with specific disambiguation goals and collaborative decision-making contexts? • Which multimodal interaction paradigms support users with ID in expressing their intent and interpreting system responses during iterative negotiation with LLM-based conversational agents?

4. Methodology and Early Progress

As part of the preliminary research, a systematic review is being conducted to establish a theoretical foundation for understanding iterative negotiation between humans and conversational AI agents in the context of HCI. This review aims to identify research gaps, track emerging trends, and inform the design of subsequent experiments.

Next, the study will employ Research through Design [34] and Participatory Design methods [35], engaging users with ID in iterative prototyping, evaluation, and refinement of multimodal interfaces with multi-round negotiation strategies through workshops and focus groups. The study will be driven and validated by three key use cases illustrating the benefits of assistive technologies for this population [36]: daily self-management (e.g., task automation, smart home personalization), skill development (e.g., task guidance, creative expression), and social participation (e.g., communication, emotional support).

Evaluation will follow a multi-stage, mixed-methods approach [37], combining short-term usability testing with long-term user experience tracking. Data will be gathered through qualitative methods (e.g., interviews, contextual observation, software logs [ 38]), and multimodal techniques (e.g., eye-tracking, speech emotion analysis, gesture recognition), alongside quantitative metrics like task completion rates [39], negotiation success rates (e.g., F1 scores [26]), and user satisfaction (e.g., UEQ scale [40]). The collected data will be analyzed using thematic analysis, statistical methods, and triangulation, with comparative studies validating solution efectiveness against existing approaches.

Throughout the research, a lessons-learned approach will identify challenges and opportunities from prototype development and user studies, distilling them into design patterns and toolkits that can favor the replicability and applicability of the acquired knowledge in independent design contexts.

5. Expected Results

The research will deliver an iterative negotiation framework for human–AI collaboration, integrating user modeling, scenario-based prompt strategies, and multimodal interaction paradigms. The framework is designed to be tailored to the abilities and contextual needs of individuals with ID, enabling them to control interconnected services, apps, and devices at an appropriate level of abstraction through intentbased natural language interactions with LLM-powered agents. The goal is to resolve ambiguities and conflicts through user-centered automated negotiations [ 41], creating smarter, more inclusive, and fluid interaction experiences. Furthermore, this research will complement and extend the W3C WCAG [ 3 ] with interface design guidelines that shift accessibility from static compliance and one-way information delivery toward dynamic empowerment through collaborative, two-way interaction. The outcomes will also inform interaction model toolkits and define the functional and architectural requirements for digital systems and end-user development platforms [33], supporting the full process from requirements specification to system deployment.

From a broader societal perspective, this research aims to discover the potential of integrating AI into daily life while considering the needs and complexities of the communities they are intended to serve. It promotes the flexible, cross-domain application of natural language technologies in assistive intelligent interaction [25] and mitigates the inequalities that techno-solutionism can generate [42]. The proposed framework will indeed enhance system controllability and digital autonomy for users with ID and other vulnerable populations, with potential benefits in language training, cognitive enhancement, and mental well-being.

6. Conclusion

Individuals with ID face significant challenges interacting with digital systems. Although LLMs ofer promising natural language capabilities, current systems still struggle with ambiguity, conflicting user preferences, and flawed outputs. This research will investigate integrating iterative negotiation strategies and multimodal interactions (e.g., visual symbols, speech, gestures) into LLM-driven agents. These combined approaches aim to enhance system usability and accessibility, empowering users with ID to engage in intent-based tasks, thereby improving their autonomy and quality of life. The primary expected outcome is a negotiation framework featuring personalized and intelligent interaction experiences for individuals with ID and broader vulnerable groups. The research will also deliver guidelines, toolkits, and system architecture requirements to promote digital inclusion and well-being.

Declaration on Generative AI

During the preparation of this work, the author used ChatGPT and Grammarly in order to: Grammar and spelling check, Paraphrase, and reword. After using this tool/service, the author reviewed and edited the content as needed and takes full responsibility for the publication’s content. [14] Y. Liu, T. Le-Cong, R. Widyasari, C. Tantithamthavorn, L. Li, X.-B. D. Le, D. Lo, Refining chatgptgenerated code: Characterizing and mitigating code quality issues, TOSEM 33 (2024) 1–26. [15] N. Lainwright, M. Pemberton, Assessing the response strategies of large language models under uncertainty: A comparative study using prompt engineering, OSF Preprints 1 (2024). [16] K. Keyvan, J. X. Huang, How to approach ambiguous queries in conversational search: A survey of techniques, approaches, tools, and challenges, CSUR 55 (2022) 1–40. [17] G. Desolda, C. Ardito, M. Matera, Empowering end users to customize their smart environments: model, composition paradigms, and domain-specific tools, TOCHI 24 (2017) 1–52. [18] D. Dippold, “can i have the scan on tuesday?” user repair in interaction with a task-oriented chatbot and the question of communication skills for ai, Journal of Pragmatics 204 (2023) 21–32. [19] J. D. Zamfirescu-Pereira, R. Y. Wong, B. Hartmann, Q. Yang, Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts, in: Proc. of CHI, 2023, pp. 1–21. [20] B. Ur, E. McManus, M. Pak Yong Ho, M. L. Littman, Practical trigger-action programming in the smart home, in: Proc. of CHI, 2014, pp. 803–812. [21] F. Catania, M. Spitale, F. Garzotto, Conversational agents in therapeutic interventions for neurodevelopmental disorders: a survey, CSUR 55 (2023) 1–34. [22] M. K. Wolters, F. Kelly, J. Kilgour, Designing a spoken dialogue interface to an intelligent cognitive assistant for people with dementia, Health Inform. J. 22 (2016) 854–866. [23] M. Spitale, S. Silleresi, F. Garzotto, M. J. Matarić, Using socially assistive robots in speech-language therapy for children with language impairments, Int. J. Soc. Robot. 15 (2023) 1525–1542. [24] D. Morra, G. Caslini, M. Mores, F. Garzotto, M. Matera, Makenodes: Opening connected-iot making to people with intellectual disability, IJHCS 190 (2024) 103325. [25] M. C. Safari, S. Wass, E. Thygesen, Motivation of people with intellectual disabilities in technology design activities: the role of autonomy, competence, and relatedness, BIT 42 (2023) 89–107. [26] K. Chawla, J. Ramirez, R. Clever, et al., CaSiNo: A corpus of campsite negotiation dialogues for automatic negotiation systems, Proc. NAACL-HLT 2021 1 (2021) 3167–3185. [27] Z. Zhang, L. Liao, X. Zhu, T.-S. Chua, Z. Liu, Y. Huang, M. Huang, Learning goal-oriented dialogue policy with opposite agent awareness, arXiv preprint arXiv:2004.09731 (2020). [28] H. Zhan, Y. Wang, Z. Li, et al., Let‘s negotiate! a survey of negotiation dialogue systems, in:

Y. Graham, M. Purver (Eds.), Findings of EACL 2024, ACL, St. Julian’s, Malta, 2024, pp. 2019–2031. [29] M. L. Fransen, E. G. Smit, P. W. Verlegh, Strategies and motives for resistance to persuasion: An integrative framework, Frontiers in Psychology 6 (2015) 1201. [30] K. Georgila, C. Nelson, D. Traum, Single-agent vs. multi-agent techniques for concurrent reinforcement learning of negotiation dialogue policies, in: Proc. ACL, 2014, pp. 500–510. [31] J. Li, M. Liu, Z. Zheng, et al., Dadgraph: A discourse-aware dialogue graph neural network for multiparty dialogue machine reading comprehension, in: Proc. IJCNN, IEEE, 2021, pp. 1–8. [32] J. Brett, L. Thompson, Negotiation, OBHDP 136 (2016) 68–79. [33] H. Lieberman, F. Paternò, M. Klann, V. Wulf, End-user development: An emerging paradigm, in:

End user development, Springer, 2006, pp. 1–8. [34] J. Zimmerman, J. Forlizzi, S. Evenson, Research through design as a method for interaction design research in hci, in: Proc. of CHI, 2007, pp. 493–502. [35] M. J. Muller, S. Kuhn, Participatory design, CACM 36 (1993) 24–28. [36] A. Klavina, P. Pérez-Fuster, J. Daems, et al., The use of assistive technology to promote practical skills in persons with autism spectrum disorder and intellectual disabilities: A systematic review, Digital Health 10 (2024) 20552076241281260. [37] J. Brannen, Mixing methods: The entry of qualitative and quantitative approaches into the research process, Int. J. Soc. Res. Methodol. 8 (2005) 173–184. [38] D. M. Hilbert, D. F. Redmiles, Extracting usability information from user interface events, CSUR 32 (2000) 384–421. [39] B. Albert, T. Tullis, Measuring the user experience: collecting, analyzing, and presenting usability metrics, Newnes, 2013. [40] M. Schrepp, A. Hinderks, J. Thomaschewski, Construction of a benchmark for the user experience questionnaire (ueq), Int. J. Interact. Multimed. Artif. Intell. 4 (2017) 40–44. [41] F. Paternø, M. Burnett, G. Fischer, et al., Artificial intelligence versus end-user development: a panel on what are the tradeofs in daily automations?, in: Interact, Springer, 2021, pp. 340–343. [42] D. Wang, R. Denton, A. K. Sinha, S. e. a. Sheth, Unanticipated lessons from communities: Navigating society-centeredresearch in the ai era, in: Proc. of CHI EA ’25, ACM, New York, NY, USA, 2025.

References

[1] A. P. Association , et al., Diagnostic and statistical manual of mental disorders: DSM-5 , American psychiatric association, 2013 .

[2] A. W. Wong , C. C. Chan , C. W. Li-Tsang , et al., Competence of people with intellectual disabilities on using human-computer interface , Research in Developmental Disabilities 30 ( 2009 ) 107 - 123 .

[3] W3C , Web content accessibility guidelines (wcag) 2.1 , https://www.w3.org/TR/WCAG21/, 2018 . Accessed: 2025 -07-14.

[4] D. Lussier-Desrochers , C. L. Normand , A. Romero-Torres , et al., Bridging the digital divide for people with intellectual disability , Cyberpsychology: Journal of Psychosocial Research on Cyberspace 11 ( 2017 ).

[5] Z. Ding , Towards intent-based user interfaces: Charting the design space of intent-ai interactions across task types , arXiv preprint arXiv:2404.18196 ( 2024 ).

[6] B. Shneiderman , P. Maes , Direct manipulation vs. interface agents, interactions 4 ( 1997 ) 42 - 61 .

[7] R. Sheehan , A. Hassiotis , Digital mental health and intellectual disabilities: state of the evidence and future directions , BMJ Ment Health 20 ( 2017 ) 107 - 111 .

[8] T. Calò , L. De Russis, Enhancing smart home interaction through multimodal command disambiguation , Personal and Ubiquitous Computing 28 ( 2024 ) 985 - 1000 .

[9] S. R. Petrick , On natural language based computer systems , IBM J. Res. Dev . 20 ( 1976 ) 314 - 325 .

[10] S. Ouyang , J. M. Zhang , M. Harman , M. Wang , Llm is like a box of chocolates: the non-determinism of chatgpt in code generation , arXiv e-prints ( 2023 ) arXiv- 2308 .

[11] G. L. Scoccia , Exploring early adopters' perceptions of chatgpt as a code generation tool , in: ASEW, IEEE, 2023 , pp. 88 - 93 .

[12] A. Stampf , M. Colley , B. Girst , E. Rukzio, Exploring passenger-automated vehicle negotiation utilizing large language models for natural interaction , in: Proc. AutomotiveUI , 2024 , pp. 350 - 362 .

[13] M. Andrao , D. Morra , T. Paccosi , et al., ” this sounds unclear”: Evaluating chatgpt capability in translating end-user prompts into ready-to-deploy python code ., in: Proc. AVI , 2024 , pp. 1 - 4 .