=Paper=
{{Paper
|id=Vol-3825/poster2-1
|storemode=property
|title=A Beyond Diagnosis Approach: Fostering Trust in AI's
Supportive Role in Healthcare
|pdfUrl=https://ceur-ws.org/Vol-3825/poster2-1.pdf
|volume=Vol-3825
|authors=Mohammed Ali Tahtali,Corné Dirne
|dblpUrl=https://dblp.org/rec/conf/hhai/TahtaliD24
}}
==A Beyond Diagnosis Approach: Fostering Trust in AI's
Supportive Role in Healthcare==
https://ceur-ws.org/Vol-3825/poster2-1.pdf
A Beyond Diagnosis Approach: Fostering Trust in AI's
Supportive Role in Healthcare
Mohammed Ali Tahtali1,2, ∗,†, Corné Dirne2,†
1 Department of Industrial Engineering & Innovation Sciences, Eindhoven University of Technology,
Noord-Brabant, Eindhoven, The Netherlands
2 Department of Industrial Engineering en Management, Fontys University of Applied Sciences, Noord-
Brabant, Eindhoven, The Netherlands
Abstract
This study examines the potential of Artificial Intelligence (AI) to gain trust and improve
outcomes in healthcare by serving as a supportive tool rather than replacing human judgment.
Despite AI's advancements in diagnosis and treatment, scepticism among individuals persists
due to concerns over AI's lack of empathy and the importance of human expertise. By focusing
on AI's role as an augmentative rather than substitutive technology, we aim to identify
strategies for integrating AI in a manner that complements human skills, thereby enhancing
acceptance and trust in AI and improving care for patients.
Keywords
Trust in AI, Healthcare Integration and Augmentative AI Applications 1
1. Introduction
As Artificial Intelligence (AI) continues to advance, its role has significantly shifted from
performing routine, repetitive tasks to making complex decisions that traditionally
required expert human judgment. This transformation is particularly evident in critical
sectors such as government and healthcare, where AI is now instrumental in assisting with
decisions once solely made by judges, doctors and other trained professionals. AI's invasion
into these sectors parallels human expertise and, in some instances, even surpasses it in
terms of efficiency and accuracy. Applications in healthcare include, for example, AI for
image recognition in radiology [1], the use of AI for complex diagnoses [2] and personalised
treatment plans [3]. Despite AI's demonstrated accuracy in these critical tasks, there are
considerable challenges in healthcare, particularly scepticism among people about relying
on AI's decisions.
HHAI-WS 2024: Workshops at the Third International Conference on Hybrid Human-Artificial Intelligence
(HHAI), June 10—14, 2024, Malmö, Sweden
1∗ Corresponding author.
† These authors contributed equally.
m.a.tahtali@tue.nl (M.A. Tahtali); c.dirne@fontys.nl (C. Dirne)
0009-0009-8080-0959 (M.A. Tahtali); 0009-0001-2508-6168 (C. Dirne)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Delving deeper into this scepticism, the trust of medical professionals in AI, as explored
in the literature, reveals a multifaceted landscape. Longoni et al. [4] emphasise the concerns
over AI's ability to navigate complex, individual patient scenarios and maintain the human
elements of empathy and understanding in healthcare. Complementing this, research from
Juravle et al. [5], Promberger and Baron [6] and Seitz et al. [7] highlight a clear preference
for human decision-making over AI among both patients and healthcare providers, rooted
in human expertise and emotional empathy connections - areas where AI currently falls
short. Furthermore, the importance of explainability in AI systems, as emphasised in studies
by Tucci et al. [8], Schwartz et al. [9], Naiseh et al. [10] and Alam and Mueller [11], is critical,
arguing that making AI decisions understandable is vital for fostering trust among medical
professionals and patients.
However, it is essential to note that the majority of tasks AI has been assigned in these
studies focus on diagnosis, often sparking debate regarding its role and effectiveness.
Despite these challenges, research by Verma et al. [12], Tahtali et al. [13] and Longoni et al.
[4] suggests a potential area for AI to gain trust among medical professionals and possibly
among patients as well, when it supports rather than replaces the final decision-making
process. For instance, by automating triage and operational tasks, such as predicting bed
availability and summarizing patient consultations, AI significantly expands healthcare
capabilities [13]. This automation enhances patient care and underlines AI's transformative
impact on the sector. Highlighting AI's ability to extend beyond its current diagnostic roles,
the findings support a more widely accepted model where AI serves as a supportive tool
rather than a replacement. Such a paradigm shift could improve trust in AI applications
within healthcare.
Our study aims to explore and identify more effective and acceptable ways of integrating
AI into a broader range of healthcare tasks. By focusing on AI applications that complement
and augment human expertise rather than replacing it, we seek to foster wider acceptance
and trust in AI among patients. Consequently, the central research question we propose is:
How can AI applications in healthcare, extending beyond direct diagnosis, gain the trust of
patients and contribute to enhancing overall patient care?
2. Research design
Our (ongoing) study explores patients' trust in an AI smart camera, currently operated
as a supportive tool for monitoring individuals post-intensive care. The traditional method
involves daily manual patient assessments, such as vital signs. Yet, introducing this AI tool
aims to augment the existing process, potentially transforming it into a more continuous,
AI-driven monitoring system. However, patients' acceptance and trust in AI's (video based
monitoring) capabilities remain uncertain. Through a qualitative research approach, this
paper aims to uncover the nuances of acceptability surrounding the AI support tool,
emphasizing the camera's role as a complement to, rather than a replacement for, human
expertise in healthcare practices.
Employing semi-structured interviews, we engage patients to determine their
perspectives on and receptiveness to AI technology. We focus on various topics, such as the
camera's usability, trust in the signals it provides and the nurse-patient relationship. This
methodological choice facilitates a deeper understanding of their expectations and
apprehensions, thereby informing the development of AI technologies that foster trust and
meet the healthcare sector's needs. Our research is about gauging readiness and envisioning
AI's integration into healthcare workflows in ways that align with professional standards
and patient comfort. This research seeks to illuminate paths forward that ensure AI's role
in healthcare maximizes benefits while maintaining the irreplaceable value of human
clinical insight.
Acknowledgements
The authors extend their heartfelt gratitude to all participants willing to share their
insights and experiences, thereby making this research possible. Special thanks go to
Noortje Somers for her assistance in conducting interviews, performing analyses and
participating in discussions.
This work is part of the project "Trust in Algorithms" (project number 023.017.109),
under the auspices of the Teacher Promotion Grant program, generously funded by the
Dutch Research Council (NWO).
References
[1] M. Huisman et al., “An international survey on AI in radiology in 1,041 radiologists
and radiology residents part 1: fear of replacement, knowledge, and attitude,” Eur.
Radiol., vol. 31, no. 9, pp. 7058–7066, 2021, doi: 10.1007/s00330-021-07781-5.
[2] A. V. Eriksen, S. Möller, and J. Ryg, “Use of GPT-4 to Diagnose Complex Clinical Cases,”
Nejm Ai, vol. 1, no. 1, pp. 2023–2025, 2023, doi: 10.1056/aip2300031.
[3] Y. Meng, J. Sun, N. Qu, G. Zhang, T. Yu, and H. Piao, “Application of radiomics for
personalised treatment of cancer patients,” Cancer Management and Research, vol.
11. Dove Medical Press Ltd, pp. 10851–10858, 2019. doi: 10.2147/CMAR.S232473.
[4] C. Longoni, A. Bonezzi, and C. K. Morewedge, “Resistance to Medical Artificial
Intelligence,” J. Consum. Res., vol. 46, no. 4, pp. 629–650, Dec. 2019, doi:
10.1093/jcr/ucz013.
[5] G. Juravle, A. Boudouraki, M. Terziyska, and C. Rezlescu, “Trust in artificial
intelligence for medical diagnoses,” in Progress in Brain Research, vol. 253, Elsevier
B.V., 2020, pp. 263–282. doi: 10.1016/bs.pbr.2020.06.006.
[6] M. Promberger and J. Baron, “Do patients trust computers?,” J. Behav. Decis. Mak., vol.
19, no. 5, pp. 455–468, 2006, doi: 10.1002/bdm.542.
[7] L. Seitz, S. Bekmeier-Feuerhahn, and K. Gohil, “Can we trust a chatbot like a
physician? A qualitative study on understanding the emergence of trust toward
diagnostic chatbots,” Int. J. Hum. Comput. Stud., vol. 165, Sep. 2022, doi:
10.1016/j.ijhcs.2022.102848.
[8] V. Tucci, J. Saary, and T. E. Doyle, “Factors influencing trust in medical artificial
intelligence for healthcare professionals: a narrative review,” J. Med. Artif. Intell., vol.
5, no. November 2021, pp. 0–2, 2022, doi: 10.21037/jmai-21-25.
[9] J. M. Schwartz et al., “Factors Influencing Clinician Trust in Predictive Clinical
Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive
Study,” JMIR Hum. Factors, vol. 9, no. 2, Apr. 2022, doi: 10.2196/33960.
[10] M. Naiseh, D. Al-Thani, N. Jiang, and R. Ali, “How the different explanation classes
impact trust calibration: The case of clinical decision support systems,” Int. J. Hum.
Comput. Stud., vol. 169, Jan. 2023, doi: 10.1016/j.ijhcs.2022.102941.
[11] L. Alam and S. Mueller, “Examining the effect of explanation on satisfaction and trust
in AI diagnostic systems,” BMC Med. Inform. Decis. Mak., vol. 21, no. 1, Dec. 2021, doi:
10.1186/s12911-021-01542-6.
[12] H. Verma et al., “Rethinking the Role of AI with Physicians in Oncology: Revealing
Perspectives from Clinical and Research Workflows,” in Conference on Human
Factors in Computing Systems - Proceedings, Apr. 2023. doi:
10.1145/3544548.3581506.
[13] Tahtali MA, Snijders C, Dirne C, Le Blanc P. “Prioritising Trust: Podiatrists' Preference
for AI in Supportive Over Diagnostic Roles in Healthcare: A Qualitative Analysis”,
JMIR, Preprints. 30/03/2024:59010. doi: 10.2196/preprints.59010