Medication safety poses a significant challenge in healthcare, with adverse reactions harming patients and increasing costs. We present two solutions based on Large Language Models (LLMs): HELIOT, a Clinical Decision Support System for managing adverse drug reactions, and an approach that uses textual-drug information to predict drug-drug interactions (DDIs). HELIOT examines patient-specific clinical narratives to provide contextual alerts, reducing alert fatigue by over 50%. Our DDI system analyzes molecular structures, organisms, and drug target genes, achieving a sensitivity of 0.978 and an accuracy of 0.919 across 13 validation datasets, with smaller LLMs (2-3 billion parameters) outperforming larger ones. Both systems demonstrate the potential of LLMs to enhance medication safety through advanced language processing and pharmaceutical data interpretation. Future work will focus on practical validation and integration into healthcare systems.
The annual burden of medication management errors in England’s healthcare system is striking, with
approximately 237 million recorded incidents, 28% of which pose significant clinical risks, resulting in
thousands of deaths and substantial financial costs. Similarly, in the United States, medication-related
mistakes and adverse drug reactions create an enormous economic impact, totaling billions of dollars
annually, with drug-drug interactions accounting for 18% of the amount spent addressing adverse
drug reactions (ADRs). Traditional Clinical Decision Support Systems (CDSSs) have limitations in this
context [
On the other hand, traditional DDI identification relies on time-consuming experimental methods [
The remainder of this paper is structured as follows. Section 2 presents our LLM-based CDSS for ADR management, detailing its approach, architecture, and evaluation results. Section 3 describes our
CEUR Workshop
ISSN1613-0073 text-based approach for DDI prediction, including its methodology and performance across multiple datasets. Section 4 outlines future directions and challenges for LLM applications in medication safety. Finally, section 5 concludes the paper.
CDSSs for medication safety face the challenge of balancing accurate alerting with alert fatigue. They struggle to interpret unstructured clinical narratives, including patient-specific medication tolerances and reaction histories. HELIOT is a novel LLM-based CDSS that leverages natural language processing to analyze clinical notes and provide contextually appropriate medication safety recommendations. Its innovation lies in interpreting unstructured text about patient medication experiences and diferentiating between reaction types. This section outlines HELIOT’s architecture, prototype development, and performance evaluation compared to traditional CDSSs.
HELIOT uses Retrieval Augmentation Generation (RAG) to help physicians make medication decisions
based on patients’ adverse reaction histories. RAG enhances LLMs by retrieving relevant
information before generating responses, improving accuracy and contextual relevance. The proposed CDSS
employs parallel retrieval processes, simultaneously gathering pharmaceutical data (ingredients,
contraindications, side efects) and analyzing patient clinical notes to identify problematic ingredients.
During this process, the system maintains and updates longitudinal patient records of adverse reactions
and tolerances across encounters, ensuring continuity of care even in facilities without integrated EHR
systems. The decision support logic utilizes the persona pattern [
To assess the potential of the proposed CDSS, we developed a full-functional prototype. To this aim,
we first created a comprehensive pharmaceutical knowledge base. Starting with the Italian Medicines
Agency (AIFA) 1 database containing 106,962 approved medications, we collected essential information
including drug codes, names, forms, and ATC (Anatomical Therapeutic Chemical) classifications. We
then acquired 19,188 leaflet files through Farmadati 2 web services, preprocessing them to obtain detailed
medication compositions and properties. To ensure clinical relevance, we created a representative subset
following literature-based distribution patterns [
We evaluated our system using 1,000 synthetic adverse reaction cases across seven clinical classes, achieving a macro-averaged F1 score of 0.9869. The system attained perfect classification for three critical categories: Direct Active Ingredient Reactivity, Drug Class Cross-Reactivity with Documented Tolerance, and Drug Class Cross-Reactivity Without Documented Tolerance. Significantly, the system could reduce interruptive alerts by 50.2% compared to traditional systems by appropriately categorizing cases as requiring interruptive alerts (45.5%), non-interruptive alerts (14.9%), or no alerts (39.6%). The system processed each case eficiently (average 2.775 seconds) due to optimizations including eficient data retrieval, an in-memory synonyms dictionary, and parallel database operations. These results suggest LLM-based approaches can substantially improve adverse drug reaction management over rule-based systems, particularly where clinical information exists primarily in unstructured formats.
Existing computational DDI prediction methods require complex feature engineering and specialized
architectures, limiting their practical implementation [
Our approach simultaneously processes multiple drug characteristics through textual inputs combining
SMILES notation (representing molecular structure), target organisms, and gene interaction information.
The underlying assumption follows established literature: ”two drugs potentially interact when a
drug alters the other drug’s therapeutic efects through targeted genes or signaling pathways” [
Our evaluation revealed distinct performance patterns across adaptation approaches. Zero-shot results showed limited efectiveness, with even proprietary models achieving modest sensitivity (0.5413-0.5927). Few-shot learning improved performance, particularly with similarity-based selection, reaching an accuracy of 0.8376 with Claude 3.5. Fine-tuning dramatically enhanced performance, with smaller models showing remarkable improvements. Across 13 external datasets, fine-tuned Phi-3.5 showed exceptional sensitivity (average 0.978) and a high accuracy (0.919), surpassing larger models such as GPT-4 and traditional baselines. These results ofer significant clinical implications. The high sensitivity of fine-tuned LLMs addresses a critical safety priority in medication management, where missing interactions pose greater risks than false positives. The superior performance of smaller LLMs enables deployment on standard hardware without specialized infrastructure, addressing accessibility and privacy concerns through local processing capabilities. These systems could support clinical decisionmaking throughout the medication management process, from pre-prescription screening to emergency medicine. The finding that task-specific adaptation outweighs model size suggests eficient pathways for developing focused AI tools for healthcare applications, potentially improving medication safety while maintaining operational eficiency in everyday clinical workflows.
Our LLM-based approaches are promising. HELIOT can process unstructured clinical narratives in various healthcare settings, from hospitals to primary care practices. At the same time, our DDI prediction approach ofers value for research and clinical prescribing, particularly in complex patientspecific situations. Future work will focus on validation with real clinical data, expanding capabilities to address more complex clinical scenarios, and optimizing performance for point-of-care deployment. Collaboration with healthcare institutions remains essential for the comprehensive evaluation of the impact of the proposed systems on clinical workflows and decision-making processes, ensuring approaches that improve medication safety while integrating with existing healthcare systems.
Our work demonstrates how LLMs can transform medication safety management through contextual understanding and natural language processing capabilities. By enabling systems to process unstructured clinical information and complex pharmaceutical data, we address fundamental limitations in current adverse drug reaction management and interaction prediction approaches. The performance of smaller LLMs highlights the feasibility of practical clinical deployment without extensive computational resources. These findings suggest a promising path forward for AI applications in healthcare that balance efectiveness with accessibility, potentially improving patient outcomes while seamlessly integrating into clinical workflows. Continued collaboration between technical researchers and healthcare practitioners will be essential as the field evolves for successful implementation.