Personalized medication planning is the process of generating a plan of drug administrations that meets a given set of medical goals that are specific to the individual patient. The planning process must take into account general health safety constraints, helpful or harmful interactions between drugs, and individual physiological differences in responses to medications. The resulting personalized medication plan defines what drugs are administered, when, and at what dosage: too little is ineffective; too much is toxic.
Medication planning is a complex process, manually carried out by healthcare professionals. Its complexity is often encountered in mitigating harmful drug interactions in patients with multiple diseases [1], or in combination therapy, where multiple medications are used to synergistically improve therapeutic effects while minimizing side effects [2,3]. Indeed, a combination of drugs can result in effects no drug can achieve alone [4].
To demonstrate the difficulty of solving medication planning problems, let us examine a relatively simple instance. Suppose the system is to consider only two types of medicine, both affecting a specific property of interest ๐. Assume that for both types, the time it takes the medicine to clear the body is 24 hours. Let us also assume there are five different available dosages for each medicine. The medical objective is to have property ๐ reach a level of at least 51 in the spleen, but not exceed 53.8. While this problem seems small, solving it could be challenging due to the many possible treatment plans. Assuming each medicine is allowed to be administered at most once, there are 24 โข 24 administration times and 5 โข 5 dosage options, resulting in 14,400 combinations.
Not only are there numerous treatment combinations, but each administration's effects are also non-linear and multidimensional, as they simultaneously impact several organs and change over time. Furthermore, because each patient may require a different treatment plan tailored to their specific needs and conditions, a new plan must be created for each individual. All of these characteristics make the problem even more challenging to compute manually.
To address the complexities of personalized medication planning, researchers have turned to artificial intelligence (AI) planning techniques. AI planning is a form of sequential decisionmaking over time. It aims to find an ordered set of actions that leads from an initial state to a goal state. In medication planning, the AI planning process aims to automatically find an ordered set of drug administrations that progresses from the current health condition of a patient to the desired medical condition. It must not violate any medical safety constraints along the way.
The wide variety of treatment combinations, as well as the non-linearity and multidimensionality of the treatment effects, make it challenging for users to comprehend the implications of a given plan. Moreover, as medication plans vary between patients, explanations or clarifications of each plan must be generated anew.
The complexity and sensitivity of medical treatment problems result in a high demand for explainability. Since mistakes in the treatment plan can have major irreversible consequences, including death, users are not willing to follow automatically generated treatment plans without understanding the reasoning and potential implications. Indeed, the use of explainable artificial intelligence planning (XAIP) [5] is important here. Its goal is to make the planners transparent to users, thereby establishing users trust.
We propose a framework for explainable general medication planning (XGMP). The approach we suggest visualizes the personalized treatment plan's effect on the patient's body, thus making the plan transparent and easy to understand, for healthcare professionals and even for non-professional users. We evaluated this approach using the general medication planning (GMP) representation by Alon et al., with medical data as reported in [6,7]. To the best of our knowledge, no steps were taken towards explainable medication planning.
Explainability is an emerging and crucial field in artificial intelligence (AI). Its primary objective is to make the automated decision-making processes of AI systems transparent and interpretable to users by providing explanations that elucidate the underlying reasoning behind the system's outputs. These explanations aim to foster trust and confidence in AI systems, enabling users to comprehend the rationale behind the decisions made by the system.
The demand for explainability becomes paramount in domains where human lives are at stake, such as healthcare and the legal system. In the healthcare domains, the consequences of erroneous decisions can be severe, e.g., misdiagnoses, and improper treatments. Consequently, users in this field are more likely to demand comprehensive explanations for the AI system's outputs, as the implications of incorrect decisions can be far-reaching and potentially life-altering.
Personalized medicine further increases the necessity for explainability. In personalized medicine, there is no single protocol to treat a certain condition. The treatment is tailored to the individual patient based on their background medical conditions, preferences, and factors such as age. The variety of possible treatments under various conditions complicates the problem and requires different explanations, as the treatment for one patient may not be relevant for another, even if the same medical goal is required.
We begin with a short description of the medication planning process and basic terminology. We then discuss the need for explainability in this domain.
The general medication planning problem (GMP) is concerned with selecting drugs to be administered, as well as determining the dosage and schedule of the chosen medications [8,9,10]. It is therefore, in some aspects at least, a generalization of dosing regimen planning [11], which assumes drugs have already been selected, and deals with administration dosages and schedules.
Medication planning is also closely related to the process of planning treatments for patients with multiple diseases, by merging available multiple single-disease clinical guidelines. The latter process includes substituting drugs when adverse or redundant interactions occur, adjusting and scheduling tests to monitor for such interactions, and other related tasks [12,13,14,15,16,17,18]. This process produces plans that span weeks or months, and involve selecting drugs from the set recommended by guidelines. It may also recommend medical tests, and actions to take per their results. In contrast, GMP is carried out from first principles, personalizing the dosage and hourly medication schedules, using models of how medicines (drugs) spread through the body and interact with it, and with each other (pharmacokinetic and pharmacodynamic models-see below). However, it does not address testing, or chronic conditions.
Medication planning is a model-based planning approach. It uses models that predict how drugs spread in the body (pharmacokinetic models), and how they interact with in (pharmacodynamic models). These are explained below.
Once a drug is introduced into the body, it is generally absorbed, carried and circulating by the bloodstream throughout the body. The drug reaches various biological sites (bio-sites) and may accumulate for some time, before it is eventually cleared out of the body. The concentration of a drug in various bio-sites, known as its biodistribution, undergoes changes over time, which can be described by pharmacokinetic (PK) models of varying complexity. These models range from simple 1-3 compartment exponential decay models [19,20] to more advanced models that account separately for multiple kinetic processes (see [21]). Alternatively, biodistribution trajectories can also be represented by explicit curves [22,6], obtained from clinical trials.
For example, in Figure 1, we see the biodistribution trajectories of a specific drug administered to a mouse (nanoparticle #11, in [6]). Drug concentrations (percentage of initial dosage per gram of tissue) were measured in four bio-sites (kidney, lung, spleen, liver), at several time points (measured in hours since the administration at time ๐ก 0 = 0). Such trajectories change between medicines, but may also change between patients. The horizontal axis shows the time since administration. The vertical axis shows the concentration per gram of tissue as a percentage of the injection dosage. Each line shows the PK trajectory at a different bio-site.
When the drug reaches a target bio-site, it may affect the properties of that bio-site. These effects can be characterized using pharmacodynamic (PD) models [23]. PD models describe the relationship between the drug concentration at any given bio-site, or the body as a whole in simple models, and the resulting therapeutic effect.
PK and PD models are combined to form PKPD models [23], which predict the expected magnitude of drug therapeutic effect over time. Figure 2 illustrates the connection between PK and PD models. The PK model yields the concentration of the administrated drug in the body at a specific time post-administration. Subsequently, the PD model utilizes this data to calculate the biochemical effect of that drug on the patient's body. PKPD models (and their component models) have been and continue to be an active area of research in medicine and pharmacology, with entire journals devoted to their investigation.
Medication planning involves medical goals that are specified in terms of properties of different bio-sites (or the body taken as a whole), taking into account temporal pharmaceutical dynamics and kinematics. It combines information about the rate of accumulation and clearance of drugs in different bio-sites (from PKPD models) with information about toxicity and personal health constraints and patient activities to meet target levels of the drug or its biological effects. The process then selects drugs, determines their dosage, and the schedule of their administrations to a patient.
Alon et al. [9,10] presented the general personalized medication planning, and used a planning representation using pddl+ [24] to plan using multiple drugs, affecting multiple bio-sites over time (e.g., as in Fig. 1). This representation allows for an arbitrary number of medicines, each may be administered repeatedly if needed. The interactions of the drugs are modeled, so that the planner can avoid harmful interactions, and replace one drug with another (or with a combination of drugs). This extends the work of Alaboud et al. [8], which introduced the use of automated (AI) planning to address medication planning for maintenance goals, of a single drug and its associated PK model. Unfortunately, these investigations has not addressed the need for explaining the resulting plans.
There are various aspects of explainable artificial intelligence planning (XAIP), all of which are missing in medication planning. One common XAIP technique is to intervene in the problem representation, forcing the planner to execute the user's suggestions [25,26]. The user iteratively asks questions, where each question yields a new plan. By comparing the original plan with the new plan derived from the user questions, users infer the reasons that led to the original plan. An XAIP investigation related to a medical domain is presented by Korikov et al. [27], which considers the appointment scheduling problem. They use a counterfactual explanation technique that explains to the user what should have been different in order to achieve the user's suggested outcome.
In this paper, we focus on visualization, as the basis for interaction with a user. In general, visualization methods can also be utilized to explain the planner choices. Chakraborti et al. [28] introduce a visualization of the top-๐ plans as a graph where nodes represent actions and edges represent the transitions between actions. This visualization does not allow visualizing durative actions (whose effects change over time), or actions taken simultaneously. Similarly, Kumar et al. [29] present a visualization system for classical planning domains, i.e., all variables have binary values. They allow for both changes in the domain (in action structures) and in the problem (in initial state and goal states). As medication planning is not carried out in classical planning domains (durative actions, simultaneous actions, constraints), it is not compatible with with the proposed visualization method.
We will briefly describe the GMP pddl+ representation as was proposed by Alon et al. [9,10].
A pddl+ planning problem [24] can be described by the following tuple: โจ๐ฑ, ๐ฎ, ๐ 0 , ๐, ๐ข, ๐, โฐ ^, ๐ซ ^โฉ where ๐ฑ is a set of state variables either propositional or nu-meric, ๐ฎ is a set of states, where each state is a complete assignment of values to all variables ๐ฃ โ ๐ฑ, ๐ 0 โ ๐ฎ is an initial state, ๐ is a set of constraints on possible assignments of values, and ๐ข is a goal description (a set of conditions over variables). ๐ is a set of instantaneous actions that change the values of variables when selected by the agent, and โฐ ^, ๐ซ ^sets of events and processes (resp.) that change the values of variables instantaneously or overtime, outside of the control of the agent. A plan is a timed sequence of (parallel) actions, which starts from the initial state and reaches a goal state while not violating any constraint in ๐.
From a medical perspective, a patient's body can be viewed as a set ๐ต of bio-sites, such as organs and blood. Basic pharmacological models often depict the entire body as a single bio-site (|๐ต| = 1), but in more complex models multiple bio-sites are represented.
Their work allows for the representation of ๐ต bio-sites, where each bio-site is represented as a set of ๐ biochemical properties. Their values are numeric fluent indicating the concentration levels or other measures of interest and generally vary between bio-sites. Table 1 shows an example of 12 property variables, used in the experiments. The property ๐ 11 (first row) measures the concentration of nanoparticle #11 (Fig. 1) in six different bio-sites, at a specific time (e.g., liver[๐ 11 ] = 2.97, and kidney[๐ 11 ] = 9.2). Data was taken from [6]. A second property, measuring the mu-opioid receptor (MOR) activity, is shown in the second row. Its values in this case are derived from PKPD model parameters reported elsewhere [7]. The initial state ๐ 0 of a patient's body may be represented by setting the values of properties, in each bio-site, to current values. For properties measuring drug concentration, initial values in all bio-sites are zero.
Organs The administration of dosage ๐ of a drug type ๐ at time ๐ก is represented as a pddl action. This representation allows for repetitive administrations of the same drug type as long as they are not administered at the same time. Drugs from different types may be taken n parallel.
Each drug administration may affect several bio-sites and several properties simultaneously. Note that several drug administrations may affect the same bio-site property simultaneously, even if these administrations were not at the same time, since administrations have durative effects.
The medicine effects over time are represented by pddl+ events. For every medicine ๐, there is at least a single property ๐ in every bio-site ๐, i.e., ๐[๐] โ ๐ฑ, which represents the concentration level of medicine ๐ in bio-site ๐. In this representation, the medicine levels are estimated directly from biodistribution trajectories (e.g., Fig. 1) in the problem description in pddl+.
As a drug is accumulated in a bio-site (measured by its concentration level), it causes changes in other biochemical properties within the same bio-site. These changes can be predicted using PD models. The combination of the PK and PD model types, known as a PKPD model, allows for the estimation of how the accumulation and clearance of a drug change in biochemical properties influence various bio-sites over time [19,23,30].
The PKPD drug effects are also represented using pddl+ events. This representation utilizes the direct action (direct effect) model, a common PKPD model in medical literature [23,30]. This model describes the relationship between the time-dependent concentration, and the effects of the drug, measured in relevant units varying between drugs.
Different drugs may affect the same property simultaneously. As these will be handled by different events, their effects will increase the value of the property according to the PKPD effect of the associated medicine type. This naturally follows the Loewe additive drug interaction model [31,23], whereby drugs can affect the same property, but at different "strength". Contraindicated drugs (may not be taken together) are handled by constraints (see below).
Goals ๐ข and Safety Constraints ๐ Given the definitions of states and actions above, it seems a simple matter to define goal states in terms of target levels for properties of interest, at a specific set of bio-sites (therapeutic sites). However, medically, the planner must also ensure that the levels of all properties are maintained at safe levels, before the target levels are reached, as well as after.
They use events to impose limits on the maximal and/or minimal values of a property at any moment. These limits can come from medical defaults, or they may be personalized for specific health conditions of a patient. For example, if a patient has diabetes, the glucose level must stay below a given threshold โ at all times. Such a constraint on the property ๐ of bio-site ๐ can be expressed as ๐[๐] โโท โ, where โโท โ {>, โฅ, =, โค, <}. Constraints can be also placed to prevent interactions between drugs.
The goal description ๐ข has two components in the pddl+ representation of GMP. The first involves specifying target levels for properties in the set of therapeutic sites. These target levels can be personalized and differ between patients. The second component ensures that constraints are maintained after these target levels are achieved.
Once the goal conditions are first satisfied at time ๐ก ๐ , safety constraints should be upheld not only in the interval [0, ๐ก ๐ ] but also in the extended interval [๐ก ๐ , โ), bearing in mind that action effects have finite durations. Thus, a second subgoal introduced using pddl+ checks that all administered medication had been eliminated from the patient's body after the first component has been achieved. Personalization Patients, who seek treatment for the same goal, vary in their medical history (e.g., background medical conditions leading to differences in safety constraints) and treatment preferences (e.g., due to age, sex, levels of activity). Two patients with the same medical goal may still require different treatment plans due to differences in background medical conditions, such as diabetes, pregnancy, etc. Patient diversity may cause differences in their PKPD responses, both in biodistribution trajectories, as well as ๐ธ ๐๐๐ฅ and ๐ธ๐ถ 50 parameters.
Using clinical data from mice and rats, we implemented medication planning problems in pddl+ as described. The PKPD models are taken from databases of possible nanoparticle-based drug carriers [6] and pain relief drugs [7]. We use the ENHSP-20 numeric planner [32] to solve medication problems (see [10] for more details).
We propose visualizing the PK and PD effects to help users understand the treatment's impact on the patient body and easily compare between plans.
Consider a medication planning problem where the goal is to achieve a minimal value of 44 for mu-opioid receptor (MOR) activity in the heart and a maximal value of 46 (i.e., there is a safety constraint of 46 for this property in the heart) 1 . The planner suggested the following plan (denoted as plan A; dosage is measured in units of nM (nano-molar)): 0: (administer_med m9 a1 d100) 0: --waiting--[2.0] 2.0: (administer_med m10 a1 d100) 2.0: --waiting-- [28.0] This plan suggests administering a dosage 100nM of medicine ๐9, waiting two hours and then administering a dosage 100nM of medicine ๐10. Lastly, the planner waits another 26 hours until clearance occurs at time 28 from the beginning of the plan. While this very common plan format is detailed enough to follow, the exact effect of the plan on the body remains unclear. Understanding the pharmacokinetic (PK) and pharmacodynamic (PD) effects among users will increase their trust in the system and the suggested treatment and making the process transparent.
Figure 3 presents the visualization of the plan A (described above). The horizontal axis shows the time (in hours) since the start of the plan (first administered drug). The vertical axis shows MOR activity levels. The red line shows the PKPD effect of administering 100 nM of ๐9 alone at time 0, while the blue line shows the PKPD effect of administering 100 nM of ๐10 alone two hours after the plan start (i.e., had it been injected without ๐9 being present). The horizontal line (light blue, a level of MOR activity of 46) describes the safety constraint. The total PKPD effect of the two drugs is shown in purple. All of these are presented for the heart: the curves would be different for other bio-sites.
The visualization also highlights limitations of the clinical data. The sudden drops to zero towards the end of the plan span (i.e., 23-25 hours) signify that the different drugs have cleared the bio-site and thus their effect is reduced. In reality, we expect such clearance to be more gradual. However, the planning process is restricted to using the actual clinical data given in [6], which provides measurements at a resolution of a few hours (i.e., approximately 3-10 data points, depending on the drug and bio-site). The planner uses linear interpolation for points in-between measured data. As the actual clearance time is not given, the planner extrapolates it from the last point given in the data, towards zero. The result is that the clearance time is arbitrarily set as an hour following the last measured data point available from the database.
The PK and PD effects of an alternative plan can be visualized side-by-side with the original plan. This approach highlights the differences between the plans, allowing for a clear and easy comparison of their respective effects. Such differences may be due to variantions in drug administration timing or dosages, or due to differences in medical conditions or considerations.
Comparing plans with alternative schedules for the same drugs The visualization can demonstrate differences in effects due to variations in administration timing and dosages. Consider the plan effect described in Figure 3. One might ask: "What would be the effect of the plan if both medicines were administered simultaneously?". The plan (denoted plan B) produced by the planner in this case is two hours shorter than plan A: 0: (administer_med m9 a1 d100) 0: (administer_med m10 a1 d100) 0: --waiting--[26.0] Figure 4 presents the PKPD effect of executing plan B. Contrasting it with plan A (Fig. 3), we indeed see that while plan B is two hours shorter than plan A, it violates the medical safety constraint. Comparing Plans with Alternative Medical Considerations Personalization of the medical safety constraints, or other medical considerations, may also alter the plan. It may help the user visualize and contrast alternative treatment plans, resulting from such considerations. Consider the medication planning problem of achieving MOR activity of 29 in the heart, maintaining it lower than 37 in the same bio-site. The planner initially suggested administering 20nM of ๐5, which in our experiments is an opiod (specifically, Morphine). Due to its severe potential side-effects (one of which is long-term formation of dependence), a medical professional may ask for an alternative that excludes Morphine. When we pose this to the planner, it indeed finds a plan using a different medicine (๐11) at a dosage of 40nM. Figure 5 clearly shows the differences between the two suggested treatment plans. While both plans achieve the medical goal without violating the safety constraint, the plan involving medicine ๐5 has a shorter duration of 26 hours, compared to the plan using medicine ๐11 (50 hours). Additionally, the ๐5 plan has a lower maximal MOR activity in the heart compared to the ๐11 plan. However, the ๐11 plan achieves the desired medical goal (MOR activity of 27) more quickly.
We introduced a visualization method for the general medication planning (GMP), a relatively new and underexplored area in personalized medical treatment planning. The experiments conducted using real medical data showcased the effectiveness of this method in making unclear treatment plans easily understandable, even for non-professional users.
While the visualization method offers simplicity and clarity, it currently is not able to present other reason for using/not using certain medications, such as potential nausea, dizziness, weakness, and resistance to the administered medicines. Modeling such effects remains a future work.
Alexander Shleyfman's work was partially supported by ISF grant #2443/23. G.K. thanks K. Ushi, as always. Alon is grateful to the Azrieli Foundation for the award of an Azrieli Fellowship.