When automating plan generation for a real-world sequential decision problem, the objective is often not to replace the human planner, but rather to facilitate an iterative reasoning and elicitation process. In this process, the human's role is to guide the planner according to their preferences and expertise. In this context, explanations that address users' questions are crucial to improve their understanding of potential solutions and increase their trust in the system. We present a platform that implements this iterative planning approach and provides explanations to user questions based on conflicting goals and preferences. The platform supports both a classical template-based interface and a multi-agent Large Language Model (LLM) architecture that enables interactive explanations tailored to the user and context. The integration of online user studies allows for the evaluation of the efectiveness of the explanations and the impact of the communication interface. The platform code is available at https://github.com/r-eifler/IPEXCO-frontend.
Real-world problems are often oversubscribed, meaning that due to constraints such as limited resources
or time, not all goals and user preferences can be satisfied. An iterative planning process, as outlined
in [
In the following we provide an overview of the main platform features, the iterative planning process, goal conflict explanations and the user study support. Then we present the interactive explanation interface based on a multi-agent LLM architecture. Finally, we give an overview of the modular platform architecture.
The platform implements an iterative planning process. In each iteration step, the user is presented a sample plan (if one exists). Based on , the user must then decide which goals and preferences the sample plan +1 in the next iteration should satisfy. If the goals and preferences enforced by the user are unsolvable, no sample plan can be computed. To restore solvability, the user must then decide which goals and preferences to forego. During the exploration of the plan space new preferences may emerge, and goals which are to restrictive might be dropped. Our platform facilitates this iterative process by ofering interfaces to access the individual iteration steps, along with the preferences and goals satisfied in each step. Additionally, it allows users to define temporal goals, reflecting their preferences. The planning task is given by a PDDL domain and problem file and the goals are defined during the planning process via diferent interface options, which are outlined next.
Goal Creation At the beginning of the planning process, the user defines all known goals and
potential preferences. As the planning process progresses, the initial goals and preferences are refined
based on the analysis of sample plans. Goals are defined by LTL f [
First, it is possible to define templates for each domain to cover commonly used goals and temporal preferences. Such a template maps a natural language description, e. g. “Load package $Pi before package $Pj into truck $T” to an LTLf formula: ¬( , ) U (, ). To restrict the objects ofered for selection, the allowed types (e.g. $Pi must be of type package) and facts that must/must not be satisfied in the initial state (e.g. ¬( , )) can be specified. This helps to ensure that only well-formed goals can be instantiated. In Figure 1 on the left a list of possible templates for a logistics domain are shown. The first step in creating a goal is choosing one of those templates. In the second step the objects are selected.
The second option is to use a LLM-based goal translator, that translates a natural language description
of the goal into an LTLf formula. An example is given on the right in Figure 1. The LLM infers the
delivery location from the planning task. The translated LTLf formulas shown in this figure are intended
to be shown exclusively to expert users. An alternative approach to verify the translation involves the
use of a reverse translation of the users’ question, which is then converted back into natural language
without ambiguities. Several tools such as NL2LTL [
Iterative Planning Process During an iterative planning process the user explores the plan space by iteratively enforcing goals and inspecting the resulting sample plans. The interface is designed to provide users with the information they need to make informed decisions about which goals to enforce. To make this decision, users need an overview of the existing iteration steps and access to more detailed information for each step. At the same time, users should be able to select the goals to be enforced in the next iteration step. Our tool implements this process as follows.
The overview of the iteration steps is shown on the left side of Figure 2. This view allows to quickly identify which steps were unsolvable and solvable along with the utility they achieved. The details view of an iteration step ofers, in addition to the information provided in the overview, the following details. For a solvable iteration step (a plan satisfying the enforced goals exists), the additional satisfied reference goals, the unsatisfied reference goals and the enforced goals are listed. The sample plan satisfying the enforced goals, is accessible and displayed in a separate view. By default, the plan is displayed as a list of actions; alternatively, it is possible to implement a domain-dependent plan visualization. For an unsolvable iteration step (no plan satisfying the enforced goals exists), the enforced goals are listed.
The interface to create the next iteration step is located in a side panel, as shown on the right in Figure 2. To create a new step, a user must select at least one enforced goal. These goals can be either existing or newly created. Additionally, users can also select or create new reference goals. Reference goals represent objectives and preferences the user is interested in, but that need not be satisfied by the next sample plan. The reference goals are considered in the explanations ofering insights on how to satisfy them in subsequent iteration steps.
Goal-conflict explanations in planning were introduced in [
Based on MUS and MCS questions about unsolvable as well as solvable iteration steps can be addressed.
In an unsolvable step, the MUS of the enforced goals provide insights into why the step is unsolvable,
while the MCS ofer guidance on how to restore solvability. In a solvable step questions such as “Why is
goal not satisfied?” , “What happens if I enforce goal ?” or “How can I satisfy goal ?” can be addressed.
These questions refer to goals that are not satisfied by the sample plan, but the user is considering
enforcing them in the next iteration step and wants to identify the implications of doing so. This
approach ofers contrastive explanations [
Explanations are provided in the details view of the iteration steps. The access to the explanations and their presentation difers depending on the chosen interface. Currently, two options are supported. In the template based explanation interface the user is restricted to a predefined list of questions and the answer lists all explanations. The questions and explanations are displayed in a chat interface, as shown on the left in Figure 3. For an unsolvable step the questions refer to the entire selection of enforced goals. Therefore, the explanation interface is displayed above the list of enforced goals. In a solvable step the questions refer to individual unsatisfied reference goals. Therefore, for each unsatisfied goal there is a dedicated explanation interface accessible by clicking on the goal (see left Figure 3).
The LLM-Chat based explanation interface is displayed in the top of the iteration step details view for both solvable and unsolvable steps (see right Figure 3). Users can submit questions and receive answers via a chat interface. The interface employs a hybrid approach. The LLM-agents function as a bridge, translating the user’s natural language into the formal language of the planner, which serves as the reasoner and computes the MUS and MCS. More examples and details can be found in the next section.
As described in Section 3, IPEXCO supports an LLM-powered explanation interface introduced in [
Similar approaches involving LLM to enrich explainer system outputs have been proposed in
explainable machine learning [
Goal Goal Description
Question
Type Classifier
Response
Explanation User
Question
Question Type
DIRECT
Question Topic EFQUERY-GT
Goal Translator
Dispatcher ( ) Explanation Translator
ℰ Question FOLLOW-UP
Question EFQUERY-noGT
Question
Explanation
Framework Explanations & prior Information natural language formal language goal creation protocol explanations protocol
LLM Agent
Multi-Agent LLM Architecture Our system (Figure 4) consists of three main components: Question Type Classifier , which analyzes the user’s question to determine its type and how it should be processed (it can directly answer simple questions like “What can I ask”, identify follow-up questions that reference previous explanations, or route supported questions to the explanation service while translating the question into a formal request); Goal Translator, which converts natural language goals into LTLf formulas that the planning system understands and creates clear natural language descriptions of these goals; and Explanation Translator, which converts technical explanations from the formal system into natural language responses tailored to the user’s original question. The system maintains context across interactions, allowing for natural conversation flow with follow-up questions and references to previous explanations. For example, a user might ask “Why can’t I achieve goal X?” followed by “How can I fix that?” without needing to restate the original goal. Beyond allowing a natural language interaction, this system also allows more complex interactions with explanations including selection and summarization, which can be actively requested by users.
Example interaction We demonstrate the capabilities of the LLM-chat based explanation interface through an example interaction in an unsolvable iteration step. In the UI, these examples would be formatted in Markdown for enhanced readability.
User: What are the conflicts? LLM: Here are the current conflicts: [Bring friend to sports center, parent does sports, bring grandma’s groceries to her house, parent buys groceries, grandma and friend are not together in car], [Bring friend to sports center, parent does sports, bring groceries back home, bring grandma’s groceries to her house], [Bring grandma’s groceries to her house, do shopping before parent sports] These conflicts indicate issues with achieving your goals based on the currently enforced conditions.
User: Thanks, is there one goal that is involved in all of these? LLM: Yes, the goal "bring grandma’s groceries to her house" appears in all of the listed conflicts. This indicates that it is a common factor contributing to the conflicts preventing successful planning.
User: So removing it is enough to make the task solvable, right? LLM: Yes, removing the goal "bring grandma’s groceries to her house" would be sufficient to resolve the conflicts and make the planning task solvable. By forgoing this goal, you would eliminate all conflicts currently preventing you from achieving your objectives.
In order to evaluate the usefulness of explanations and the impact of diferent interfaces, user studies are required. To ensure a high-quality evaluation, participants should be given comparable tasks and enough freedom to develop their own strategies and use the explanations according to their preferences.
The platform provides a controlled environment through the use of planning processes with a fixed set of goals. Each goal is assigned a utility that serves as a proxy for an intrinsic preference of the participants. This approach ensures that all users are presented with the same optimization task, which is to maximize the utility of a solvable iteration step. Note that the computation of explanations can require a significant amount of time. The fixed set of goals allows for the pre-computation of explanations, ensuring a responsive experience. Certain features of the iterative planning process may be disabled. Participants who do not have access to sample plans, but only to the information and explanations provided in the iteration step details view, must rely on the explanations. To encourage a high performance, achievable utility for the task and the utility of the best iteration step are displayed.
The platform facilitates online user studies by leveraging the recruitment platform Prolific. User studies are composed of diferent parts that the user study coordinator can use to set up a user study to their needs. The options are: description steps (general information and instructions), external links (for example for questionnaires), a tool manual (general usage instructions for the tool and the selected interfaces for explanations), planning task information (description of the domain, the specific instance, and list of available goals), and planning tasks (iterative planning process with the objective to maximize the utility). Each part is associated with a minimum or maximum processing time, which restricts the progress of participants. This incentivizes users to allocate a reasonable amount of time to instructions or questionnaires and to limit the processing time of the optimization task. Given the complexity of the tool, it is essential that users receive a training to understand and fully use all its features. For this purpose, it is possible to select a task as an introductory task that contains additional instructions for the user. All actions performed by the user are tracked with timestamps, allowing for the analysis of user strategies and decisions over time. The platform also supports the monitoring of participants during the study, by providing access to metrics such as completion time and achieved utility. For a comprehensive statistical analysis, the user data can be exported as a JSON file.
The tool is a web platform, i. e. it runs directly in the browser and does not require installation on individual users’ machines. This feature is particularly advantageous for conducting user studies.
The system features a modular architecture, in which explanations and plans are provided by individual services. The back-end and the explanation and planning services implement REST APIs, and communication between the back-end and a service is asynchronous. The services themselves maintain a queue for incoming jobs and schedule them based on resource availability. This enables the integration of additional planner and explanation services, and facilitates the expansion of the platform. The supported questions are also modular, so that they can be extended depending on the explanation service selected. The presentation of explanations is not easily generalizable. However, the template-based approach can be easily extended, and the prompts of the LLM-based translators are configurable to accommodate diferent formal explanation languages.
The included planner service is based on Fast Downward (FD) [
We have introduced IPEXCO, an online platform that facilitates iterative planning with explanations. The platform ofers interfaces to goal-conflict explanation, leveraging LLMs to enhance interactivity. The support for user studies enables the evaluation of information and the communication of explanations.
Moving forward, we aim to incorporate additional explanation methods from planning [
This work was funded by the European Union’s Horizon Europe Research and Innovation program under the grant agreement TUPLES No 101070149, and was supported by the Artificial and Natural Intelligence Toulouse Institute (ANITI), funded by the French Investing for the Future PIA3 program under the Grant agreement ANR-19-PI3A-000.
During the preparation of this work, the author(s) used DeepL and ChatGPT in order to: Grammar and spelling check and improve writing style. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.
The presented platform uses ChatGPT to implement the interactive natural language explanations. Examples generated in this work are clearly labelled as such.