The objective of this proposal is to bridge the gap between Deep Learning (DL) and System Dynamics (SD) by developing an interpretable neural system dynamics framework. While DL excels at learning complex models and making accurate predictions, it lacks interpretability and causal reliability. Traditional SD approaches, on the other hand, provide transparency and causal insights but are limited in scalability and require extensive domain knowledge. To overcome these limitations, this project introduces a Neural System Dynamics pipeline, integrating Concept-Based Interpretability, Mechanistic Interpretability, and Causal Machine Learning. This framework combines the predictive power of DL with the interpretability of traditional SD models, resulting in both causal reliability and scalability. The eficacy of the proposed pipeline will be validated through real-world applications of the EU-funded AutoMoTIF project, which is focused on autonomous multimodal transportation systems. The long-term goal is to collect actionable insights that support the integration of explainability and safety in autonomous systems.
The field of System Dynamics (SD) has long focused on modeling complex systems that underpin many application domains. In transportation logistics, for example, dynamical systems are used to model supply chain operations, trafic congestion, fleet management, and urban mobility planning.
Traditional SD models rely on diferential equations and expert-defined rules to represent the evolution of a system over time. These models provide interpretable causal pathways and have long been valued for their transparency and accountability. However, they are constrained by simplifying assumptions that often fail to capture the full complexity of real-world systems and sufer from limited scalability as the number of interacting variables increases.
Contemporary Deep Learning (DL) techniques ofer a promising alternative to overcome the
aforementioned limitations of traditional SD modeling. DL algorithms are indeed capable of learning automatically
the non-linear relationships that underpin dynamical systems’ behaviors from large-scale data [
As a first step, concept-based interpretability methods will be employed to identify a set of semantically meaningful high-level variables (termed “concepts”) that describe understandable characteristics and magnitudes of interest. CML techniques will then be implemented to detect the causal dependencies among the selected high-level variables. Finally, mechanistically interpretable modeling techniques will be leveraged to infer a set of interpretable structural dynamic equations that govern the system’s behavior. Such equations will be determined by taking into account the previously identified causal dependencies. This will not only allow for increased interpretability but will also contribute to anchoring the models to the real-world causal structure, making them substantially more reliable and trustworthy for safety-critical applications.
As a final result, this pipeline should be able to return neural models that track the evolution of systems over time, both by operating on semantically meaningful and actionable variables. It will be implemented and evaluated on a real-world scenario from the EU-funded project AutoMoTIF, where SD is involved in modeling the interoperability of multi-modal transportation terminals. A more detailed description of the real-world application is provided in Sec. 1.1.
Pros Interpretability Causal Reliability Actionability
System Dynamics
Cons Limited Scalability High Dependency on Expert Knowledge
Interpretable Neural System Dynamics
Pros Intrinsic Interpretability Human-Aligned Explanations Casual Reliability
Scalability Actionability Data-driven pattern recognition
Deep Learning
Pros Scalability Data-driven pattern recognition
Cons Lack of Interpretability No Causal Reasoning Overfitting Risk
The doctoral research proposed herein will be carried out under the EU-funded AutoMoTIF project1 (Automation towards multimodal transportation and integration of freight). The project’s core focus lies in the formulation of strategies, the development of business and governance models, and the generation of regulatory recommendations. These are designed to facilitate the integration and interoperability of automated transport systems. The project’s overarching objective is to automate multimodal freight lfows and logistics supply chains within the intra-European network, thereby enhancing operational eficiency and addressing existing regulatory and technological gaps.
Within this project, System Dynamics plays a crucial role in modeling and optimizing multimodal
terminal operations, helping stakeholders to analyze and predict system behavior under diferent
operational conditions. However, the complexity of these environments calls for data-driven AI approaches,
which, despite their predictive power, often lack interpretability and causal reliability − critical aspects
for risk assessment and certification. This challenge aligns with the broader Trustworthy AI paradigm,
which underscores transparency, reliability, and human oversight [
Whilst the importance of eXplainable AI is becoming increasingly acknowledged, achieving
interpretability remains a complex process. The opacity of DL algorithms represents a major challenge for
contemporary AI research. In particular, the DL community must navigate various forms of opacity,
which vary based on the diferent aspects of a model’s structure and functioning that are focused on, as
well as on the specific users and stakeholders involved [
Semantic opacity refers to the challenge of deciphering what a model’s learned representations mean in terms understandable to humans. This issue is particularly relevant in Neural Networks (NNs), where internal representations are often abstract and distributed without explicit meanings. Humans generally process information through high-level concepts, whereas NNs operate within a multi-dimensional feature space that obscures the semantics of the features involved and how these relate to categories comprehensible by the layman user.
Mechanistic opacity refers to the dificulty of detailing precisely the mechanisms through which
the various components of a model interact with one another and thus contribute to generating the
overall model’s behavior [
In the XAI literature, these two diferent opacity forms have been mostly addressed separately by referring to diferent paradigms and implementing distinct strategies and techniques. This fragmented approach obstructs the creation of a cohesive, mathematically rigorous framework capable of addressing multiple interpretability challenges that stem from considering these two aspects of opacity together. Explaining the mechanisms underpinning the inference process of a DL model (e.g., via equation modeling) contributes minimally to the overall interpretability of the model’s behavior if the features remain low-level and semantically meaningless. Conversely, mapping low-level features to high-level concepts has limited value if the model’s decision-making mechanisms remain opaque.
Causal Reliability. Related to the aforementioned forms of opacity is another fundamental issue,
which is orthogonal to the problem of (mechanistic and semantic) opacity but intrinsically connected
to it. This is the problem we refer to as causal reliability [
The Need for an Integrated Approach. Opacity and causal reliability have been mostly treated
as separate issues in contemporary AI research. Indeed, while opacity represents the target problem
of XAI, causal reliability is at the heart of another growing research field, that of Causal Machine
Learning (CML) [
This project proposes a novel Interpretable Neural System Dynamics (INSD) pipeline that combines Concept-Based Interpretability, Causal Learning, and Mechanistic Interpretability to construct causallyreliable neural system dynamics models that operate on human-interpretable variables while preserving the flexibility and scalability of DL approaches (Figure 2). The pipeline consists of three distinct learning steps: 1. Concept Learning: In the first step, concept-based interpretability (CBI) methods are used to extract high-level semantically interpretable variables (“concepts”) from raw data. 2. Causal Learning: In the second step, causal machine learning (CML) and causal discovery (CD) techniques are leveraged to identify the causal dependencies among these high-level concepts, thereby representing them in the form of a causal directed graph. 3. Equation Learning: In the third step, mechanistic interpretability methods are involved to derive explicit and interpretable dynamic equations that allow to predict the behavior of the targetsystem over time.
After the three learning steps, the final model integrates the learned concepts, causal relationships, and governing equations to emulate the underlying dynamical system. Unlike traditional black-box neural networks, this model ofers full interpretability, enabling users to trace predictions back to meaningful variables and causal influences. Furthermore, it provides actionable insights, allowing decision-makers to simulate interventions, predict long-term efects, and better understand the system’s behavior under diferent conditions. This ensures both transparency and practical applicability, bridging the gap between deep learning’s flexibility and human-comprehensible system dynamics. The following paragraphs provide a detailed breakdown of each methodological component.
When applied to system dynamics, DL algorithms typically generate latent representations of system
states that are usually not understandable and are arduous to interpret. For instance, in the context of
epidemiological modeling, deep learning algorithms have the potential to discern underlying patterns
of disease transmission; however, they have dificulty formulating these patterns using conventional
epidemiological factors, such as contact rate or incubation period. This semantic opacity restricts their
reliability for critical decision-making processes. Concept-based Explainable AI introduces a potentially
efective solution to these limitations by aligning AI reasoning with human-understandable abstractions
rather than opaque latent representations [
Background Knowledge
Data
Semantically Interpretable Concepts Concept Learner
Structural Causal Learner
Neural Equation Learner ...
time Causal Graph
Structural Equations
Resulting Model
Concept-based models are currently underdeveloped concerning the temporal dimension, i.e. the
evolution of interpretable concepts over time. This project would enable human interventions over
evolving representations, constituting a significant advance. In addition, this class of methods faces
generalization and compositionality challenges because, similar to standard deep learning architectures,
they are essentially associative models [
terminal setting, while a DL model might forecast freight congestion patterns, it may not clarify the reasons behind delays. By employing concept-based interpretability, logistics-related concepts such as terminal workload, handling eficiency, and waiting times for various transport modes are incorporated. This ensures predictions reflect real-world operational factors accurately. Consequently, this approach makes AI-driven simulations more transparent and actionable for terminal operators.
Causal reasoning plays a crucial role in System Dynamics, as these models explicitly represent causal connections between system variables. Contrarily, DL-based approaches rely on statistical correlations rather than true causal frameworks, which limits their ability to provide strong, interpretable predictions. A promising direction for overcoming these limitations is provided by recently developed CML techniques and, in particular, the framework of Neural Causal Models (NCMs). These techniques aim to uncover the underlying causal structure of a system by learning a graph that captures causal dependencies between concepts. Based on this learned graph, we can then infer equations that describe the system’s evolution in a causally reliable manner, ensuring that the resulting models generalize more robustly and provide deeper insights into the underlying mechanisms governing the data.
Running Example: Automated Terminal Operation in AutoMoTIF. A DL model might predict regular train delays at an intermodal terminal and identify a correlation between high truck trafic and these delayed departures. However, without causal reasoning, the underlying cause can remain unclear. A causal model could determine whether truck congestion is directly causing train delays or if another external factor, such as ineficient crane operation, is primarily responsible. By simulating counterfactual scenarios, such as “What if we increased crane availability?”, causal DL enables logistics operators to make proactive and data-driven decisions.
Mechanistic Interpretability seeks to unravel DL models by reverse-engineering them to reveal their internal structures and decision-making processes. This research field is highly pertinent to System Dynamics, where comprehending a model’s inner workings holds equal importance as its predictive accuracy. Traditional System Dynamics models use clearly defined equations and feedback loops, which make them inherently easy to interpret. The formal use of a causal loop diagram to describe a feedback system naturally leads to a connection with graph-based AI architectures like Graph Neural Networks (GNNs). GNNs ofer an intuitive and organized way to represent dynamical systems, bringing benefits regarding intrinsic interpretability by exploiting relational inductive bias. Specifically, mechanistic interpretability techniques can help to trace information propagation to understand internal interactions. This is essential to develop structured, human-interpretable representations of dynamic processes.
Running Example: Automated Terminal Operation in AutoMoTIF. Consider a simulation of an intermodal terminal where a DL-based model recommends rerouting trucks via a secondary access road. Without mechanistic interpretability, the reasoning behind this decision remains unclear − whether it stems from predicted congestion, infrastructure constraints, or other operational factors. Taking advantage of the intrinsically interpretable model, we can identify modular components within the model, track the flow of information within the model, and uncover the key interactions influencing routing choices. This structured understanding enhances both the interpretability and trustworthiness of AI-driven logistics systems.
RQ1: How can a system dynamics framework ensure transparency and accountability while maintaining the predictive power of deep learning? Working Hypothesis: Integrating traditional equation-based modeling with deep learning can achieve this balance.
RQ2: How can traditional modeling techniques and deep learning be efectively combined to leverage their strengths? Working Hypothesis: An optimal integration may involve combining three key areas of XAI research: concept-based interpretability, mechanistic interpretability, and causal machine learning. RQ3: How can methods from the aforementioned distinct fields be integrated to develop an interpretable neural system dynamics framework? Working Hypothesis: Concept-based interpretability can be used to learn high-level concepts and map raw data to meaningful variables, causal learning to infer dependencies among these variables, and mechanistic interpretability to define and parameterize system equations.
To refine this framework, the investigation focuses on: (i) how concept-based techniques can identify high-level variables from raw data; (ii) how causal learning can infer dependencies using data and background knowledge; and (iii) how to determine the structure and parameters of governing equations in an interpretable manner. Additionally, the framework’s adaptability for modeling intermodal transportation logistics is examined, with an emphasis on ensuring safety and accountability. The integration of traditional equation-based modeling with deep learning, leveraging methods from these XAI fields, is hypothesized to achieve an interpretable and reliable system dynamics framework. This approach ensures partial verifiability, actionability, and control in real-world applications. Specific Objectives and Milestones. The project’s main objective is to design, implement, and evaluate an integrated pipeline combining system dynamics modeling with deep learning, applying it to AutoMoTIF and assessing generalisability. To achieve this objective, the following intermediate milestones are planned: M1: State-of-the-Art Analysis. Review concept-based, mechanistic, and causal learning interpretability via a systematic literature review [D1: survey paper].
M2: Pipeline Blueprint. Design a blueprint for integrating dynamical system modeling with deep learning through literature review and theoretical modeling [D2: short conference paper]. M3: Use-Case Implementation. Develop and validate the pipeline on a toy example [D3: conference paper (e.g., NeurIPS, ICML)].
M4: Real-World Scalability Study. Scale the pipeline to AutoMoTIF [D4: applied research paper]. M5: Generalization Study. Assess applicability to other real-world scenarios [D5: generalization study report or journal paper].
M6: Dissertation Completion. Compile research findings into the PhD thesis [ D6: dissertation].
The project will run for 48 months. It started in January 2025, and termination is planned for December 2028. The structure of the project’s timeline is reported in Fig. 3.
Research Phase Design Phase Development Phase Deployment Phase Evaluation Phase Completion Phase
State-of-the-Art Analysis
Survey Paper Pipeline Blueprint
Short Conference Paper Use-Case Implementation
Conference Paper Real-World Scalability Study
Applied Research Paper
Generalization Study Generalization Study Report
Dissertation Completion 25-01 25-04 25-07 25-10 26-01 26-04 26-07 26-10 27-01 27-04 27-07 27-10 28-01 28-04 PhD Dissertation 28-07 28-10
Expected Contribution and Impact. Through the development of a unified interpretability framework for DL-based System Dynamics models, this research aspires to bridge the current divide between theoretical advancements in eXplainable AI and their application in high-stakes, real-world environments. By bringing together causal, mechanistic, and concept-based perspectives within a cohesive methodology, the project is expected to deliver not only novel algorithms and formal models but also practical tools that empower users to understand, trust, and efectively intervene in AI-driven processes. The anticipated contributions extend beyond the specific context of multimodal logistics, ofering generalizable insights for any domain that relies on the interplay of Deep Learning and System Dynamics. These insights may be applied to a wide range of fields, including, but not limited to, transportation, healthcare, environmental monitoring, and finance. Ultimately, this work aims to establish both a conceptual and operational foundation for interpretable, trustworthy AI in settings where transparency and accountability are not optional but essential for safety, compliance, and societal acceptance.
This work was partly supported by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract no. 24.00184 (AutoMoTIF project). The project has been selected within the EU Horizon Europe programme under grant agreement no. 101147693. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the funding agencies, which cannot be held responsible for them.