Continual learning-the ability to acquire, retain, and refine knowledge over time-has always been fundamental to intelligence, both human and artificial. Historically, diferent AI paradigms have acknowledged this need, albeit with varying priorities: early expert and production systems focused on incremental knowledge consolidation, while reinforcement learning emphasised dynamic adaptation. With the rise of deep learning, deep continual learning has primarily focused on learning robust and reusable representations over time to solve sequences of increasingly complex tasks. However, the emergence of Large Language Models (LLMs) and foundation models has raised the question: Do we still need continual learning when centralised, monolithic models can tackle diverse tasks with access to internet-scale knowledge? We argue that continual learning remains essential for three key reasons: (i) continual pre-training is still currently necessary to ensure foundation models remain up to date, mitigating knowledge staleness and distribution shifts while eficiently integrating new information; (ii) continual fine-tuning enables models to specialise and personalise eficiently, adapting to domain-specific tasks, user preferences, and real-world constraints without full retraining, avoiding the need for computationally expensive long context-windows; (iii) continual compositionality ofers a scalable and modular approach to intelligence, enabling the orchestration of foundation models and agents to be dynamically composed, recombined, and adapted in an open-ended way. While continual pre-training and fine-tuning are currently explored as niche research directions, we argue it is continual compositionality that will mark the re-birth of continual learning. The future of AI will not be defined by a single static model but by an ecosystem of continually evolving and interacting models, making continual learning more relevant than ever.
In recent years, artificial intelligence (AI) systems have begun to surpass human performance in many
domains such as natural language processing (NLP) and computer vision. However, these models
are typically static in nature and do not naturally update their understanding as new data emerges
over time. In contrast, humans tend to approach problems as sequential learning tasks, building on
past information without forgetting previously learned knowledge or requiring rehearsal to retain it
[
Continual Learning research therefore revolves around two primary goals: adaptation and memory consolidation. Adaptation emphasises rapid responsiveness, enabling an agent to quickly adjust
CEUR
ceur-ws.org
its behaviour or representations to maximise a utility function given the current task, situation or
environment [
Historically, these two goals have been addressed with varying degrees of importance in diferent
research methodologies, contexts and communities. Early expert systems, for instance, focused on
consolidating incremental domain knowledge, yet lacked flexibility in rapidly adapting to new information
without significant manual efort. Reinforcement learning (RL) methods, such as CHILD, introduced
by Ring [
More recently, the emergence of LLMs has changed the focus of AI research towards
transformerbased models with less focus on ’traditional’ Machine Learning (ML). These models, pre-trained on
vast datasets, have demonstrated remarkable capabilities to learn rich generalisations of the world
[
Applying continual learning, with its two main aims of adaptability and memory consolidation, to Foundation Models (FMs) is a way to overcome some of their inherent shortcomings: since a FM’s parameters are fixed at deployment, every model can be seen as a snapshot of the world at the point of training. Practical use of FMs demands post-training adaptation, through fine-tuning or personalisation for a downstream task. Therefore, their static nature poses a significant challenge — they lack the intrinsic adaptability required to stay current in rapidly changing environments.
So far, FM-based agents that actively interact with their environment have been proposed as a
promising solution, leveraging continual adaptation to progressively improve capabilities [
Given this context, CL for foundational models is developing along three directions: Continual
Pre-Training (CPT), Continual Fine-Tuning (CFT) and Continual Compositionality &
Orchestration (CCO). Human perception provides an instructive analogy: during infancy, critical periods
enable rapid specialisation and consolidation of broad sensory capabilities such as vision and language
discrimination [
We therefore contend that Continual Compositionality and Orchestration represents the most
promising and necessary direction for future continual learning research. Unlike CPT and CFT, CCO
inherently supports high-frequency adaptation, allowing dynamic orchestration, recombination and
collaborative interaction among multiple FMs or agents. Recent advances in FMs have primarily emerged
not from additional computational resources, but rather through enhanced reasoning abilities [
In this paper, we first review Continual Pre-Training ( 4.1), which equips large-scale foundational models with adaptive, resource-eficient mechanisms to incorporate new knowledge without catastrophic forgetting. We then examine Continual Fine-Tuning (4.2), enabling precise specialisation to downstream tasks while retaining broad, generalisable representations. Finally, we argue that Continual Compositionality & Orchestration (4.3) - with its high-frequency, modular coordination of specialised agents - ofers the most promising path forward. By moving from monolithic snapshots to dynamic, decentralised ecosystems of models, CCO can drive the next wave of resilient, scalable, and sustainable AI systems.
The rise of Deep Learning (DL) in 2012 marked a pivotal moment for the entire ML research community. Initial eforts in this domain concentrated on utilising deep neural networks for representation learning, allowing models to capture abstract and hierarchical features from data. Nevertheless, when trained sequentially on multiple tasks, these models remained susceptible to catastrophic forgetting. This highlights the need for CL, which has had a large impact on the broader field of ML.
A widely used definition, often taken to specify Deep Continual Learning, is ofered by Lesort et al.
[
This is a key diference between CL and traditional machine learning approaches, which typically require retraining on a static dataset to incorporate new information.
Early studies focused on the problem of catastrophic forgetting in neural networks [19, 20], where authors discovered the degradation of model performance on previous tasks while learning a new one. To overcome the issue, diferent basic approaches were proposed [ 21, 22, 23, 24]. These early works laid the foundations for more sophisticated approaches that emerged in the following years. The main approaches within CL can be categorised into three main schools of thought: regularisation-based methods, dynamic architectures, and memory-based techniques [25, 26].
Regularisation-based methods aim to mitigate catastrophic forgetting by adding constraints to the learning process, ensuring that important weights from previous tasks or domains are preserved. EWC [24] adds a regularisation term to the loss function to preserve important weights from previous tasks; SI [27] uses a similar approach but focuses on the Fisher information matrix to identify important weights; LWF [23] introduces a distillation loss to retain knowledge from previous tasks. Dynamic architectures involve modifying the model’s structure to accommodate new tasks while retaining previously learned knowledge. PNNs [28] add new subnetworks for each task while keeping the previously learned parameters frozen; LWTA [21] divides the networks into diferent blocks and the forward propagation is done only by local winner weights; PackNet [29] prunes the networks and uses masks to filter weights for diferent tasks; Piggyback [ 30] drops the weights training [31] and focuses only on weights masking; a similar approach is used in HAT [32], but masking the units instead of the weights; SupSup [33] simplifies this even more using only a seed to generate random weights and a weighted sum of masks.
Memory-based methods utilise stored examples from past tasks to reinforce prior knowledge during training. GDumb [34] randomly selects a subset of exemplars from previous tasks and uses them to train the model; GEM [35] uses episodic memory to store exemplars from previous tasks and ensure that the model does not forget them during training; iCaRL [36] utilises stored examples from past tasks to replay and reinforce prior knowledge during training; potentially, one could also use latent generative replay to generate new samples from the previous tasks [37, 38].
For a clearer academic evaluation of diferent CL methods, three main scenarios have been defined and are widely used within the community [39]. In Task-incremental learning, the aim is to incrementally learn a set of distinct tasks based on a given task-id. Whereas in Domain-incremental learning, the context or input distribution varies over time, whilst the task remains constant (e.g. learning to drive in diferent weather conditions). Finally, in Class-incremental learning, the aim is to incrementally learn to discriminate between a growing number of objects or classes, where task identification is also required. This last scenario is naturally the most dificult to solve, as well as the most applicable within real-world scenarios.
Diferently from CL models, Foundation Models (FM) are characterised by their large scale, pretraining on huge amounts of data, and ability to perform a wide range of tasks with a relatively small amount of fine-tuning. These models, such as BERT [ 40], GPT-3 [41], and CLIP [42], have demonstrated remarkable performance across various benchmarks and applications. However, one of the biggest problems faced by FM models is homogenisation [43], where a single unified model trained on diverse data results in generalised internal knowledge representations. While this approach enables model transfer across tasks, it also averages out critical domain-specific nuances, leading to inherent biases from the most dominant data sources.
More generally speaking, a monolithic AI system, which is a single large model that is trained on a wide range of tasks and domains [44], is typically pre-trained on massive datasets and then fine-tuned for specific tasks. Such models often require extensive computational resources for training and inference, making them less accessible for smaller organisations and researchers. Additionally, a monolithic model can sufer from knowledge staleness, where the model’s performance degrades over time as new data becomes available. Furthermore, their centralised nature poses risks related to single points of failure making them less robust in critical applications.
On the other hand, continual learning ofers a more sustainable and adaptive alternative by enabling models to learn incrementally, adapt to new tasks, and operate eficiently in dynamic settings [ 25]. This paradigm shift is essential for addressing the limitations of monolithic AI and fostering more equitable and resilient AI systems. Continual learning methods could benefit FMs in particular by enabling them to adapt to new tasks and domains without requiring extensive retraining [45], thereby improving their performance over time. By incorporating continual learning strategies, FMs can mitigate issues such as knowledge staleness and ineficiency in adaptation. Furthermore, these methods can help reduce the social and environmental impact of FMs by minimising the need for large-scale retraining, which often requires significant computational resources and energy consumption.
Recently, it has been shown that foundational models such as LLMs have the ability to both “reason” and generalise through the use of techniques such as chain of thought (CoT) prompting, by breaking a complex problem into a series of intermediate steps [46]. This trend of utilising the zero-shot capabilities of LLMs, perhaps with the addition of prompt engineering and later downstream-task fine-tuning, is, however, subject to many diferent limitations such as the brittleness and inconsistency of the generalised reasoning steps.
Foundation models such as GPT-3, BERT and DALL-E can quickly become outdated as the real-world
data they are trained on changes [41, 43]. This leads to model staleness over time, where often full
model retraining is the only solution used to mitigate this. Recent estimates indicate that the cost
of training models of a scale comparable to GPT-4.5 or similar architectures likely reached tens of
millions of dollars due to the enormous compute and energy resources required [47]. Beyond the
economic impact, the energy required for such massive compute workloads translates into significant
environmental impacts. Studies by Schwartz et al. [
State of the art large scale models are typically trained on vast, diverse datasets to capture a wide range
of linguistic patterns and knowledge. Whilst this training approach enables impressive generalisation
across tasks, it results in a system that is too general to properly address the requirements of individual
users or specialised domains [
Further, conventional personalisation methods, relying on post-training adjustments or Test Time
Training (TTT) face significant challenges. In TTT or test time adaptation (TTA), a model adjusts its
parameters during inference based on the current input or an auxiliary task, aiming to better align
with the data distribution at test time [
The rising computational and financial demands for training state of the art models have led to a
concentration of resources within a small number of organisations. This centralisation means that only
a few giants of industry possess the capability to develop, maintain, and update foundation models
at scale [
While the challenges of centralisation, high retraining costs, and limited personalisation have long
constrained the evolution of large-scale AI systems, these can each be addressed in diferent ways
by continual learning. There are two fundamentally diferent types of forgetting in the context of
foundational models that necessitate continual learning as a solution: task-shift and time-shift forgetting
[
Here, we outline the three main research directions that are crucial for the future of Continual Learning, namely Continual Pre-Training, Continual Fine-Tuning and Continual Compositionality & Orchestration, visualised in Figure 1. In the following sections, we will outline the requirements for each of these separate components, challenges they face, open problems that are yet to be solved and how these methods address the challenges within this context.
CPT refers to the process of incrementally updating the knowledge of FMs through exposure to new
data after their initial pre-training phase [
We now look to five main motivating factors to advance the field of CPT.
Dynamic Knowledge Integration: CPT is primarily driven by the necessity to keep FMs relevant
and efective over time. Even large-scale foundation models can quickly become outdated as data
distributions shift and new knowledge emerges [
Methodological Evolution: CPT is also motivated by the ongoing improvements in model
architectures. Even when there is no new data or major distribution shift, updating foundation models can still
be valuable for adopting architectural advances that improve eficiency or performance. For instance,
transitioning from an encoder-decoder to a decoder-only architecture benefits from reusing existing
pre-trained models, which avoids the need to retrain from scratch and significantly reduces
computational cost. CPT enables this kind of update by initialising new models from previous checkpoints,
allowing them to retain useful learned knowledge while gradually adapting to architectural changes
[
Resource Eficiency: Retraining FMs entirely from scratch on increasingly vast datasets becomes
computationally prohibitive over time. CPT significantly reduces these computational burdens by
incrementally updating models with new or domain-specific data, circumventing the need for repeated,
costly training cycles [
Mitigation of Catastrophic Forgetting: Static models, once trained, are frozen at their initial
knowledge cut-of, which creates a gap between initial training and real-world deployment needs. In
contrast, CPT fosters a continual evolution of models, bridging the gap between initial pre-training
and practical, lifelong learning scenarios. Notably, recent studies find that CPT can make models more
robust to catastrophic forgetting of earlier knowledge, especially when using self-supervised objectives,
highlighting CPT as a key enabler for foundation models to learn continuously like humans [
Selective Forgetting: In addition to mitigating catastrophic forgetting, CPT has the potential to
support selective forgetting, where specific information embedded in a foundation model is intentionally
removed over time [
CPT is still in the early stages of development, and bridging the gap between research and production
remains challenging: while CPT techniques show promise in controlled experiments, their long-term
stability and efectiveness over months of deployment in real-world settings remain under-explored [
Handling catastrophic forgetting: Catastrophic forgetting, the phenomenon where a continually
updated model loses previously acquired knowledge, remains a critical challenge in CPT [
Balancing Eficiency vs. Model Drift: FMs often have hundreds of billions of parameters, so
retraining them on every new dataset is extremely computationally expensive [
Avoiding reinforcement of biases in pre-training: Continuously ingesting new data can also
reinforce biases or fairness issues if not carefully controlled. If the incoming data is skewed or uncurated,
the model might amplify these biases over time, undermining responsible use. For example, biomedical
FMs trained on federated data must address fairness across diverse populations while preserving privacy
[
To address the challenges of CPT and unlock its full potential, several promising solutions and future research directions have been proposed, including:
Incremental pre-training strategies: Recent works propose reusing or initialising from previous
model weights to maintain continuity, as in recyclable tuning methods that carry over knowledge from
an old model to a new one [45]. Recent work has demonstrated the benefits of structured or multi-stage
CPT. For instance, in [
Selective memory and rehearsal methods: To combat forgetting, memory replay methods have
shown promise in the context of foundation models. Rather than relying only on the latest data, the
model can intermittently rehearse on representative samples of past data (or tasks). In practice, storing
raw past data for a foundation model may be impractical or raise privacy concerns such that recent
work leverages latent replay, where the model saves a cache of compact feature representations or
embeddings of past examples instead of the raw inputs [
Self-supervised continual adaptation techniques: CPT largely relies on self-supervised learning
objectives such as predicting masked tokens, next sentence prediction, and image-text contrastive
learning, because these allow the use of unlabelled streaming data. An intriguing finding from recent
research is that self-supervised objectives can themselves be leveraged to improve CL. In [
Continual fine-tuning is the practice of applying a stream of lightweight, task‑specific updates to a model after deployment, allowing it to evolve alongside newly arriving data rather than remaining ifxed after a single adaptation.
Fine-tuning is attractive because it it far cheaper than full retraining, requires only task-specific data,
and can be executed on modest hardware [
Although FMs have demonstrated impressive versatility across diferent tasks, with the ability to generalise efectively to various domains, their static nature limits the capacity to incorporate new knowledge, adapt to specialised fields, and personalise outputs over time. CFT presents an important opportunity to make foundation models more flexible, eficient and responsive to real-world changes, making them more useful in more dynamic environments. While continual pre-training focuses on updating a model’s general representations using broad, often unlabelled data, continual fine-tuning instead aims to incrementally adapt the model to specific downstream tasks using labelled or structured data, with an emphasis on retaining prior knowledge while learning new information.
CFT in the context of foundation models like LLMs comes with several challenges such as:
Balancing Specificity vs. Generalisation : CFT must maintain a delicate equilibrium between
tailoring a model to a specific downstream task and preserving its broad, generalisable knowledge. When
a model undergoes CFT, its internal representation becomes optimised to capture patterns necessary for
solving a specific downstream task. While this adaptation enhances the performance on domain specific
tasks, it risks eroding the broad, general-purpose representations learned during pre-training. This
comes back to the stability-plasticity dilemma where models must remain plastic enough to integrate
task-specific knowledge while being stable enough to retain the broad representations acquired from
prior experiences [
Eficient Adaptation Without Catastrophic Forgetting : In a similar vein to CPT, catastrophic
forgetting is also experienced by CFT methods, however within this context eficient adaptation also
needs to be considered. This concept refers to the process of updating a pre-trained foundation model
to perform well on new tasks, domains, or data distributions while minimising computational resources,
data requirements, and training time. In the context of LLMs and other foundation models, eficiency
has become increasingly critical as these models grow to billions of parameters. Eficient finetuning
techniques like LoRA [
Data eficiency and privacy concerns in continual fine-tuning : CFT deals with the dual challenges of data scarcity and privacy concerns, especially in specialised domains to efectively adapt models to new tasks. As foundation models are adapted to increasingly specialised domains, high-quality, domain-specific data becomes progressively scarcer [ 86].
Despite these limitations, numerous methods have been developed to tackle these challenges.
Parameter Eficient Fine-Tuning A key methodological tool in the context of CFT is represented
by Parameter Eficient Fine-Tuning (PEFT) methods. These techniques aim to achieve performance
comparable to or even surpass full model fine-tuning while updating only a small number of trainable
parameters, either by selectively updating a subset of the model’s parameters [
By updating only a limited number of parameters, PEFT approaches reduce computational overhead and, when applied properly, can help mitigate the risk of catastrophic forgetting. Among these PEFTs, Prompt Adapters and LoRA are the most widely used. LoRA works by introducing low-rank updates to the pre-trained model weights, expressed as: = 0 + (1) where the pre-trained model 0 is kept frozen, while the low-rank matrices and are updated.
Prompt-based techniques like L2P and CoDA Prompt [87, 88] incrementally learn from novel data by designing task-specific prompts that guide the model’s attention toward relevant information for each new task, facilitating seamless integration of new knowledge without overwriting existing capabilities.
Similarly, LoRA-based CL approaches, such as C-LoRA and DualLoRA [
Adapter techniques like Continuous Adapter (C-ADA) and Adapter-based Continual Learning (ACL) [90, 91] instead ofer more eficient solutions for CL. C-ADA introduces a Continual Adapter Layer that extends weights for new tasks while freezing old ones, preserving prior knowledge. It also employs a Scaling & Shifting module to align feature spaces between pre-training and downstream datasets. Similarly, ACL utilises lightweight, task-specific adapters within a fixed pretrained feature extractor and incorporates a task-specific head that groups previously learned classes into an ”out-of-distribution” category, facilitating efective feature discrimination.
Model Merging: A particularly valuable approach when facing dynamic environments is given by Model Merging. The key point in this class of methods is to combine multiple specialised models learned over time, to create systems that preserve knowledge while adapting to new tasks. The central challenge in model merging is parameter interference, where integrating diferent models leads to performance degradation. Recent research has developed several innovative solutions to this problem. TIES-MERGING [92] addresses interference by strategically resetting minimally changed parameters and resolving sign conflicts between models. In contrast, DARE [ 93] employs a diferent strategy by randomly dropping redundant delta parameters and rescaling the remaining ones, efectively sparsifying merged models without significant performance loss.
While early model merging techniques focused on the static combination of pre-existing expert models, more recent approaches support dynamic integration as new tasks emerge over time. MagMax [94] introduces sequential fine-tuning with maximum magnitude weight selection to efectively incorporate new information while preserving earlier learning. Representation Surgery [95] tackles representation bias by inserting lightweight task-specific modules that realign internal representations between merged models. Adaptive LoRA Merging [96] moves beyond fixed-weight combinations by dynamically computing merging coeficients that balance contributions from new and old domains.
Recent trends in CFT have shifted towards the adaptive integration of lightweight modules, like
adapters [
Meta Learning for Continual Adaptation: An alternative perspective on the challenge of CFT
is ofered by meta learning approaches for continual adaptation. In fact, these methods integrate
adaptability into the core learning objective, enabling models to rapidly adjust to new tasks with
minimal data and computation. Traditional meta-learning approaches like Model-Agnostic
MetaLearning (MAML) [97] operate by finding parameter initialisations that enable rapid adaptation across
a distribution of tasks. When applied to CL scenarios, these methods can be extended to discover
parameter configurations that not only adapt quickly but also resist catastrophic forgetting. For instance,
ANML [
Recent works have combined meta-learning with parameter-eficient fine-tuning techniques to enhance CL. AutoLoRA [98] introduces a meta-learning framework that automatically identifies the optimal rank for each LoRA layer, improving adaptation eficiency to new tasks while maintaining performance on previous ones. Similarly, Meta-LoRA [99] presents a memory-eficient method for automatic sample re-weighting during fine-tuning, facilitating eficient continual adaptation across various domains. These approaches exemplify the potential of meta-learning to enhance the adaptability and eficiency of foundation models in dynamic environments.
Federated Learning (FL) and Decentralised Fine-Tuning Strategies: Moving towards the
direction of a distributed and decentralised AI development, FL is essential for efectively adapting
large FMs across organisations, while preserving data privay and optimising computational resources.
When applied to FMs, however, FL faces unique challenges. The vast number of parameters in modern
FMs makes server-client communication prohibitively expensive. Diferent clients naturally generate
data with varying distributions, creating potential conflicts in optimisation objectives. Frameworks like
FATE-LLM [
Continual Compositionality & Orchestration refers to the dynamic integration of multiple AI agents over time, to solve higher-level tasks. It is the key component towards a distributed and decentralised AI framework.
Large models solved most of the tasks addressed by AI methods for decades. In fact, using a Transformerbased architecture, pre-trained on some large dataset, we now know how to solve almost every task, provided enough data and computational power. Broadly speaking, modern FMs have achieved superhuman performance on most of the traditional machine learning benchmarks, making them obsolete and less relevant for current AI research.
For this reason, we are now shifting towards higher level tasks, which require a higher level of intelligence, that current state-of-the-art models do not exhibit. This trend is demonstrated by the emergence of several more complex benchmarks, that could drive the research beyond current LLMs. One such benchmark is ARC-AGI [101], which consists of simple grid transformations. Despite being simple for a human solver, it poses great challenges, requiring strong abstraction skills and inductive reasoning. Another example is BIG-bench [102], a collection of more than 200 tasks, designed to test the limits of current large models. The benchmark covers a broad set of tasks, including linguistics, mathematics, common-sense reasoning, social bias detection and more. In both cases, human performance significantly surpasses the current best models, highlighting the need for alternative solutions.
Furthermore, latest large models, e.g. GPT 4.5, demonstrate that we are rapidly moving towards a diminishing returns regime, meaning that just increasing the model dimensions and the number of GPUs employed is no longer enough to obtain noticeable improvements [103]. We believe that a paradigmatic change is needed in the AI community in order to push research forward and to obtain more intelligent behaviours.
Our proposal is to address some of the shortcomings that current solutions exhibit, particularly regarding scalability and sustainability matters, with Continual Compositionality and Orchestration approaches. Nowadays, the de facto standard is to have single, monolithic models, trained once and deployed without any guarantees on their utility over time. CCO instead represents a framework built on the communication between multiple AI models, which share their knowledge and skills in terms of model parameters, deep representations or final predictions.
Within this paradigm, the orchestration among the agents becomes crucial: depending on the task, diferent modules are selected to be composed in various fashions. In other words, rather than adjusting a single network to a dynamic scenario, CCO employs a modular approach, where diferent module compositions can be used to adapt to non-stationary environments over time. Also, such a framework does not impose constraints on the scalability of the system, i.e. the number of modules involved, and it is more sustainable by design, since the modules are trained once and then re-used over time in multiple ways.
With the advancements in capabilities of LLMs, there has been a growing research focus on building LLM-based agent architectures, in which multiple models are composed and coordinated to solve complex tasks [104, 105]. We can place agents on a spectrum based on their level of autonomy in order to diferentiate their architectures. On one end, fully autonomous systems, in which agents interact with significant freedom; on the other, predictable and structured workflows, in which agents follow predefined steps and communication patterns [ 106]. More structured and predetermined workflows might be preferable for domains that require more precision and accountability, such as mathematical reasoning, scientific research, law, medicine, and software development.
Task Decomposition and Specialisation: An efective strategy to enable LLMs to solve complex
problems is to break them down into simpler, more manageable sub-tasks [
Within this context, Mixture-of-Experts (MoE) models have emerged as a prominent line of research, ofering a natural implementation of the divide and conquer paradigm. MoE architectures aim to scale model capacity eficiently by activating only a subset of experts per input, leading to improved performance [113, 114]. However, a core challenge lies in achieving efective expert specialisation — ensuring that each expert acquires unique and distinct skills, with little overlap with the others. Preliminary works suggests that expert models specialise on superficial patterns, such as token IDs, rather than extracting high-level semantic information [115]. Although recent eforts have proposed solutions towards more meaningful expert specialisation [116], we still lack a clear understanding of these mechanisms, especially when considering distributed and decentralised AI frameworks. Further research is needed to ensure that MoE models can robustly and adaptively decompose complex tasks in dynamic, multi-agent environments.
Role-based Collaboration and Interactions: In a compositional framework, interactions among agents represents a key ingredient. One of the most common strategies to compose multiple agents to work together in solving a task is the so-called role-based collaboration. LLM-based agents assume clearly defined, specialised roles (such as domain experts, assistants, etc.) in order to solve a higher level goal, with each of them being assigned individual sub-tasks by other agents. Optimal role assignments and agent adaptability to dynamic tasks requirements are still areas of open research [104]. One challenge is that, while LLMs are able to simulate many common roles, there are still many roles that they might not be able to capture accurately, such as uncommon roles rarely seen in the training corpus, or roles corresponding to human characters with particular cognitive-psychological traits [117, 118, 105].
Additionally, agent interaction play an essential role for multi-round tasks, where multiple iterative feedbacks loops — both from the environment and from other agents — are required to achieve the objective. Such tasks demand dynamic coordination, contextual adaptation, and the ability to reason over partial progress. Recent studies have begun to explore this promising topic, introducing novel solutions for LLM collaboration [119, 120]. This underscores the relevance of multi-agent interactions as a key challenge for CCO.
Propagation of errors: Lastly, the propagation of errors is an additional open problem in ensuring robustness in model integration. Erroneous outputs, hallucinations and biases from one agent can have cascading efects, getting amplified and spread through model interactions and impacting the whole system [104].
The concept of multiple models collaborating within a shared environment is well established in the AI field, and is rooted in the foundational definition of multi-agent systems (MAS). Such systems consist of multiple autonomous agents, each with its own goals and motivations, that are capable of interacting with one another. For this reason, a central focus of this paradigm is on cooperation and coordination among agents.
The CCO framework can be viewed as a concrete instantiation of MAS, where models such as LLMs must be orchestrated to achieve a common objective. A key research direction in this context is the dynamic selection and composition of the most appropriate agents for a given task. Indeed, in the CCO framework the goal is to enable automatic models composition, that can evolve dynamically over time in response to variations in the environment or in the task given by the user. This marks a significant advancement over existing compositionality frameworks, e.g. LangGraph1, AutoGen [121], in which the orchestration is largely static and predefined by the programmer, thereby limiting both flexibility and the capacity to generalise across tasks and environments.
Another component enabling continual learning capabilities in the CCO framework could be the use
of experience accumulation modules, in the form of memory modules and skill libraries, as proposed in
works like GITM, Voyager, AppAgent and MemPrompt [122, 123, 124, 125]. These approaches allow
models to dynamically acquire new knowledge and skills, as a result of interactions with other agents,
humans and the environment [105, 108, 126]. Such knowledge and skills can be stored in natural
language or code form, and later retrieved and added to the model input context as needed. These
mechanisms rely on test-time inference and incorporate the new knowledge and skills in the input
context window of the model. One limitation might be that, depending on the LLM used, the context
window capacity might limit the amount of task information that can be incorporated; however, research
advancements in this area are enabling ultra-long context windows of 1M tokens or more [
An additional crucial aspect lies in the communication between agents. In the case of LLM agents, communication can occur through natural language, which has the added benefit of being easily interpretable by human users — a property that enhances transparency and human-in-the-loop control [127]. However, the CCO framework is designed to be model-agnostic and general-purpose, extending beyond language models to integrate a diverse set of AI components — such as computer vision models, time series processors, rule-based systems, symbolic modules or even hard-coded functions. To support such heterogeneity, the system requires a robust communication protocol, that accommodates decentralisation, asynchrony and diferent data formats, while enabling eficient knowledge exchange.
Furthermore, an important open question in the design of the CCO protocol is in what type of knowledge should be shared among agents, to maximise collaboration without unnecessary overheads. Depending on the use case, this could include (i) model parameters, either in entirety or specific subnetworks / modules; (ii) the model internal representation, e.g. latent vectors or output logits; (iii) training data, as raw samples or abstracted via data generators.
Over the past decade, Continual Learning (CL) has established itself as a prominent research area
exploring a variety of domains, problem settings and applications [25]. In Computer Vision, significant
efort has been committed to class incremental and domain incremental scenarios, where models must
progressively recognise new categories without forgetting previously learned ones, even as the input
domains evolve [128]. In Reinforcement Learning, CL has focused on the ability of agents to adapt
dynamically to evolving environments while retaining past knowledge and abilities, particularly in task
incremental and multi-task settings [129]. More recently, CL research has extended its scope to include
LLMs and Foundational Models, where the challenge lies in enabling models to continuously acquire
new linguistic capabilities or domain knowledge without catastrophic forgetting [
Given the increasingly consolidated position of CL within the research community, we believe the time is right for the field to take a decisive step forward. Rather than representing a separate area of study, CL should become a critical and fundamental component in the prototyping and evolution of modern AI systems. In particular, the rapid rise of foundational models has captured the attention of both academia and industry, asserting itself as one of the most prominent and transformative trends in contemporary AI research. These models showcase broad generalisation capabilities across tasks and modalities; however, they still exhibit a critical limitation: their knowledge is inherently static and ifxed at training time. This immutability poses a major challenge in dynamic real-world environments, where new data and information continuously emerge.
To address this gap, foundational models must evolve towards true continuous adaptation, progressively updating, refining, and extending their knowledge over time. Continual learning principles can provide a concrete and resourceful solution in this context. As previously discussed, Continual Pretraining and Continual Fine-tuning represent emerging research directions that aim to integrate the principles of CL with large-scale models, enabling them to remain updated, accurate, and contextually relevant, thereby mitigating outdated or incorrect outputs [45]. However, CPT and CFT inherently require relatively low-frequency adaptation cycles and depend on substantial computational resources. Thus, while beneficial, these approaches alone might not address the dynamic adaptability required by real-world applications fully.
Real-world change, in contrast, is often expressed at the level of orchestration: new tools appear [
Foundational models are generally efective across broad tasks; however, when maximising
performance in specialised areas, such as mathematics or physics, monolithic models may underperform
compared to models specifically distilled or trained for those individual tasks [ 130, 131]. We believe
that creating and deploying CL models which can continually evolve and combine knowledge from
diferent sources represents a promising and sustainable architectural solution for AI systems. Moreover,
developing models that are a mixture of many components [
Furthermore, aligning with recent advances, foundational models can also be improved through
human feedback within a Reinforcement Learning from Human Feedback (RLHF) framework, where
humans actively guide model evolution, shaping future AI outputs and capabilities in turn [
In summary, Continual Learning, specifically continual compositionality and orchestration, represents not only a promising research direction, but the cornerstone of AI’s next paradigm shift. By transitioning from incremental improvements within individual models to dynamically composable and collaborative AI ecosystems, CL can drive a new generation of adaptive, scalable and human-aligned AI systems.
The remarkable capabilities of large foundational models highlight their potential in solving complex, diverse tasks across multiple domains. However, despite their robust generalisation abilities, these models are inherently static and struggle to adapt continually to evolving real-world data and tasks. To overcome these limitations, continual learning emerges as an indispensable tool, ofering a multitude of methodologies to enhance adaptability, eficiency and sustainability within foundational models.
In this paper, we have highlighted three pivotal areas of continual learning critical to the evolution of FMs: continual pre-training, continual fine-tuning, and continual compositionality and orchestration. CPT equips very large, organisation-scale models with the mechanisms to incrementally incorporate new knowledge, capabilities or methodologies, maintaining their relevance and mitigating catastrophic forgetting through adaptive, resource-eficient updates. As such, it is mainly within the remit of industrial laboratories and cloud providers who possess the necessary data and compute. CFT remains valuable, although comparatively secondary, to enable precise adaptation to specialised tasks and domains, efectively balancing specific task performance with the retention of generalisable knowledge. Techniques such as PEFT, meta-learning and model merging were identified as promising approaches to achieving efective adaptation while managing computational resources and limiting data drift.
CCO, by contrast, is where academic research can and should place its primary emphasis. Moving from monolithic models to a decentralised ecosystems composed of specialised, modular agents facilitates adaptability, enhances scalability and reduces centralisation risks by enabling modular model replacement, upgrade and collaborative interaction. By studying and advancing CCO, the research community can catalyse an open, decentralised, and circular economy of AI components. Such a decentralised ecosystem not only encourages innovation and democratises access, but also mitigates the computational and environmental costs associated with continually retraining large, static models.
Ultimately, continual learning positions itself not merely as an optional enhancement but as a foundational requirement for future AI systems. As AI evolves from static to dynamic, from centralised to decentralised, and from monolithic to modular, the integration of continual learning methodologies will be crucial. Embracing continual learning will therefore be instrumental in building resilient, flexible and context-aware AI systems, capable of sustainably adapting to the ever-changing landscape of real-world challenges.
Research partly funded by PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - ”FAIR - Future Artificial Intelligence Research” - Spoke 1 ”Human-centered AI”, funded by the European Commission under the NextGeneration EU programme.
During the preparation of this work, the author(s) used OpenAI ChatGPT-4o in order to: Grammar and spelling check, Paraphrase and reword. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges, 2019.
URL: http://arxiv.org/abs/1907.00182. doi:10.48550/arXiv.1907.00182, arXiv:1907.00182 [cs]. [19] M. McCloskey, N. J. Cohen, Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem, in: G. H. Bower (Ed.), Psychology of Learning and Motivation, volume 24, Academic Press, 1989, pp. 109–165. URL: https://www.sciencedirect.com/science/ article/pii/S0079742108605368. doi:10.1016/S0079-7421(08)60536-8. [20] R. Ratclif, Connectionist models of recognition memory: constraints imposed by learning and forgetting functions., Psychological review 97 (1990) 285. [21] R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, J. Schmidhuber, Compete to Compute, in: C. J.
Burges, L. Bottou, M. Welling, Z. Ghahramani, K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems, volume 26, Curran Associates, Inc., 2013. URL: https://proceedings. neurips.cc/paper_files/paper/2013/file/8f1d43620bc6bb580df6e80b0dc05c48-Paper.pdf. [22] I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, Y. Bengio, An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks, 2015. URL: http://arxiv.org/abs/1312.6211. doi:10.48550/arXiv.1312.6211, arXiv:1312.6211 [stat]. [23] Z. Li, D. Hoiem, Learning without Forgetting, 2017. URL: http://arxiv.org/abs/1606.09282. doi:10.
48550/arXiv.1606.09282, arXiv:1606.09282 [cs]. [24] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, R. Hadsell, Overcoming catastrophic forgetting in neural networks, Proceedings of the National Academy of Sciences 114 (2017) 3521–3526. URL: http://arxiv.org/abs/1612.00796. doi:10.1073/pnas.1611835114, arXiv:1612.00796 [cs]. [25] L. Wang, X. Zhang, H. Su, J. Zhu, A comprehensive survey of continual learning: Theory, method and application, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024). [26] B. Wickramasinghe, G. Saha, K. Roy, Continual learning: A review of techniques, challenges, and future directions, IEEE Transactions on Artificial Intelligence 5 (2023) 2526–2546. [27] F. Zenke, B. Poole, S. Ganguli, Continual Learning Through Synaptic Intelligence, 2017. URL: http://arxiv.org/abs/1703.04200. doi:10.48550/arXiv.1703.04200, arXiv:1703.04200 [cs]. [28] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, R. Hadsell, Progressive Neural Networks, 2022. URL: http://arxiv.org/abs/1606.04671. doi:10. 48550/arXiv.1606.04671, arXiv:1606.04671 [cs]. [29] A. Mallya, S. Lazebnik, PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning, 2018. URL: http://arxiv.org/abs/1711.05769. doi:10.48550/arXiv.1711.05769, arXiv:1711.05769 [cs]. [30] A. Mallya, D. Davis, S. Lazebnik, Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights, 2018. URL: http://arxiv.org/abs/1801.06519. doi:10.48550/arXiv. 1801.06519, arXiv:1801.06519 [cs]. [31] J. Frankle, M. Carbin, The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks, 2019. URL: http://arxiv.org/abs/1803.03635. doi:10.48550/arXiv.1803.03635, arXiv:1803.03635 [cs]. [32] J. Serrà, D. Surís, M. Miron, A. Karatzoglou, Overcoming catastrophic forgetting with hard attention to the task, 2018. URL: http://arxiv.org/abs/1801.01423. doi:10.48550/arXiv.1801. 01423, arXiv:1801.01423 [cs]. [33] M. Wortsman, V. Ramanujan, R. Liu, A. Kembhavi, M. Rastegari, J. Yosinski, A. Farhadi, Supermasks in Superposition, in: Advances in Neural Information Processing Systems, volume 33, Curran Associates, Inc., 2020, pp. 15173–15184. URL: https://proceedings.neurips.cc/paper/2020/ hash/ad1f8bb9b51f023cdc80cf94bb615aa9-Abstract.html. [34] A. Prabhu, P. H. S. Torr, P. K. Dokania, GDumb: A Simple Approach that Questions Our Progress in Continual Learning, in: A. Vedaldi, H. Bischof, T. Brox, J.-M. Frahm (Eds.), Computer Vision – ECCV 2020, volume 12347, Springer International Publishing, Cham, 2020, pp. 524–540. URL: https://link.springer.com/10.1007/978-3-030-58536-5_31. doi:10.1007/978-3-030-58536-5_31, series Title: Lecture Notes in Computer Science. [35] D. Lopez-Paz, M. Ranzato, Gradient Episodic Memory for Continual Learning, 2022. URL: http: //arxiv.org/abs/1706.08840. doi:10.48550/arXiv.1706.08840, arXiv:1706.08840 [cs]. [36] S.-A. Rebufi, A. Kolesnikov, G. Sperl, C. H. Lampert, iCaRL: Incremental Classifier and Representation Learning, 2017. URL: http://arxiv.org/abs/1611.07725. doi:10.48550/arXiv.1611.07725, arXiv:1611.07725 [cs]. [37] L. Pellegrini, G. Grafieti, V. Lomonaco, D. Maltoni, Latent Replay for Real-Time Continual Learning, 2020. URL: http://arxiv.org/abs/1912.01100. doi:10.48550/arXiv.1912.01100, arXiv:1912.01100 [cs]. [38] G. Grafieti, D. Maltoni, L. Pellegrini, V. Lomonaco, Generative negative replay for continual learning, Neural Networks 162 (2023) 369–383. [39] G. M. Van de Ven, A. S. Tolias, Three scenarios for continual learning, arXiv preprint arXiv:1904.07734 (2019). [40] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019. URL: http://arxiv.org/abs/1810.04805. doi:10.48550/ arXiv.1810.04805, arXiv:1810.04805 [cs]. [41] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, D. Amodei, Language Models are FewShot Learners, 2020. URL: http://arxiv.org/abs/2005.14165. doi:10.48550/arXiv.2005.14165, arXiv:2005.14165 [cs]. [42] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning Transferable Visual Models From Natural Language Supervision, 2021. URL: http://arxiv.org/abs/2103.00020. doi:10.48550/arXiv.2103. 00020, arXiv:2103.00020 [cs]. [43] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. v. Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. Chatterji, A. Chen, K. Creel, J. Q. Davis, D. Demszky, C. Donahue, M. Doumbouya, E. Durmus, S. Ermon, J. Etchemendy, K. Ethayarajh, L. Fei-Fei, C. Finn, T. Gale, L. Gillespie, K. Goel, N. Goodman, S. Grossman, N. Guha, T. Hashimoto, P. Henderson, J. Hewitt, D. E. Ho, J. Hong, K. Hsu, J. Huang, T. Icard, S. Jain, D. Jurafsky, P. Kalluri, S. Karamcheti, G. Keeling, F. Khani, O. Khattab, P. W. Koh, M. Krass, R. Krishna, R. Kuditipudi, A. Kumar, F. Ladhak, M. Lee, T. Lee, J. Leskovec, I. Levent, X. L. Li, X. Li, T. Ma, A. Malik, C. D. Manning, S. Mirchandani, E. Mitchell, Z. Munyikwa, S. Nair, A. Narayan, D. Narayanan, B. Newman, A. Nie, J. C. Niebles, H. Nilforoshan, J. Nyarko, G. Ogut, L. Orr, I. Papadimitriou, J. S. Park, C. Piech, E. Portelance, C. Potts, A. Raghunathan, R. Reich, H. Ren, F. Rong, Y. Roohani, C. Ruiz, J. Ryan, C. Ré, D. Sadigh, S. Sagawa, K. Santhanam, A. Shih, K. Srinivasan, A. Tamkin, R. Taori, A. W. Thomas, F. Tramèr, R. E. Wang, W. Wang, B. Wu, J. Wu, Y. Wu, S. M. Xie, M. Yasunaga, J. You, M. Zaharia, M. Zhang, T. Zhang, X. Zhang, Y. Zhang, L. Zheng, K. Zhou, P. Liang, On the Opportunities and Risks of Foundation Models, 2022. URL: http://arxiv.org/abs/2108.07258. doi:10.48550/arXiv.2108.07258, arXiv:2108.07258 [cs]. [44] C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al., A comprehensive survey on pretrained foundation models: A history from bert to chatgpt, International Journal of Machine Learning and Cybernetics (2024) 1–65. [45] Y. Yang, J. Zhou, X. Ding, T. Huai, S. Liu, Q. Chen, Y. Xie, L. He, Recent advances of foundation language models-based continual learning: A survey, ACM Computing Surveys 57 (2025) 1–38. [46] A. Plaat, A. Wong, S. Verberne, J. Broekens, N. v. Stein, T. Back, Reasoning with Large Language Models, a Survey, 2024. URL: http://arxiv.org/abs/2407.11511. doi:10.48550/arXiv.2407.11511, arXiv:2407.11511 [cs] version: 1. [47] J. Hofmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. v. d. Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, L. Sifre, Training Compute-Optimal Large Language Models, 2022. URL: http://arxiv.org/abs/2203.15556. doi:10. 01.012. doi:10.1016/j.neunet.2019.01.012. [86] H. Du, S. Liu, L. Zheng, Y. Cao, A. Nakamura, L. Chen, Privacy in fine-tuning large language models: Attacks, defenses, and future directions, 2025. URL: https://arxiv.org/abs/2412.16504. arXiv:2412.16504. [87] Z. Wang, Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, T. Pfister, Learning to prompt for continual learning, 2022. URL: https://arxiv.org/abs/2112.08654. arXiv:2112.08654. [88] J. S. Smith, L. Karlinsky, V. Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, Z. Kira, Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning, 2023. URL: https://arxiv.org/abs/2211.13218. arXiv:2211.13218. [89] H. Chen, J. Li, N. Gazagnadou, W. Zhuang, C. Chen, L. Lyu, Dual low-rank adaptation for continual learning with pre-trained models, 2024. URL: https://arxiv.org/abs/2411.00623. arXiv:2411.00623. [90] W. Zhang, Y. Huang, T. Zhang, Q. Zou, W.-S. Zheng, R. Wang, Adapter learning in pretrained feature extractor for continual learning of diseases, 2023. URL: https://arxiv.org/abs/2304.09042. arXiv:2304.09042. [91] X. Gao, S. Dong, Y. He, Q. Wang, Y. Gong, Beyond prompt learning: Continual adapter for eficient rehearsal-free continual learning, 2024. URL: https://arxiv.org/abs/2407.10281. arXiv:2407.10281. [92] P. Yadav, D. Tam, L. Choshen, C. Rafel, M. Bansal, Ties-merging: Resolving interference when merging models, 2023. URL: https://arxiv.org/abs/2306.01708. arXiv:2306.01708. [93] L. Yu, B. Yu, H. Yu, F. Huang, Y. Li, Language models are super mario: Absorbing abilities from homologous models as a free lunch, 2024. URL: https://arxiv.org/abs/2311.03099. arXiv:2311.03099. [94] D. Marczak, B. Twardowski, T. Trzciński, S. Cygert, Magmax: Leveraging model merging for seamless continual learning, 2024. URL: https://arxiv.org/abs/2407.06322. arXiv:2407.06322. [95] E. Yang, L. Shen, Z. Wang, G. Guo, X. Chen, X. Wang, D. Tao, Representation surgery for multi-task model merging, 2024. URL: https://arxiv.org/abs/2402.02705. arXiv:2402.02705. [96] E. N. Coleman, L. Quarantiello, J. Hurtado, V. Lomonaco, Adaptive LoRA merging for eficient domain incremental learning, in: Adaptive Foundation Models: Evolving AI for Personalized and Eficient Learning, 2024. URL: https://openreview.net/forum?id=tlB5eonGEk. [97] C. Finn, P. Abbeel, S. Levine, Model-agnostic meta-learning for fast adaptation of deep networks, 2017. URL: https://arxiv.org/abs/1703.03400. arXiv:1703.03400. [98] R. Zhang, R. Qiang, S. A. Somayajula, P. Xie, Autolora: Automatically tuning matrix ranks in low-rank adaptation based on meta learning, 2024. URL: https://arxiv.org/abs/2403.09113. arXiv:2403.09113. [99] W. Li, L. Zou, M. Tang, Q. Yu, W. Li, C. Li, META-LORA: Memory-eficient sample reweighting for ifne-tuning large language models, in: O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, S. Schockaert (Eds.), Proceedings of the 31st International Conference on Computational Linguistics, Association for Computational Linguistics, Abu Dhabi, UAE, 2025, pp. 8504–8517.
URL: https://aclanthology.org/2025.coling-main.568/. [100] L. Palazzo, M. Pennisi, F. P. Salanitri, G. Bellitto, S. Palazzo, C. Spampinato, Fedrewind: Rewinding continual model exchange for decentralized federated learning, 2024. URL: https://arxiv.org/abs/ 2411.09842. arXiv:2411.09842. [101] F. Chollet, Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI), https://github.com/fchollet/ARC-AGI, 2019. URL: https://github.com/fchollet/ARC-AGI. [102] A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al., Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, arXiv preprint arXiv:2206.04615 (2022). [103] C. Luo, Has llm reached the scaling ceiling yet? unified insights into llm regularities and constraints, arXiv preprint arXiv:2412.16443 (2024). [104] K.-T. Tran, D. Dao, M.-D. Nguyen, Q.-V. Pham, B. O’Sullivan, H. D. Nguyen, Multi-agent collaboration mechanisms: A survey of llms, arXiv preprint arXiv:2501.06322 (2025). [105] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, et al., A survey on large language model based autonomous agents, Frontiers of Computer Science 18 (2024) 186345. [106] B. Liu, X. Li, J. Zhang, J. Wang, T. He, S. Hong, H. Liu, S. Zhang, K. Song, K. Zhu, Y. Cheng, S. Wang, X. Wang, Y. Luo, H. Jin, P. Zhang, O. Liu, J. Chen, H. Zhang, Z. Yu, H. Shi, B. Li, D. Wu, F. Teng, X. Jia, J. Xu, J. Xiang, Y. Lin, T. Liu, T. Liu, Y. Su, H. Sun, G. Berseth, J. Nie, I. Foster, L. Ward, Q. Wu, Y. Gu, M. Zhuge, X. Tang, H. Wang, J. You, C. Wang, J. Pei, Q. Yang, X. Qi, C. Wu, Advances and challenges in foundation agents: From brain-inspired intelligence to evolutionary, collaborative, and safe systems, 2025. URL: https://arxiv.org/abs/2504.01990. arXiv:2504.01990. [107] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., Chain-ofthought prompting elicits reasoning in large language models, Advances in neural information processing systems 35 (2022) 24824–24837. [108] Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, et al., The rise and potential of large language model based agents: A survey, Science China Information Sciences 68 (2025) 121101. [109] b. ichter, A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, D. Kalashnikov, S. Levine, Y. Lu, C. Parada, K. Rao, P. Sermanet, A. T. Toshev, V. Vanhoucke, F. Xia, T. Xiao, P. Xu, M. Yan, N. Brown, M. Ahn, O. Cortes, N. Sievers, C. Tan, S. Xu, D. Reyes, J. Rettinghouse, J. Quiambao, P. Pastor, L. Luu, K.-H. Lee, Y. Kuang, S. Jesmonth, N. J. Joshi, K. Jefrey, R. J. Ruano, J. Hsu, K. Gopalakrishnan, B. David, A. Zeng, C. K. Fu, Do as i can, not as i say: Grounding language in robotic afordances, in: K. Liu, D. Kulic, J. Ichnowski (Eds.), Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 287–318. URL: https://proceedings.mlr.press/v205/ichter23a. html. [110] B. Xu, Z. Peng, B. Lei, S. Mukherjee, Y. Liu, D. Xu, Rewoo: Decoupling reasoning from observations for eficient augmented language models, CoRR abs/2305.18323 (2023). URL: https://doi.org/10. 48550/arXiv.2305.18323. [111] S. S. Raman, V. Cohen, E. Rosen, I. Idrees, D. Paulius, S. Tellex, Planning with large language models via corrective re-prompting, in: NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022. URL: https://openreview.net/forum?id=cMDMRBe1TKs. [112] S. Yao, D. Yu, J. Zhao, I. Shafran, T. Grifiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models, Advances in neural information processing systems 36 (2023) 11809–11822. [113] N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al., Glam: Eficient scaling of language models with mixture-of-experts, in: International conference on machine learning, PMLR, 2022, pp. 5547–5569. [114] B. Lin, Z. Tang, Y. Ye, J. Cui, B. Zhu, P. Jin, J. Huang, J. Zhang, Y. Pang, M. Ning, et al., Moe-llava:
Mixture of experts for large vision-language models, arXiv preprint arXiv:2401.15947 (2024). [115] F. Xue, Z. Zheng, Y. Fu, J. Ni, Z. Zheng, W. Zhou, Y. You, Openmoe: An early efort on open mixture-of-experts language models, arXiv preprint arXiv:2402.01739 (2024). [116] D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, Y. Wu, et al., Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models, arXiv preprint arXiv:2401.06066 (2024). [117] K. A. Fischer, Reflective linguistic programming (rlp): A stepping stone in socially-aware agi (socialagi), arXiv preprint arXiv:2305.12647 (2023). [118] C. Li, J. Wang, K. Zhu, Y. Zhang, W. Hou, J. Lian, X. Xie, Emotionprompt: Leveraging psychology for large language models enhancement via emotional stimulus, arXiv preprint arXiv:2307.11760 (2023). [119] X. Wang, S. Zhang, S. Li, K. Li, K. Kallidromitis, Y. Kato, K. Kozuka, T. Darrell, Segllm: Multi-round reasoning segmentation with large language models, in: The Thirteenth International Conference on Learning Representations, ???? [120] Z. Zhou, X. Zhang, S. Tan, L. Zhang, C. Li, Collaborative evolution: Multi-round learning between large and small language models for emergent fake news detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 2025, pp. 1210–1218. [121] Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al., Autogen: Enabling next-gen llm applications via multi-agent conversation, arXiv preprint arXiv:2308.08155 (2023). [122] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, A. Anandkumar, Voyager: An open-ended embodied agent with large language models, Transactions on Machine Learning Research (2024). URL: https://openreview.net/forum?id=ehfRiF0R3a. [123] X. Zhu, Y. Chen, H. Tian, C. Tao, W. Su, C. Yang, G. Huang, B. Li, L. Lu, X. Wang, Y. Qiao, Z. Zhang, J. Dai, Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory, 2023. URL: https://arxiv.org/abs/2305. 17144. arXiv:2305.17144. [124] C. Zhang, Z. Yang, J. Liu, Y. Han, X. Chen, Z. Huang, B. Fu, G. Yu, Appagent: Multimodal agents as smartphone users, arXiv preprint arXiv:2312.13771 (2023). [125] A. Madaan, N. Tandon, P. Clark, Y. Yang, Memory-assisted prompt editing to improve gpt-3 after deployment, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 2833–2861. [126] T. Sumers, S. Yao, K. Narasimhan, T. Grifiths, Cognitive architectures for language agents, Transactions on Machine Learning Research (2024). URL: https://openreview.net/forum?id= 1i6ZCvflQJ, survey Certification. [127] G. Li, H. Hammoud, H. Itani, D. Khizbullin, B. Ghanem, Camel: Communicative agents for” mind” exploration of large language model society, Advances in Neural Information Processing Systems 36 (2023) 51991–52008. [128] H. Liu, Y. Zhou, B. Liu, J. Zhao, R. Yao, Z. Shao, Incremental learning with neural networks for computer vision: a survey, Artificial intelligence review 56 (2023) 4557–4589. [129] K. Khetarpal, M. Riemer, I. Rish, D. Precup, Towards continual reinforcement learning: A review and perspectives, Journal of Artificial Intelligence Research 75 (2022) 1401–1476. [130] J. Gou, B. Yu, S. J. Maybank, D. Tao, Knowledge distillation: A survey, International Journal of
Computer Vision 129 (2021) 1789–1819. [131] C. Yang, Y. Zhu, W. Lu, Y. Wang, Q. Chen, C. Gao, B. Yan, Y. Chen, Survey on knowledge distillation for large language models: methods, evaluation, and application, ACM Transactions on Intelligent Systems and Technology (2024). [132] A. Carta, A. Cossu, V. Lomonaco, D. Bacciu, Ex-model: Continual learning from a stream of trained models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3790–3799. [133] Z. Kenton, T. Everitt, L. Weidinger, I. Gabriel, V. Mikulik, G. Irving, Alignment of language agents, arXiv preprint arXiv:2103.14659 (2021). [134] W. Du, Z. M. Kim, V. Raheja, D. Kumar, D. Kang, Read, revise, repeat: A system demonstration for human-in-the-loop iterative text revision, in: Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022), 2022, pp. 96–108.