Despite the remarkable success of large language models (LLMs), they still face bottlenecks while deploying in dy namic, real-world settings with primary challenges being concept drift and the high cost of gradient-based adapta tion. Traditional fine-tuning (FT) struggles to adapt to non stationary data streams without resulting in catastrophic for getting or requiring extensive manual data curation. To ad dress these limitations within the streaming and continual learning paradigm, we propose the Self-Optimizing Lifelong Autonomous Reasoner (SOLAR) which is an open-ended au tonomous agent that leverages parameter-level meta-learning to self-improve, treating model weights as an environment for exploration. It initiates the process by consolidating a strong prior over common-sense knowledge making it efective for transfer-learning. By utilizing a multi-level reinforcement learning approach, SOLAR autonomously discovers adaptation strategies, enabling eficient test-time adaptation to unseen domains. Crucially, SOLAR maintains an evolving knowledge base of valid modification strategies, implicitly acting as an episodic memory bufer to balance plasticity (adaptation to new tasks) and stability (retention of meta-knowledge). Experiments demonstrate that SOLAR outperforms strong baselines on commonsense, mathemati cal, medical, coding, social and logical reasoning tasks, marking a significant step toward autonomous agents capable of lifelong adaptation in evolving environments.
Large Language Models (LLMs) possess remarkable emer gent abilities due to massive pretraining. However, deploy ing them in streaming environments reveals a critical weak ness which is the inability to adapt to non-stationary data distributions (concept drift) without expensive retraining or human intervention. While Parameter-Eficient Fine-Tuning (PEFT) techniques like LoRA (Hu et al. 20222) reduce the parameter update volume, they remain static solutions that do not inherently address the stability-plasticity dilemma central to Continual Learning (CL). Existing adaptation strategies often rely on generic, hand crafted heuristics that fail to generalize across the shifting temporal dependencies of real-world streams. This disconnect necessitates a system that can not only adapt parameters on the lfybut also learn how to adapt based on accumulating experience. We propose that the high-dimensional weight space of an LLMcontains rich meta-knowledge that, if navi gated autonomously, can yield bespoke adaptation strategies for novel tasks. This motivates our primary research question: RQ: Can LLMs learn to modify their internal representation space autonomously to handle concept drift, analogous to how humans assimilate and restructure knowledge in lifelong learning scenarios? To answer this, we investigate the cognitive science of life long learning. As humans, we do not merely memorize new data, instead we restructure our internal schematics in order to be able to accommodate new information while simultaneously retaining prior heuristics. This process is what has essentially enabled humans to navigate non-stationary environments. For instance, a student adapts their study strategy based on the nature of a new subject (plasticity) without unlearning how to study generally (stability). Current LLM adaptation, by contrast, is often rigid since models consume task data “as-is”, failing to develop bespoke internal trans formation strategies. To replicate this cognitive lfexibility, we introduce SOLAR (Self-Optimizing Lifelong Autonomous Reasoner). It functions as a meta-learning agent that decouples rapid task adaptation (streaming machine learning) from long-term strategy retention (continual learning). By discovering and validating parameter-level modifications, SOLAR enables eficient adaptation to unseen tasks while populating a persistent knowledge base to mitigate catastrophic forgetting. This work thus bridges the gap between static parameter generation and dynamic, lifelong self-evolution. Further more, by grounding the search space in neural network weights, we target generalized principles of model capability rather than task-specific memorization. Just as scaling laws (Kaplan et al. 2020) predict performance based on size, we posit that predictable weightmodification patterns exist that allow for rapid, data-eficient adaptation to concept drift, minimizing the lag between detecting a distributional shift and deploying an updated model. The remainder of this paper is organized as follows. In Sec tion 2, we highlight the motivation for our approach in detail. Section 3 presents the literature survey conducted, Section 4 contains the methodology with implementation specifics in Section 5. Experimental results are provided in Section 6 and in Section 7, we present our concluding remarks.
Our primary motivation stems from human psychology and pedagogy. For example, consider a human student who is preparing to take an end-sem examination of a machine learning course. Quite often, students tend to rely on their prior prepared notes for preparation. These notes are often derived from the lecture content, textbooks or information available on the internet. Thus instead of relying on the raw content, students assimilate and rewrite the information in the form of notes as per their own intrinsic reasoning skill and aptitude. This improves the capability of students to comprehend the content better and therefore respond well to the exam questions. This phenomenon of reinterpreting and augmenting external knowledge in a way that is easier to understand as well as developing the necessary skill-sets is not limited to just taking exams, but seems to be universally true of human learning across tasks. Furthermore, depending on one’s interests, humans assimilate information in diferent ways - some might condense the information into a visual diagram, some into text, or some might rely more on concrete mathematical descriptions. Such restructuring or development of internal knowledge as well as assimilation or rewriting of external information, as part of the learning process is in contrast with how LLMs undergo currently training and adaptation. Given a new task, current LLMs consume and learn from the task data "as-is" via finetuning or in-context learning. The issue with this, just like in the human setting, such data may not be in an optimal format (or volume) for learning, or there might not be the relevant skill-set developed to learn it and current approaches do not enable models to develop bespoke strategies for how to best transform themselves internally or even learn from their training data. In this work, we therefore investigate the question as to if it is possible for even LLMs, analogous to humans, to suggest strategies by themselves which can enable them to perform better on a given task.
A secondary source of motivation as to why we ground our strategy search space in the neural
network weights is because unlike task-specific knowledge, the weight-level meta-knowledge represents
generalized principles about how neural network parameters relate to model capabilities, thereby
providing crucial insights for self-evolving agents. There are several prior research works which have
shown that there exists a positive correlation between types of neural network weight patterns and
downstream model performance characteristics. For example, scaling laws research [
Test-Time Training (TTT) is a recently emerging class of approaches which updates model weights
at inference time using techniques such as input perplexity or cross-entropy minimization on only
unlabeled test data enabling self supervised enhancement of LLM performance [
Adversarial Fine Tuning is another emerging class of techniques where in two LLM instances are
made to debate with each other about a topic or one instance serves as a challenger or teacher and the
other instance serves as a solver or student to generate synthetic data, either from unlabeled prompts
or even from scratch itself and use approaches like majority voting to create pseudo-labels which can
further be used for updating model’s knowledge accordingly [
Reinforcement Learning (RL) is a well established approach for pushing the capabilities of LLMs and
recent works such as SEAL [
Parameter Generation is another research direction which has seen several pioneering works such as
RPG [
In this section, we describe the framework of our proposed approach (see Figure 1). SOLAR starts by
treating the LLMs own weights as environment variables to explore, upon which it would systematically
propose scientific hypotheses to modify the internal representation space appropriately so as to adapt
the LLM to the unseen task. A major challenge for the design, therefore, is the high dimensionality and
non-convexity of the LLM weight space itself which makes the initialization and subsequent exploration
process extremely complex. To overcome this, we work only with low-rank parameters [
Once the weights have been initialized1 for exploration, SOLAR then uses a foundation-model-based
agent, which is for now simply an LLM trained using reinforcement learning (RL) to come up with
probable hypotheses at inference time for weight-space exploration using test-time scaling and compute.
To however, facilitate the training process, it is necessary to first curate by hand a seed knowledge base,
consisting of either proven or plausible weight modification strategies, which will then serve as the
action space for LLM’s initial stages of exploration during RL training. This would be a multi-stage
recipe consisting of three distinct progressively harder levels. Level I consists of training the LLM
to produce only single valid and eficient self-edits (a self-edit as the name suggests is basically a
modification strategy proposed by an LLM to update its own weights depending on the task) from
among the ones present in the initial knowledge base. Level II comprises of training the model to
output chain-of-self-edits, since coupling strategies sequentially is also helpful (moreover if viewed
in a abstract sense, it can be considered in efect as a single complex edit which can be decomposed
into simpler instances). Level III is a significantly challenging aspect both for the LLMs as well as from
implementation perspective as well, which is basically letting LLMs to explore the hypothesis space in
its entirety, thereby going beyond human-crafted approaches. A positive performance in Level III would
be a significant leap as it could possibly open up new frontiers in training and fine-tuning paradigms as
has been similarly done in other areas as well such as neural architecture search [
After plausible hypothesis have been generated by the foundation model-based agent and implemented,
its necessary to test the hypothesis. For this purpose, we create a separate evaluation split if available.
However, since SOLAR is designed to adapt LLMs eficiently to unseen tasks as well, the dataset for
evaluation itself would be generated on the fly using adversarial approaches involving multiple instances
of an LLM, one proposing and one solving questions on a particular topic as in SQLM [
Primary architectural detail in SOLAR’s framework is the design of the weight-space exploration
initializer. As mentioned in Section 4, we use a convolution based decoder model for this purpose. We
assume that we have access to either the unseen task’s description or atleast a handful of unlabeled
examples representative of its requirements. We then send them to an open-sourced text encoder for
embedding extraction. This extraction process can be formally represented as, = Encoder(, ),
where Encoder(·, ·) denotes the embedding extraction function parameterized by , and represents
the extracted embedding corresponding to prompt . We use an encoder-based language model
architecture for this purpose i.e., Sentence-BERT (all-MiniLM-L6-v2 specifically) [
Say, the dimension of prompt embeddings is [, , , ] where , , and denote batch size,
length of prompt batch (i.e., number of prompts), sequence length, and hidden dimension, respectively.
The decoder (see Figure 3) consists of multiple sequential layers, each performing 5 2D convolutions.
These convolutions are divided into three categories: i) width convolution that operates on (, )
2It is to be noted that BERT’s supported sequence length is only 512 and for longer sequences, padding should be done.
However, in our use case, maximum sequence length is only 384 and thus padding is not necessary.
dimension, ii) height convolution that operates on (, ) dimension) iii) layer-wise convolution
that on (, ) dimension) , with notations Conv , Conv , and Conv. Each layer consists of two
Conv , two Conv and one Conv. Given this, the forward operation of the decoder block is,
= Conv1 (Conv1 (−1 ))
= Conv2 (︁ Conv2 (−1 )︁)
= Conv (︁ ( + + )/3
︁)
where is hidden state output by the th layer, 0 is prompt embedding encoded by the condition
extractor, and is learnable bias. Through this process, input is transformed from dimension [, , , ]
to [, ′, ′, ′] which is then compatible to be converted into a flattened LoRA adapter for the LLM
3. In this work, the base LLM used is Qwen2.5-0.5B-Instruct [
In this work, we focus on the domain of common-sense reasoning and select 4 datasets for evaluation,
namely HellaSwag [
→ (128,200,300) → (128,100,256) → (256,50,200) → (512,50,200) Subsequently, prompt-checkpoint pairing is done as follows. Given a dataset , it is first divided it into non-overlapping prompt batches [1, · · · , , · · · , ]. Denote the trained LLM checkpoints of this dataset as = [1, · · · , , · · · , ]. Then randomly a batch of prompts and a corresponding checkpoint is picked to create a pair {, }, which then serves as an input-output data point for training the decoder. The objective function for training is the mean squared error (MSE) loss between the output from the decoder’s last block for a particular prompt batch and the training checkpoint associated with it.
Next crucial step is the hand-crafting of seed knowledge base. To this end, we identify five primary
families of strategies5, each containing its own sub-strategies as well, namely
• Test-Time Training (TTT) using input perplexity minimization [
tamper with internal probability distribution of next-tokens unlike other families which modify the parameters explicitly.
We first formulate the objective for outer-loop RL training which generates adaptation strategies AS,
as in [
EAS∼LM (·|) [(AS, , )] ]︁ It is to be noted that the reward assigned to a given action depends on the model parameters at the time the action is taken (since is updated to ′, which is then evaluated). An implication of this is that the while modeling the RL state, one must therefore include in the policy’s parameters as well along with , even though the policy’s observation is limited to (because it is extremely infeasible to directly place in the LLM’s context window). Therefore, the (state, action, reward) triples which have been collected by using an older model weights, old, will not be aligned for the current model current. Hence, an on-policy approach should be adapted, by which adaptation strategies are sampled from and, even more importantly, the rewards itself will be calculated using the current model. In particular, the specific on-policy approach used is ReST EM [61] where samples are first generated 6 from the current model and are filtered by using binary feedback [ (AS, , ) is 1 if on , AS improves LM ’s performance and is 0 otherwise]. The model is then fine-tuned on these samples and this continues in an iterative manner (See Algorithm 1).
A subtle detail, which hasn’t yet been covered is the exact nature of the adaptation strategy itself.
This depends on the particular strategy family being used, however the format is consistent across
all which is basically a JSON object specifying the particular configurations to be used 7. It contains
a field, family which takes values TTT, LoRA and TTS. Currently, the following choices have been
experimented
• For TTT, we use [
• For LS, we use [
6.1. Setup
As described in Section 5, the base LLM used is Qwen2.5-0.5B-Instruct, domain is
common-sensereasoning and evaluation datasets are ARC-c, BoolQ, HellaSwag and ARC-e. Baselines used include
quite recent works such as DnD [
All experiments were conducted on a high-performance computing node running Ubuntu 22.04.1. The backend processor was EPYC 8434P, which had 48 physical cores (96 logical threads), 256 GB of system RAM and a maximum clock speed of 2.5 GHz. Four NVIDIA RTX A6000 GPUs, each with 48 GB of dedicated VRAM were utilized. Python version used was 3.12.11 and GPU-accelerated tasks were managed using CUDA version 12.4. 9DOM is a data-free framework for LoRA merging. It separates parameters into magnitude and direction components and merges them independently, thereby reducing the impact of magnitude diferences on the directional alignment of the merged models, thus helping in preserving task information. It also uses a data-free, layer-wise gradient descent method with orthogonal constraints to mitigate interference during the merging of direction components. For evaluation on a target dataset, LoRA’s of remaining datasets are merged and used.
Algorithm 1 Sequential Multi-Level RL Loop for Adaptation Strategy (AS) Generation of SOLAR 1: Input: Base LM , dataset context , evaluation metric , initial knowledge base 2: Init: Low-rank adapter generator , sampled adapters ← Sampler(, ) 3: Level I (Single-edit self-training): 4: for iteration = 1, . . . , 1 do 5: Propose single-edit AS from 6: Apply AS and obtain weights 7: Evaluate 8: Compute reward 9: if > threshold1 then 10: ← RL_Update(, , AS) 11: end if 12: end for 13: Level II (Chained/compositional strategies): 14: for iteration = 1, . . . , 2 do 15: Propose chain of edits 16: Sequentially apply chain 17: Evaluate final weights 18: Compute reward 19: if > threshold2 then 20: Add chain to KB 21: ← RL_Update(, , AS) 22: end if 23: end for 24: Level III (Open-ended exploration): 25: for iteration = 1, . . . , 3 do 26: Generate unconstrained AS 27: Validate (syntax/safety); if invalid continue 28: Apply AS conservatively (strong meta-reg) 29: Evaluate and compute reward 30: if > threshold3 then 31: ← ∪ {AS}; ← RL_Update(, , AS) 32: else 33: Penalize harmful proposals in policy update 34: end if 35: end for 36: Return: Refined parameters * , enriched KB *
AS ∼ LM (, ) ′ ← ApplyStrategy(, AS, )
Ans ∼ LM ′ (· | )
← (Ans, ) 0 ← ;
AS = [1, . . . , ], ∈ ← ApplyStrategy( −1 , , )
Ans ∼ LM (· | )
← (Ans, ) ← ∪ {AS} AS ∼ LM (· | ) (novel structure)
′ ← ApplyStrategy(, AS, ) Ans ∼ LM ′ (· | ); ← (Ans, ) The major results of this work are presented in Table 1 wherein we conduct experiments of 5 benchmarks which are in the domain of common-sense reasoning and also on 5 out-of-domain benchmarks namely GSM-MC and MATH-MC 10 to evaluate mathematical reasoning, DivLogicEval [66] for logical reasoning, SocialIQA [67] for reasoning about social interactions and CodeMMLU [68] for reasoning about coderelated tasks. It can be seen that SOLAR in its initial version itself outperforms the task-specific training LoRA’s, TTL, DOM and even DnD by a significant margin, showcasing the promising potential it is capable of, if further levels of RL training11 is completed as well.
Following were the adaptation strategies identified, which enabled SOLAR to reach the accuracy levels presented, 10GSM-MC and MATH-MC are multiple choice versions of the standard GSM-8K [63] and MATH [64] datasets. They were selected for two reasons - ease of evaluation and correlation with performance on their subjective counterparts [65]. 11This might be quite time-intensives however with current version itself taking around 4 days using 2 A6000 GPU’s. The reason for using only 2 despite 4, is because Qwen family has 14 attention heads and the vllm serves used for improved eficiency in inference requires this number to be divisible by the number of GPU’s which is only possible if either 2 or 7 are available. 1e-5, "batch_size”: 4, "shuffle_data”: True} • For ARC-c and SocialIQA, it was LS family with configuration {“ times”: 5, "learning_rate”: 0.1} • For BoolQ, GSM-MC and MATH-MC, it was LoRA family with TS-mixing strategy and the configuration was {“lambda”: 0.5} • For HellaSwag, DivLogicEval and CodeMMLU, it was TTS family. Ex:, for Hellaswag, the corresponding configuration was {“ num_prompt_batches”: 20, "method”: max_confidence}, indicative of the ensemble approach
A primary efect we would like to isolate and study is that of initial prompt batch provided to start the
LLM adaptation process using SOLAR. It would be ideal if SOLAR results in similar performance even
if a highly representative, diverse and influential prompt batch is used. For this purpose, inspired by
[
=
1 |V| − 1 ∑︁ (, ), = ⌈ · · (|V| − 1)⌉ ̸= Samples are then scored by by (1) influence and (2) diversity. The influence score () is obtained by a difusion simulation 12. For this, first initialize an active set active = {}, then iteratively sample an active node and attempt to activate each neighbor ∈ 1() with probability (, ). Newly activated nodes join active. This process is repeated until no active nodes remain. Let () be the total number of visited nodes. Diversity penalty () measures overlap with already selected nodes: () = −
∑︁ ⃒⃒ () ∩ selected⃒⃒ , () = () + () =1 12The simulation is run 20 times and is then averaged to obtain the final value. Finally, greedy graph search is done to select the final prompt subset . For this, start with = ∅ and at each round pick * = arg max (),
∈/ * is then added to and diversity penalties only for neighbors of * are updated13. This process continues until || reaches the target size which in our case is 128.
Fortunately, the influence of the initial prompt batch was marginal (with just a 0.3% improvement in accuracy when averaged across all evaluation datasets), indicating that SOLAR can eficiently adapt LLMs to unseen datasets without the requirement of high-quality or manually curated dataset. Only a handful of unlabeled prompt instances which are merely indicative of the task sufice.
In this paper, we introduce SOLAR which is a novel paradigm for Streaming and Continual Learning by empowering LLMs to autonomously discover and retain parameter- level adaptation strategies. By bridging the gap between rapid test-time adaptation (plasticity) and long-term meta-knowledge retention (stability), SOLAR addresses the core challenges of deploying agents in non-stationary environments. While currently reliant on a seed knowledge base, the framework lays the groundwork for fully autonomous, self-evolving systems capable of navigating the open-ended drifts of the real world. Another key tradeof is that of real- time adaptation versus computation. While SOLAR’s training phase is compute-intensive, the inference-time application of learned strategies is rapid. By pre-compiling complex adaptation routines into the knowledge base, SOLAR shifts the computational burden from the streaming phase to the ofline meta-learning phase. This allows the agent to react to concept drift in near real-time by simply retrieving and applying a cached strategy, rather than performing expensive gradient descent from scratch every time. 13Note that the influence scores are precomputed.
The authors would like to thank Professor Sashikumaar Ganesan, from the Department of Computational and Data Science at Indian Institute of Science, Bangalore for feedback and additional compute resources required to execute this project.
During the preparation of this work, the authors used Large Language Models (GPT-5.2, Claude Opus 4.5 and Gemini-3) as a writing assistant tool for drafting content, to generate literature review, for abstract drafting, to paraphrase and reword, to improve writing style, for grammar and spelling check as well as to generate the images used in the paper. The process was interactive. After writing the core content, the authors used LLMs with specific prompts to refine the text. These prompts included requests to “check for grammatical errors,” “rephrase this sentence for clarity,” “make this paragraph more concise,” or “suggest alternative phrasing to improve flow.” The LLMs were not used to generate any scientific ideas, experimental results, data analysis or other core intellectual contributions of the paper. After using these tool(s)/service(s), the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. //arxiv.org/abs/2501.19050. arXiv:2501.19050. [56] Z. Zhao, T. Shen, D. Zhu, Z. Li, J. Su, X. Wang, K. Kuang, F. Wu, Merging loras like playing lego: Pushing the modularity of lora to extremes through rank-wise clustering, 2024. URL: https: //arxiv.org/abs/2409.16167. arXiv:2409.16167. [57] L. Chen, M. Prabhudesai, K. Fragkiadaki, H. Liu, D. Pathak, Self-questioning language models, 2025. URL: https://arxiv.org/abs/2508.03682. arXiv:2508.03682. [58] G. Zhang, F. Meng, G. Wan, Z. Li, K. Wang, Z. Yin, L. Bai, S. Yan, Latentevolve: Self-evolving test-time scaling in latent space, 2025. URL: https://arxiv.org/abs/2509.24771. arXiv:2509.24771. [59] A. Karan, Y. Du, Reasoning with sampling: Your base model is smarter than you think, 2025. URL: https://arxiv.org/abs/2510.14901. arXiv:2510.14901. [60] Z. Wang, D. Ma, X. Huang, D. Cai, T. Lan, J. Xu, H. Mi, X. Tang, Y. Wang, The end of manual decoding: Towards truly end-to-end language models, 2025. URL: https://arxiv.org/abs/2510.26697. arXiv:2510.26697. [61] A. Singh, J. D. Co-Reyes, R. Agarwal, A. Anand, P. Patil, X. Garcia, P. J. Liu, J. Harrison, J. Lee, K. Xu, A. Parisi, A. Kumar, A. Alemi, A. Rizkowsky, A. Nova, B. Adlam, B. Bohnet, G. Elsayed, H. Sedghi, I. Mordatch, I. Simpson, I. Gur, J. Snoek, J. Pennington, J. Hron, K. Kenealy, K. Swersky, K. Mahajan, L. Culp, L. Xiao, M. L. Bileschi, N. Constant, R. Novak, R. Liu, T. Warkentin, Y. Qian, Y. Bansal, E. Dyer, B. Neyshabur, J. Sohl-Dickstein, N. Fiedel, Beyond human data: Scaling selftraining for problem-solving with language models, 2024. URL: https://arxiv.org/abs/2312.06585. arXiv:2312.06585. [62] S. Zheng, H. Wang, C. Huang, X. Wang, T. Chen, J. Fan, S. Hu, P. Ye, Decouple and orthogonalize: A data-free framework for lora merging, 2025. URL: https://arxiv.org/abs/2505.15875. arXiv:2505.15875. [63] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al., Training verifiers to solve math word problems, arXiv preprint arXiv:2110.14168 (2021). [64] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, J. Steinhardt, Measuring mathematical problem solving with the math dataset, arXiv preprint arXiv:2103.03874 (2021). [65] Z. Zhang, Z. Jiang, L. Xu, H. Hao, R. Wang, Multiple-choice questions are eficient and robust llm evaluators, arXiv preprint arXiv:2405.11966 (2024). [66] T. T. Chung, L. Liu, M. Yu, D.-Y. Yeung, Divlogiceval: A framework for benchmarking logical reasoning evaluation in large language models, arXiv preprint arXiv:2509.15587 (2025). [67] M. Sap, H. Rashkin, D. Chen, R. LeBras, Y. Choi, Socialiqa: Commonsense reasoning about social interactions, arXiv preprint arXiv:1904.09728 (2019). [68] D. N. Manh, T. P. Chau, N. Le Hai, T. T. Doan, N. V. Nguyen, Q. Pham, N. D. Bui, Codemmlu: A multi-task benchmark for assessing code understanding capabilities of codellms, CoRR (2024).