The next generation of SNNs, energy efectiveness and memory optimisation

Abstract

Modern artificial intelligence (AI) systems, predominantly based on traditional artificial neural networks (ANNs), face fundamental limitations in energy eficiency, adaptability, and biological plausibility. These challenges stem from high computational costs, rigid learning paradigms, and the inability to perform continuous adaptation in real-time environments. In this work, we analyse these constraints, emphasising the high energy consumption of ANNs and their lack of online learning capabilities, which hinder flexibility and scalability in autonomous applications. To address these issues, we propose a neuro-inspired approach that integrates memristive spiking neural networks (SNNs) with biologically relevant mechanisms such as sleep-phase memory consolidation. Memristive hardware enables energy-eficient in-memory computation, while SNNs facilitate event-driven processing and synaptic plasticity, reducing power consumption and enhancing learning eficiency. Additionally, sleep-inspired consolidation processes, particularly those modelled after hippocampal replay, ofer a mechanism for optimising memory retention and adaptation over time. By leveraging these principles, we outline a path toward next-generation AI architectures that are both energy-eficient and dynamically adaptable, crucial for applications in autonomous robotics and edge AI systems. Our findings suggest that spiking neuromorphic solutions, when combined with biologically inspired learning mechanisms, could pave the way for a more optimal, self-sustaining AI paradigm that is less reliant on energy-intensive training and retraining cycles.

1. Introduction

Artificial Neural Networks (ANNs) have driven remarkable advances in machine learning, enabling breakthroughs in image recognition, natural language processing, and game-playing agents. However, these successes come with significant limitations that hinder the eficiency, adaptability, and biological realism of modern ANNs. Researchers have increasingly noted challenges such as the enormous energy demands of training large-scale models, the reliance on biologically implausible learning algorithms, and the inability of most networks to learn continuously in changing environments [ 1, 2 ]. These issues have practical consequences: from hefty carbon footprints and economic costs to brittle AI systems that cannot adapt on the fly or retain old knowledge[ 3, 4 ].

This article provides an in-depth analysis of the major limitations of contemporary ANNs, structured around six core issues: (1) high energy consumption, (2) ineficiency and biological implausibility of the backpropagation algorithm, (3) lack of online learning for real-time adaptability, (4) absence of neuromodulatory mechanisms for context-sensitive learning, (5) frozen weights leading absence of online learning, and (6) limited parallelism relative to biological brains. We contrast these shortcomings with principles from biological neural systems and discuss real-world consequences [ 5, 6 ].

2. High Energy Consumption in Large-Scale ANNs

Modern networks with millions or billions of parameters require significant computational resources for training and inference. Therefore, one of the most pressing limitations of modern ANNs is their

CEUR

ceur-ws.org high energy consumption, especially in large-scale deep-learning models. For example, training a single large natural language processing model can consume hundreds of megawatt-hours of electricity [ 7, 8 ], leading to economic and environmental consequences. For instance, OpenAI’s GPT-3 (175 billion parameters) reportedly required an estimated $4.6 million worth of computing and 355 GPU-years of training time [ 8 ]. Training and running large models are often only feasible for organisations with massive computing infrastructure, raising concerns about the financial costs and accessibility of AI research. High electricity consumption is also often linked to carbon-intensive energy sources, especially if data centres do not use renewable energy. For example, training a large Transformer-based model with neural architecture search resulted in approximately 626,000 pounds of 2 emissions [ 9 ], which can indirectly impact global climate change and greenhouse gas concentrations.

Currently, several strategies are being explored to curb the exponential rise in energy demand for AI training: improving hardware [ 10 ], or using sparsely activated neural networks, where only a portion of the parameters are used for any given input [ 11 ]. An alternative approach to solving high energy consumption problems could be the use of a new neuromorphic architecture that does not require constant updates of all synaptic weights but operates locally and selectively. This approach is similar to the way the nervous system works, which demonstrates exceptional energy eficiency in nature. 80–100 billion neurons in the human brain perform trillions of operations per second on roughly 20 watts of power [ 12 ]. Performing similar operations on conventional computer hardware would require energy comparable to the output of a small power plant [ 13 ].

3. Ineficiency and Biological Implausibility of Backpropagation

The backpropagation algorithm has demonstrated high efectiveness as an engineering tool; however, it faces several critical limitations, including computational ineficiency, sensitivity to changes in data, catastrophic forgetting, and long training times. The algorithm requires global synchrony and weight symmetry, which leads to the weight transport problem, reducing the flexibility and eficiency of the algorithm [ 14 ]. Additionally, backpropagation requires significant computational resources to store and update gradients, which makes scaling the algorithm to large networks challenging, especially when training with long data sequences or deep models. As credit assignment paths grow longer, the algorithm encounters the well-known vanishing and exploding gradient problems [ 15, 16 ].

In biological neural networks, we do not face these issues information exchange and learning occur locally, at the synaptic area level, without the need for global synchrony and weight symmetry. This has led to the development of several bio-inspired algorithms, such as Hebbian learning [ 17 ], target propagation [ 18 ], which attempt to distribute credit assignment in a more biologically plausible manner, reducing both computational load and improving speed and eficiency. Although these methods are still under development, they highlight that, despite their successes, backpropagation is not a universal learning principle. It works well on digital computers but faces scalability issues and cannot be used on hardware that is not a traditional von Neumann computer.

4. Lack of Online Learning and Real-Time Adaptability

Another significant limitation of most ANN implementations is the lack of online learning capabilities. In practice, the vast majority of deep neural networks are trained in an ofline, batch mode: the model is optimised on a static training dataset (often with multiple passes over the data in epochs), and once training is complete, the weights are fixed during deployment [ 1 ]. When there is a need to update the model, retraining or fine-tuning it on newly accumulated data is a time-consuming, computationally intensive process that is typically done ofline rather than instantly. As a result, there is a delay in adaptability the model may remain outdated between retraining cycles, leading to errors or suboptimal decisions during that time.

While this approach is efective for static tasks, it severely limits adaptability in dynamic environments where data distributions can shift. For example, a vision model trained on one set of lighting conditions may fail when the lighting changes [19], a language model may not understand new slang or terminology that emerged after its training data was collected, and a recommendation system may fail to catch a sudden shift in user behaviour until it is retrained much later. These shortcomings highlight the need for the implementation of online learning mechanisms or techniques for streaming data adaptation in ANNs to enable real-time learning [20, 21].

However, introducing online learning, i.e., updating weights with each new data sample or small batch, is practically incompatible with the classical approach to ANNs. Using backpropagation, in this case, may enhance issues such as catastrophic forgetting and lead to instability if the data stream has not been carefully normalized or if learning rates are too high. Scalability issues also arise: performing continuous gradient descent on streaming data means that the computational cost of training is never ”finished,” which is especially prohibitive for large models.

5. Limited Parallelism and Architectural Diferences Compared to

Biological Nervous System

In a typical ANN implementation, synaptic weights are stored in of-chip or of-core memory, and at each layer or operation, those weights must be fetched, applied to the data, and then the results written back. This constant shuttling of data between the memory and the processor is ineficient and becomes a dominant cost for large models[22]. As an IBM research report explains, “the limiting factor isn’t that the processor is too slow, but that moving data back and forth between memory and computing takes too long and uses too much energy”, and this is a fundamental limitation of conventional architectures[22]. By contrast, the brain co-locates memory and processing: synapses (which store connection strengths) are part of the neuron’s structure that also performs computation (integration of inputs). Thus, information processing in brains doesn’t sufer from a large global memory bandwidth bottleneck – neurons only communicate with other neurons they are connected to, and there is no single bus shuttling all data around. This distributed, in-memory computing nature of the brain is a major factor in its eficiency [ 22]. Neuromorphic engineering eforts are trying to emulate this by designing chips where memory (weights) is physically embedded alongside computation units, minimising data movement.

6. Memristive Approaches and Solutions

Conventional computing architectures sufer from high energy consumption due to the separation of memory and processing units (the von Neumann bottleneck). In contrast, the human brain performs computations directly within memory (at synapses) and achieves incredible energy eficiency using only about 20 W, it outperforms computers by several orders of magnitude [23, 24]. One possible solution is the use of memristors, the electronic components that change resistance depending on the current passing through them. Memristors are an electronic analog of synapses: they can operate in spiking mode, and according to some estimates, the dynamics of memristors can approach the biological dynamics of neurons [25]. This allows for energy eficiency similar to mammalian nervous systems, with ultra-low power consumption (on the order of femtojoules per event) [24]. For example, memristive synapses operating in a stable low-resistance mode have demonstrated energy consumption up to ∼ 1, 000, 000 times lower in neural network tasks compared to GPU-based systems [ 7 ].

6.1. On-chip Plasticity and Architectural Optimisations

A key advantage of neuromorphic memristive hardware is its massive parallelism and high connectivity, comparable to biological neural circuits. In the brain, each neuron connects to ∼ 10, 000 others via synapses [24], and neuromorphic memristor arrays can achieve similarly dense fan-in/fan-out. Highdensity integration (down to nanometre scales) further enables network scales to reach millions of devices [23]. The other promising direction is building neuromorphic systems of a brain-region scale; for example, memristor-based chips with > 105 synapses on-chip have been demonstrated, and large neuromorphic platforms now reach 108–109 spiking neurons by tiling many cores [24]. Although memristive hardware is still catching up to achieve the 106 − 109 neurons and 1010 synapses numbers of mammalian regions like the hippocampus, its 3D integration and nanoscale connectivity could provide interesting parallelism. By coping main structures of the brain’s architecture of distributed, concurrent processing, memristive neuromorphic systems can scale up neural networks while maintaining real-time performance.

Each memristor’s resistance can be modulated by local voltage and spike timing, naturally implementing STDP in hardware[26, 27]. In a 2019 demonstration, hybrid CMOS/memristor spiking networks with memristive synapses achieved fast one-shot learning of streaming data, highlighting the potential for continuous, lifelong learning in hardware [27]. We consider online learning as crucial for autonomous agents and robotics: a memristive SNN can adjust to new environments or changing goals during operation via local synaptic updates (e.g. STDP or reward-modulated STDP) rather than requiring full re-training as in traditional ANNs. Such local learning in memristor arrays has been validated in simulations and prototypes, showing improved performance as the network self-refines with experience [ 28, 10 ]

To maximize memristive neuromorphic performance, researchers are optimising both memristive device materials and network architectures. On the device side, engineering memristors with STDP learning with multiple intermediate states (as opposed to abrupt digital switching) improves their ability to represent synaptic weights smoothly and reliably [ 10 ]. Recent designs integrate memristor arrays for synapses with spiking neuron circuits and even memristive axon-dendrite trees, reproducing the compartmentalized processing of biological neurons [24]. Self-organising and self-learning memristive nanowire networks – could be used as a complex memristive substrate. The nanowire networks provide a densely interconnected web of memristive junctions a-la synapses, implementing hardware reservoircomputing principles [ 10 ]. These online architectural optimisations aim to approach the scale and resilience of brain microcircuits while preserving low-energy operation.

6.2. Neuromorphic Robots with Memristive SNNs

From the autonomous robotics perspective, the integration of memristive networks with SNNs paves the way for robots with brain-like control, online adaptability, and flexibility. Recent work demonstrated that a memristor-based “memristive nervous system” in a humanoid robot yields high energy eficiency and bionic sensory processing akin to biological nervous systems[25, 29]. Memristive sensory neurons can transduce analog signals from sensors directly into electrical spikes[24], processing information on-site with minimal processing computing cost. The possible result is neuromorphic robotics capable of complex perception-SNN processing-action loops under strict power budgets that traditional robot controllers cannot match [25]. Memristive SNN controllers have demonstrated real-time learning and adaptation, showing promise for agile, autonomous robots with nervous system-level eficiency [ 30].

7. Memory Optimisation in Neuromorphic Systems

In this work, we propose a novel memory optimisation through a consolidation process (Fig 1). Firstly, the forward replay enhances the route as experienced, whereas reverse replay (often occurring at the goal location) runs the sequence backwards [31]. These replay events are time-compressed (lasting ∼ 100 ms) and coincide with the hippocampus’s sharp-wave ripple oscillations (∼ 150–200 Hz). Such bidirectional and flexible reactivation of memories is believed to strengthen the neural representation of space, supporting long-term memory storage and informed planning.

7.1. Memory Consolidation Stages (Bio-Inspired Model)

Wakeful Encoding of Routes. During active exploration, dedicated neurons in the hippocampal formation encode the animal’s trajectory (Fig. 1). Place cells (hippocampal pyramidal neurons) fire at 0 START

6 END 1 specific locations in an environment, efectively mapping out “place fields” along a route[ 31]. At the same time, head-direction cells (found in connected regions like the entorhinal cortex and subiculum) ifre when the animal faces a particular direction. Together, these cells create a cognitive map of the route during the wake phase. In a neuromorphic system, a similar mechanism can be implemented with spiking memristive neurons that respond to location and orientation features, providing an internal representation of paths a robot traverses. Each segment of a route would activate a unique combination of “place-cell” and “head-direction” spiking units, encoding the spatial trajectory in memory.

Sleep Memory Optimisation. The hardware memristive implementation could use, a randomly organised nanoscale networks naturally replicate aspects of neural morphology: they form dense “axon-dendritic” meshes with synapse-like junctions that can store analog weights [32, 33, 30]. Mimicking hippocampal processes – using place-cell-like encodings, neuromodulatory reward signals, and high-frequency reactivation (replay) – provides a framework for on-line learning with of-line consolidation [34, 35].

During sleep bidirectional replay, memory optimisation is implemented through a gradient descentlike neuronal approach. This process involves the high-frequency reactivation of previously encoded sequences, particularly during sharp wave-ripple (SWR) events (∼ 200 ). During replay, neuronal activity propagates from the final step (e.g., step 6) back through the sequence ( 5 → 4 → 3 → 2 → 1 → 0), reinforcing key synaptic connections. This stochastic propagation occurs through a dynamically organised synaptic landscape, allowing for the spontaneous exploration of alternative pathways. A crucial aspect of this process is the selective strengthening of synapses that contribute to a more eficient representation of the experience. For example, if an indirect synapse between neuron 1 and neuron 5 (1 → 5) is activated but was not part of the original forward encoding, its reinforcement suggests a reconfiguration of the memory trace. This reorganisation efectively optimises the stored representation, potentially leading to more eficient recall or problem-solving upon future retrieval. Bidirectional replay thus serves as a neural computational mechanism that refines stored experiences, not just by consolidating past sequences but also by iteratively optimising connectivity to facilitate improved future performance. The concept of simultaneous bidirectional replay suggests that when a route is being replayed, it does not necessarily follow a unidirectional trajectory but can occur in both forward and backwards directions at the same time. This is particularly evident in cases where nodes at opposite ends of a path, such as 0 and 5, exhibit high-frequency oscillations simultaneously (Fig. 1). This coactivation pattern implies that neural circuits involved in memory processing or path optimisation might leverage bidirectional information flow to enhance learning eficiency. By simultaneously replaying sequences in both directions, the system can more rapidly converge on an optimal network connectivity structure. This reduces the number of iterative adjustments needed, thereby accelerating the process of discovering the most eficient connections within the network. The attached diagram, which represents a structured path from a starting point (0) to an endpoint (6) via intermediate nodes, visually supports this notion by depicting the potential bidirectional flow of activity across the network.

An artificial system with ”sleeping” memory optimisation is highly relevant to neuromorphic robots and AI: during active phases, it learns from the environment (forming new memories), and during rest phases, it autonomously replays and strengthens those memories. By consolidating important information and pruning or downscaling the rest, the system optimizes its memory usage and improves recall reliability. This brain-inspired two-stage learning (wake/sleep) could prevent catastrophic forgetting in neuromorphic chips, enabling long-term memory storage even with continuous learning. Looking ahead, the convergence of memristive hardware (for dense, low-power memory) with biologically grounded memory algorithms may yield neuromorphic processors capable of human-like memory consolidation – eficiently encoding experiences and consolidating them into lasting knowledge. Such systems would represent a significant step toward brain-inspired cognition in practical autonomous machines.

8. Conclusion

The next generation of Spiking Neural Networks (SNNs) holds great promise for bridging the gap between artificial intelligence and biological intelligence. Throughout this study, we have highlighted the limitations of traditional Artificial Neural Networks (ANNs), including their high energy consumption, reliance on biologically implausible learning algorithms, lack of real-time adaptability, and vulnerability to catastrophic forgetting. By contrast, SNNs particularly when integrated with neuromorphic hardware and memristive technologies ofer a more eficient, scalable, and biologically inspired alternative.

The adoption of memristive architectures allows for in-memory computing, reducing energy costs and overcoming the von Neumann bottleneck. Furthermore, mechanisms such as synaptic plasticity, neuromodulatory influences, and sleep-inspired memory consolidation strategies provide a pathway for lifelong learning and adaptability in neuromorphic systems. The incorporation of bidirectional hippocampal replay in artificial systems also paves the way for more robust memory optimisation, enabling eficient long-term retention and dynamic reconfiguration of learned experiences.

Future research should focus on refining these neuromorphic principles by integrating advanced materials, optimising architectural designs, and further exploring biologically inspired mechanisms for continual learning. By doing so, we can move closer to developing AI systems that match the eficiency, adaptability, and cognitive resilience of biological brains ushering in a new era of energy-eficient, intelligent computation.

9. Acknowledgments

This study was partially funded by the FAIRGROUND project: Artificial and Bio-Inspired Networked Intelligence for Constrained Autonomous Devices Fairground Bando A Cascata A Valere Sul Piano Nazionale Ripresa E Resilienza (PNRR) Missione 4, Istruzione E Ricerca-Componente 2, Dalla Ricerca Allimpresa-Linea Di Inve-Stimento 1.3, Finanziato Dallunione Europea Nextgenerationeu, Progetto Future Artificial Intelligence Fair PE0000013 CUP (Master): J53C22003010006 CUP: J43C24000230007. The author(s) acknowledge(s) the support of the APC central fund of the University of Messina.

Declaration on Generative AI

During the preparation of this work, the author(s) used GPT-4o in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.

References

[1] G. I. Parisi , R. Kemker , J. L. Part , C. Kanan , S. Wermter , Continual Lifelong Learning with Neural Networks: A Review , Neural Networks 113 ( 2019 ) 54 - 71 . ArXiv: 1802 .07569 [cs].

[2] A. E. Takesian , T. K. Hensch , Balancing plasticity/stability across brain development , Progress in Brain Research 207 ( 2013 ) 3 - 34 .

[3] Supermicro , Did you know training a single ai model can emit as much carbon as five cars in their lifetimes? , 2024 . URL: https://www.supermicro.com.

[4] Cybernews , The cost of training ai models is rising exponentially , 2025 . URL: https://www. cybernews.com.

[5] Y. Liu , C. Zhao , Y. Jiang , Y. Fang , F. Chen , Ldd: High-precision training of deep spiking neural network transformers guided by an artificial neural network , Biomimetics 9 ( 2024 ) 413 .

[6] J. C. Whittington , R. Bogacz , An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity , Neural computation 29 ( 2017 ) 1229 - 1262 .

[7] A. Fedorova , N. Jovišić , J. Vallverdù , S. Battistoni , M. Jovičić , M. Medojević , A. Toschev , E. Alshanskaia, M. Talanov , V. Erokhin , Advancing neural networks: Innovations and impacts on energy consumption , Advanced Electronic Materials 10 ( 2024 ) 2400258 .

[8] S. Aquino-Brítez , P. García-Sánchez , A. Ortiz , D. Aquino-Brítez, Towards an energy consumption index for deep learning models: A comparative analysis of architectures, gpus, and measurement tools , Sensors 25 ( 2025 ) 846 .

[9] E. Strubell , A. Ganesh , A. McCallum , Energy and policy considerations for modern deep learning research , in: Proceedings of the AAAI conference on artificial intelligence , volume 34 , 2020 , pp. 13693 - 13696 .

[10] A. Mikhaylov , A. Pimashkin , Y. Pigareva , S. Gerasimova , E. Gryaznov , S. Shchanikov , A. Zuev , M. Talanov , I. Lavrov , V. Demin , et al., Neurohybrid memristive cmos-integrated systems for biosensors and neuroprosthetics , Frontiers in neuroscience 14 ( 2020 ) 358 .

[11] P. Bizopoulos , D. Koutsouris , Sparsely activated networks , IEEE Transactions on Neural Networks and Learning Systems 32 ( 2020 ) 1304 - 1313 .

[12] H. B. Project , Learning brain: make ai more energy- eficient, 2023 . URL: https://www.humanbrainproject.eu/en/follow-hbp/news/2023/09/04/ learning -brain-make-ai-more-energy-efficient/ , accessed: 2025 -03-23.

[13] S. Lee , J. Park , Reconsidering the energy eficiency of spiking neural networks , arXiv ( 2023 ). URL: https://arxiv.org/abs/2409.08290.

[14] D. Balduzzi , H. Vanchinathan , J. Buhmann , Kickback cuts backprop's red-tape: Biologically plausible credit assignment in neural networks , in: Proceedings of the AAAI conference on artificial intelligence , volume 29 , 2015 .

[15] Z. Yu , L. Shen , L. Ding , X. Tian , Y. Chen , D. Tao , Sheared backpropagation for fine-tuning foundation models , in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024 , pp. 5883 - 5892 .

[16] W. Xiao , H. Chen , Q. Liao , T. Poggio , Biologically-plausible learning algorithms can scale to large datasets , arXiv preprint arXiv: 1811 . 03567 ( 2018 ).

[17] S. J. Cooper , Donald o. hebb's synapse and learning rule: a history and commentary , Neuroscience & Biobehavioral Reviews 28 ( 2005 ) 851 - 874 .

[18] Y. Bao , T. Shibuya , I. Sato , R. Kawakami , N. Inoue , Eficient target propagation by deriving