Many physical systems in the real world exhibit complex behavior, making it dificult to identify their dynamics. To overcome this problem, prototypes are created to provide a better insight into the behavior of such systems. However, building these prototypes using traditional methods is often slow due to the computational intensity required to accurately capture the detailed complexities within these systems and the large parameter space that needs to be explored. Data-driven approaches, such as machine learning and deep learning frameworks, have the potential to significantly speed up this process by generalizing the model. In this study, we explore the use of such an approach to understand the complex and non-linear behavior of dynamical systems, using resistive random access memory (ReRAM) devices as a case study. It is important to emphasize that modeling ReRAM is only one specific example in our wider investigation, which aims to design predictive models for a range of non-linear dynamical systems. Our work attempts to overcome the limitations of traditional research and development by using neural networks to reproduce the complex behavior of ReRAM cells accurately. We introduce a hybrid dual-input neural network architecture (HDiNN), equipped with a custom loss function, to capture both spatial and temporal patterns, improving the predictability of cell behavior under diferent conditions. This involves integrating important factors such as material properties, device geometry, and electrical interactions into our model to explain the complexities of ReRAM technology. However, our ambitions extend far beyond ReRAM to develop methods to help create innovative, durable solutions in various fields. This study highlights the impact of predictive modeling in advancing materials science and demonstrates the transformative potential of neural networks in improving the design and optimization of future technologies.
the exploration of a wide range of parameters and provide insights that guide experimental and design eforts.
Many physical systems in the real world, from natural Although simulations are useful, they can still be
timephenomena to engineered materials, show complex be- consuming, which can create a bottleneck in the research
havior. This complexity presents significant challenges, and development process. Therefore, it would be
bensuch as the dificulty of predicting system responses un- eficial to speed up this process, which can be achieved
der diferent conditions, the need for extensive compu- through the use of data-driven models such as machine
tational resources to model these behaviors accurately, learning algorithms. With such models, it is possible to
and the challenge of translating theoretical models into reduce both simulation time and the number of
experipractical applications. Efective modeling of these sys- ments required by accurately predicting the behavior of
tems requires a deep understanding of the behaviors these complex systems. This would make the research
themselves and how they can be replicated and analyzed process more eficient and increase the speed with which
in a predictive framework[
Attribution 4.0 International (CC BY 4.0).
Biolek (VTEAM) variant are characterized by their broad coverage and flexibility, being able to model a wide range of memristor behavior by manipulating various parameters [13, 14]. The Generalised Memristor Model takes this lfexibility even further by considering a wide range of behaviors and allowing the inclusion of device-specific features [15]. However, the complexity of these models requires significant computational efort and a deep understanding of device parameters to achieve accurate simulations [16, 17].
ReRAM is a class of non-volatile memory that uses the
switching properties of materials to enable data storage
and retrieval. The switching property refers to the ability
of a material within the ReRAM structure to change its
electrical resistance between a high resistance state (HRS)
and a low resistance state (LRS) in response to an applied
voltage [
Accurate model of ReRAM is essential for predicting the behaviour of these devices under various conditions, 3. Data-driven Model for ReRAM enabling engineers and researchers to simulate and analyse the efects of material properties, device geometry, Behavior Prediction electrical contacts and applied voltages on device functionality [8, 9] and in general to improve understanding The use of neural networks accelerates the simulation of and ability to efectively predicting memristive device ReRAM devices by recognizing relevant features and efbehavior. fectively representing the complex, non-linear dynamics
Many models have been proposed to improve our un- inherent in these devices. Our research aims to predict derstanding and ability to efectively predict memristor the behavior of ReRAM devices, considering a range of behavior. These models range from simple representa- factors such as material properties, device structure, elections to complex frameworks that capture the complex trical contacts, and the progression of external voltage dynamics of memristor function [11, 12, 13, 14, 15]. The over time. An extensive list of discrete parameters incorlinear ion drift model is a fundamental approach to the porated into the model is presented in Table 1. simulation of memristive devices and it is valued for Recent studies have highlighted the potential of simplicity and ease of use. However, its tendency to physics-informed neural networks (PINNs) in accurately oversimplify the complexity of memristor operation and predicting the behavior of memristive devices and resisneglect of nonlinear dynamics limits its usefulness for tive random access memory (ReRAM) [18, 19]. These in-depth analysis [11]. At the other end of the complexity methods combine traditional physics-based models with spectrum is the Simmons tunnel barrier model, which is neural network techniques to efectively simulate and notable for its integration of quantum mechanical princi- forecast device behavior. Similarly, Fan et al. [20, 21] ples via tunneling efects, providing accurate simulations introduced graph-based neural networks to capture the particularly suited to thin film memristor applications intricate details of semiconductor devices, including ma[12]. Meanwhile, models such as the Team model and its terial properties, device characteristics, and spatial relationships. While promising, these approaches would benefit from more extensive experimental validation and model used in this study describes a ReRAM cell based on comprehensive dataset descriptions to fully demonstrate the presence of mobile charged vacancies in the memristheir efectiveness. tive material. It solves a set of coupled partial diferential
To address these complex challenges, we present a equations to capture the detailed physical mechanisms
hybrid dual-input neural network (HDiNN) architec- that lead to hysteresis in the I-V characteristics of
memture. This framework combines the strengths of con- ristive devices. The charge transport model is already
volutional neural networks (CNNs), recurrent neural net- validated by measurements [10].
works (RNNs), specifically LSTM layers [ 22], and dense
networks [23, 24, 25]. RNNs are used in combination 4.1. Hybrid Neural Network Architecture
with CNNs to unravel spatial and temporal patterns in
sequential data, while CNNs extract features from discrete At the heart of our approach is the development of
data points, followed by dense networks to assimilate a hybrid neural network architecture. This
architeccontextual information [26, 22]. By embedding these ex- ture uniquely combines the sequential data processing
tensive parameters into our predictive model, we aim strength of CNNs and RNNs with the skillful parameter
to enhance the understanding of their collective impact handling and feature extraction typical of CNNs followed
on the behavior of ReRAM devices. Our methodology is by dense networks [
model to aid the simulation and optimization of ReRAM cells. The basis for the training and validation of our model is a comprehensive dataset derived from a detailed physical charge transport model[10]. The computational series data.
The dense network segment is specifically used to
manage discrete parameters, such as material properties
and geometric configurations, which are critical in
inlfuencing device performance. The outputs from both
the CNN-RNN and CNN-dense network segments are
merged to create a cohesive representation that
encapsulates both temporal dynamics and specific parameter
information. This aggregated feature set is then processed
through additional dense layers, ultimately leading to an
output layer specifically designed to predict the desired
current (I) response of a memristive device under varying
operating conditions.
4.2. Loss Function
In this work, we introduce a composite loss function,
designed to optimize model performance by minimizing
prediction error, ensuring trend accuracy, prioritizing
critical points, and being robust to outliers. The loss
function consists of several components, each addressing
diferent aspects of prediction fidelity. Below, we
mathematically describe each component and their integration
into the composite loss [
Mean Squared Error (MSE): The MSE component quantifies the average of the squares of the errors between the predicted values () and the actual values (). It is defined as:
MSE = 1 ∑︁( − =1 ) 2 (1)
of change between consecutive predictions and actual values, emphasizing the importance of capturing trends. It is computed as:
Gradient Error = 1 − 1
∑︁ (︀ |+1 − | − 1 =1 −| +1 − |︀) 2 (2)
weight to errors at peak points, where the actual data exhibits significant changes. Peaks are identified where the gradient of the actual data exceeds a defined multiple of its maximum value. The Peak MSE loss is defined as: Peak MSE = 1 ∑︁ · ( − )2· peak =1 where is the weight assigned to peak points, and is a binary indicator that equals 1 for data points identified as peaks and 0 otherwise. The formulation ensures that errors at peak points are amplified by the weight .
The final composite loss function integrates the above components with weighting factors to balance their contributions. It is defined as:
Combined Loss = · MSE + (1 − ) · Gradient Error (3) + Peak MSE (4) where is a weighting factor that balances the contribution of MSE and Gradient Error.
diferences between predicted values and ground truth ˆ. We also use the mean absolute error (MAE), to measure average absolute diference between and ˆ.
In this study, we used a dataset comprising 1000
simulations derived from the physical charge transport model
as described in Section 4. The dataset includes 16 model
parameters as inputs, a sequence of implemented volt- 6. Results and discussion
ages, and the corresponding current as outputs [10]. To
generate the dataset, the set of input parameters were The hybrid dual-input neural network (HDiNN)
develsystematically varied over a physically reasonable range, oped to estimate the behavior of ReRAM cells gives
enas shown in Table 1 [
To manage computational demands and optimize pro- its ability to mimic the current values of ReRAM devices. cessing time, we downsampled the time-series data by The current predictions were found to be in close agreerandomly selecting 20 % of the samples to retain, re- ment with both simulation results and experimental data. moving the remaining samples. This downsampling step It is important to note that since our model was trained ensures the manageability of the dataset without com- exclusively on simulation data rather than actual experipromising its diversity. mental measurements, it does not surpass the accuracy
The dataset was divided into a training set (70 % of of the simulation data. The main purpose of comparing the total data) and a test set (30 % of the total data). The the experimental data with the models predictions and training set was used to fit the network parameters and simulations is to validate the quality of the simulations generate an accurate forecasting model, while the test used in the training process. An additional consideration set was used to evaluate the performance of the models. in our research is the exclusion of real-world measure
Normalization was applied to all input parameters, ments, largely because many of the parameters outlined scaling them to a uniform range between 0 and 1 to fa- in Table 1 are not easily accessible in experimental obcilitate efective training. The model’s output was the servations, whereas they can be precisely controlled and prediction of resistance based on the given input param- recorded in simulations. eter set and sequence of voltage. Each set of experiments Another reason for using a data-driven hybrid neuwas run five times to account for the stochastic nature ral network approach is computational time eficiency. of neural network training. While a single simulation run can take approximately
To evaluate the accuracy of our model, we compared 4 hours, the total training time for the neural network its predictions with the dataset generated from the phys- is typically less than 1 hour. This significant diference ical charge transport model using standard regression in time eficiency highlights the advantage of using our metrics. The mean squared error (MSE), for a dataset of HDiNN model, especially in scenarios where rapid iteralength is calculated by measuring the average squared tion and model refinement are critical.
In quantitative terms, HDiNN designed to predict the range of nonlinear dynamical systems in various discicurrent behavior in ReRAM cells registered MSE of 0.002 plines. We will explore the potential of neural networks and MAE of 0.0029. For a more comprehensive under- to uncover the complexity of these systems. standing of the prediction quality, we evaluated the variance of the MSE for each prediction relative to the average MSE, as shown in Figure 2. Furthermore, to quali- Funding tatively show the accuracy of the model, we examined two diferent case studies where the model predictions, The research is funded by the Carl-Zeiss Foundation via the simulated data, and the actual measured data are the Project Memwerk, the Deutsche Forschungsgemeincompared, as shown in Figure 3. We evaluated the per- schaft (DFG, German Research Foundation)– Project-ID formance of our model over diferent prediction lengths 434434223 – SFB 1461. and dataset sizes (Figure 4.a,b). Despite a slight increase in MSE with longer prediction spans, the models main- References tained consistent accuracy, indicating their robustness to diferent simulation scales(Figure 4.a). Furthermore, our results revealed a positive correlation between the size of the training dataset and prediction accuracy, highlighting the importance of comprehensive training data in the development of robust memristor models (Figure 4.b).
The results of the model presented are promising, but there are several areas for future research to improve the potential of ReRAM. Despite progress, ReRAM technologies still face challenges such as inconsistent resistance conditions, limited read/write longevity, and the search for a standardized switching mechanism. Overcoming these obstacles is essential to meet the demanding reliability standards needed to bring this revolutionary data storage technology to the mass market.
For future work, our research aims to extend the applicability of our data-driven models beyond the domain of resistive random access memory devices to a broader IEEE 106 (2018) 260–285. URL: http://ieeexplore. of electronics based on memristive systems, Naieee.org/document/8267253/. doi:10.1109/JPROC. ture Electronics 1 (2018) 22–29. URL: https://api. 2018.2790840. semanticscholar.org/CorpusID:187510377. [6] S.-J. Li, B.-Y. Dong, B. Wang, Y. Li, H.-J. Sun, Y.-H. [17] C. Li, D. Belkin, Y. Li, P. Yan, M. Hu, N. Ge, H. Jiang, He, N. Xu, X.-S. Miao, Alleviating conductance E. Montgomery, P. Lin, Z. Wang, W. Song, J. P. Stranonlinearity via pulse shape designs in taox mem- chan, M. D. Barnell, Q. Wu, R. S. Williams, J. J. ristive synapses, IEEE Transactions on Electron De- Yang, Q. Xia, Eficient and self-adaptive in-situ vices 66 (2019) 810–813. doi:10.1109/TED.2018. learning in multilayer memristor neural networks, 2876065. Nature Communications 9 (2018). URL: https://api. [7] T.-C. Chang, K.-C. Chang, T.-M. Tsai, T.-J. Chu, S. M. semanticscholar.org/CorpusID:256640180.
Sze, Resistance random access memory, Materials [18] Y. Lee, K. Kim, J. Lee, A compact memristor model Today 19 (2016) 254–264. doi:https://doi.org/ based on physics-informed neural networks, Mi10.1016/j.mattod.2015.11.009. cromachines 15 (2024) 253. [8] R. Dittmann, S. Menzel, R. Waser, Nanoionic [19] Y. Sha, J. Lan, Y. Li, Q. Chen, A physicsmemristive phenomena in metal oxides: informed recurrent neural network for the valence change mechanism 70 (2021) rram modeling, Electronics 12 (2023). URL: 155–349. URL: https://www.tandfonline. https://www.mdpi.com/2079-9292/12/13/2906. com/doi/full/10.1080/00018732.2022.2084006. doi:10.3390/electronics12132906. doi:10.1080/00018732.2022.2084006. [20] G. Fan, K. L. Low, Physics-integrated machine [9] M. A. Zidan, J. P. Strachan, W. D. Lu, The future learning for eficient design and optimization of of electronics based on memristive systems, a nanoscale carbon nanotube field-efect transistor, Nature Electronics 1 (2018) 22–29. URL: https: ECS Journal of Solid State Science and Technol//www.nature.com/articles/s41928-017-0006-8. ogy 12 (2023) 091005. doi:10.1149/2162-8777/ doi:10.1038/s41928-017-0006-8. acfb38. [10] B. Spetzler, D. Abdel, F. Schwierz, M. Ziegler, [21] G. Fan, L. Shao, K. L. Low, Revolutionizing tcad P. Farrell, The role of vacancy dynamics in simulations with universal device encoding and two-dimensional memristive devices, Ad- graph attention networks, arXiv (2023). doi:10. vanced Electronic Materials (2023) 2300635. URL: 48550/ARXIV.2308.11624. https://onlinelibrary.wiley.com/doi/10.1002/aelm. [22] S. Hochreiter, J. Schmidhuber, Long short-term 202300635. doi:10.1002/aelm.202300635. memory, Neural computation 9 (1997) 1735–80. [11] D. B. Strukov, J. L. Borghetti, R. S. Williams, doi:10.1162/neco.1997.9.8.1735.
Coupled ionic and electronic transport model [23] D. P. Kingma, J. Ba, Adam: A method for stochasof thin-film semiconductor memristive be- tic optimization, arXiv preprint arXiv:1412.6980 havior, Small 5 (2009) 1058–1063. URL: (2014). https://onlinelibrary.wiley.com/doi/10.1002/ [24] Y. LeCun, et al., Generalization and network design smll.200801323. doi:10.1002/smll.200801323. strategies, Connectionism in perspective 19 (1989) [12] M. D. Pickett, D. B. Strukov, J. L. Borghetti, J. J. 18.
Yang, G. S. Snider, D. R. Stewart, R. S. Williams, [25] Y. Bengio, Learning deep architectures for ai, FounSwitching dynamics in titanium dioxide memris- dations 2 (2009) 1–55. doi:10.1561/2200000006. tive devices, Journal of Applied Physics 106 (2009) [26] K. Cho, B. van Merrienboer, C. Gulcehre, D. Bah074508. doi:10.1063/1.3236506. danau, F. Bougares, H. Schwenk, Y. Bengio, [13] S. Kvatinsky, M. Ramadan, E. Friedman, A. Kolodny, Learning phrase representations using rnn Vteam: A general model for voltage-controlled encoder-decoder for statistical machine translamemristors, Circuits and Systems II: Express tion (2014). URL: http://arxiv.org/abs/1406.1078, Briefs, IEEE Transactions on 62 (2015) 786–790. arXiv:1406.1078 [cs, stat].
doi:10.1109/TCSII.2015.2433536. [27] T. Wang, J. Roychowdhury, Well-posed models of [14] S. Kvatinsky, E. Friedman, A. Kolodny, U. Weiser, memristive devices (2016).
Team: Threshold adaptive memristor model, Cir- [28] S. Furber, Large-scale neuromorphic computing syscuits and Systems I: Regular Papers, IEEE Trans- tems 13 (2016) 051001. URL: https://iopscience.iop. actions on 60 (2013) 211–221. doi:10.1109/TCSI. org/article/10.1088/1741-2560/13/5/051001. doi:10. 2012.2215714. 1088/1741-2560/13/5/051001. [15] J. Sun, L. Yao, X. Zhang, Y. Wang, G. Cui, Gen- [29] D. Joksas, A. AlMutairi, O. Lee, M. Cubukcu, eralised mathematical model of memristor, IET A. Lombardo, H. Kurebayashi, A. J. Kenyon, Circuits, Devices & Systems 10 (2016) 244–249. A. Mehonic, Memristive, spintronic, and [16] M. A. Zidan, J. P. Strachan, W. D. Lu, The future 2d-materials-based devices to improve and