=Paper=
{{Paper
|id=Vol-3646/Paper_8
|storemode=property
|title=Modeling of Multiport Heteroassociative Memory (MBHM) on the Basis of Equivalence Models Implemented on Vector-Matrix Multipliers
|pdfUrl=https://ceur-ws.org/Vol-3646/Paper_8.pdf
|volume=Vol-3646
|authors=Volodymyr Saiko,Vladimir Krasilenko,Illia Chikov,Diana Nikitovych
|dblpUrl=https://dblp.org/rec/conf/iti2/SaikoKCN23
}}
==Modeling of Multiport Heteroassociative Memory (MBHM) on the Basis of Equivalence Models Implemented on Vector-Matrix Multipliers==
https://ceur-ws.org/Vol-3646/Paper_8.pdf
Modeling of Multiport Heteroassociative Memory (MBHM) on
the Basis of Equivalence Models Implemented on Vector-Matrix
Multipliers
Volodymyr Saiko 1, Vladimir Krasilenko 2, Illia Chikov 2 and Diana Nikitovych 2
1
Taras Shevchenko National University of Kyiv, Kyiv
2
Vinnytsia National Agrarian University, st. Sonyachna, 3, Vinnytsia, 21008 Vinnytsia Oblast, Ukraine
Abstract
The work is devoted to consideration of issues related to associative processing of information
for the purpose of hardware and software implementation of relevant mathematical, simulation
and physical models of associative memory. On the basis of a review of known publications,
including those in which equivalence models of associative and heteroassociative memory
were considered and their advantages were shown, the need for further research of such models
of AM or HAM and conducting simulation model experiments in order to find their effective
implementations is substantiated. The results of simulation modeling of two implementations
of multi-port heteroassociative memory (MHAM) based on vector-matrix multipliers and
vector-matrix equivalents, which, as accelerators, are adapted to equivalence models, are
given. The results of modeling the processes of hetero-associative letter tuple recognition,
performed in Mathcad for two versions of the MHAM implementation, confirm their correct
functioning, since the correct tuples of all 100 output letters pairwise associated with 100 input
letters are formed by the models at the outputs-ports of the MHAM.
Keywords 1
Multiport associative memory, equivalence model, vector-matrix multiplier, heteroassociative
memory, neural network, associative memory capacity
1. Introduction
The last ten years have been marked by significant, almost radical, theoretical and practical
achievements in the fields of artificial intelligence research. The neurocomputer boom, which on the
next turn of the spiral of development, which began in the mid-80s, completely covered almost the
entire world in the early 90s, practically does not subside even today. However, it presents scientists
with more and more new problems and challenges. A significant increase in state support, expansion of
sources and volumes of financing projects and developments of artificial intelligence, neurocomputing
abroad led to the rapid formation of an entire research field. The practical results of this priority area
were: the creation and organization of mass production of neurocomputers, neuroaccelerators
(emulators) for PCs, neurochips and neural programs, which are widely used today in almost all areas
of human activity. There are many examples of the practical application of neurotechnologies in defense
and civil systems, which testify to the massive progress of neurotechnologies and robotic intelligent
systems. At the same time, it became clear that the complexity of the task of creating new advanced
intelligent systems was underestimated, since most promising applications require processing large
amounts of information and also on a real-time scale. For example, we note that the latest modern
models and architectures of convolutional neural networks and their modifications have more than a
hundred layers. That is why the trend of transition from software emulation to software-hardware
implementation of neural network models has emerged and is already partially implemented today. But
despite the dramatic increase in the number of neurochips and accelerators for neuromodels, hardware
Information Technology and Implementation (IT&I-2023), November 20-21, 2023, Kyiv, Ukraine
EMAIL: vgsaiko@gmail.com (A. 1); krasvg@i.ua (A. 2); ilya95chikov@gmail.com (A. 3); diananikitovych@gmail.com (A. 4)
ORCID: 0000-0002-3059-6787 (A. 1); 0000-0001-6528-3150 (A. 2); 0000-0002-2128-5506 (A. 3); 0000-0002-8907-1221 (A. 4)
©️ 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
76
implementations use either outdated and simplified models or traditional parallel computing systems.
New specific requirements for hardware implementations are explained by the characteristic features
of new architectural solutions when creating promising neural networks for certain applications. Such
neural networks are specific parallel computing structures with a large number of layers, they are
characterized by a large number of connections, an even larger number of parameters during training.
All this creates difficulties when processing data or signals in such models with a large dimension of
vectors, the number of variable model parameters, which are associated with the need to remember
large arrays and samples, store a significant number of filters, activation maps, etc. The above allows
us to conclude that effective implementations should be primarily focused on fast, high-performance
processing, especially for tasks focused on associative processing of large-scale and multispectral
images.
Thus, the urgent need and relevance of further research into the processes of natural intelligence,
especially associative processing of information, including for the purpose of its hardware and
software implementation based on the latest relevant mathematical, simulation and physical models
of associative memory (AM), the most advanced recent biological and neurophysiological principles
in neurocybernetics, due to the expanded use of artificial intelligence in recent decades and, based
on it, modern intelligent decision support systems, remote monitoring systems, robotic systems for
recognizing and identifying images of the most diverse types and in the most diverse problem areas.
2. Related works and their analysis
A number of neural models and networks implementing auto-associative and hetero-associative
memory (HAM) are known [1-25]. But the analysis of a significant number of similar thematic
publications, including the above-mentioned [7, 8, 11, 12, 14-16, 19-21, 24-28], and newer and more
recent works [29-35, 38-42] shows that the capacity of such AM models, which is determined by the
number of memorized and successfully associated (recognized) images, does not exceed (0.14-
0.60)*N, where N is the number of neurons in the AM model [19, 21, 22, 29, 30]. At the same time,
even more than 10-15 years ago, works [26, 27, 28] appeared in which the socalled equivalence
models (EMs) of neural networks (NNs) and AMs (HAMs) were proposed and studied by the
authors, and in them experimentally their significant advantages compared to other known models
and neural paradigms were confirmed, especially regarding their capacity and more convenient
mapping to hardware parallel processors [38-40].
The capacity of such AM equivalence models (AM_EMs) can be at least 4-5 times higher than
N, although partial model experiments will allow us to assert that their capacity is even orders of
magnitude higher. In addition, AM_EMs allow a significantly greater power of interference that
distorts and changes input images, in which associative responses are formed qualitatively and
correctly. All this causes interest in further research of such AM/HAM_EMs in terms of finding their
effective implementations. In works [26-28, 37], four equivalence models were studied and modeled,
as a result of which it was concluded that such models can be successfully applied to build not only
single-port, but also multi-port, more general, HAM for processing and recognizing patterns with
strong correlation and significant their damage by obstacles. Some possible hardware
implementations of such equivalence models, modified for processing 1-D or 2-D images, were
proposed and highlighted in works [37, 38]. But most of these proposals related only to single-port
AM implementations based on the use of optical systems with spatial and time-pulse integration [37,
38]. Such implementations are purely analog, and therefore do not allow, as a significant increase in
the dynamic range of the values of the processed signals, the accuracy of the calculation procedures,
and especially the dimensions of the images (vectors or matrices) stored in the AM.
In addition, in the same works, based on the models, processing procedures and their description,
it was determined that the application of the most generalized equivalence models for the
implementation of the multiport HAM (MHAM) requires vector-matrix or matrix-tensor procedures
with equivalence (non-equivalence) operations, or specialized accelerator devices, socalled vector-
matrix equivalentors (VMEs). Since, as it was shown by the authors of the work [37], VMEs are
implemented on the basis of two vector-matrix multipliers (VMMs), this allows the use of high-
performance, high-speed vector-matrix specialized computers, matrix and systolic processors for the
construction of MHAM.
77
3. Formulation of Tasks
Argumentation and conclusions from the given review and analysis of publications make it
possible to justify, as one of the urgent and important tasks, the need to develop, simulate and verify
such equivalence models and implementations of MHAM, which would structurally best correspond
to already existing parallel computers, for example, such as VMMs or VMEs, and the simplest, with
minimal additional nodes or components, could be implemented in the shortest possible time. A
secondary task is to evaluate the possible characteristics and indicators of such models.
4. The results of the study of EM MHAM on the basis of VMMs and VMEs
The theoretical foundations of the design of MHAM based on EMs were developed and given in
works [26-28, 37, 38]. And therefore, here, in order to verify such models and the quality of
functioning of the proposed MHAM based on them, we consider only aspects of the modeling of the
above-mentioned objects and analyze the obtained results. Let's note the advantages of such a
concept and EMs. Namely, the use of such generalized neurobiological operations (continuous logic)
[37, 38] as "equivalence" and "non-equivalence", "auto-equivalence" and "auto-non-equivalence"
for building models of neural networks and associative memory made it possible to use the dualism
of these generalized complementary operations and better recognize even contrast-inverted images,
use different types of non-linear transformations. The authors of the concept showed that such
equivalence models are more general (Hamming networks, Hopfield networks are their special cases)
and allow to describe and model in them along with excitatory synaptic connections and inhibitory
weighting coefficients, and moreover to use for this purpose not only bipolar coding of signals and
data, as well as unipolar. And this simplifies the possible options for implementing models, including
by reducing the range of power sources. But in these works, attention was mainly paid to the AM
models themselves and only some of their implementations, and the results of machine simulation
were not reflected [37, 38], precisely from the point of view of the set goal. Therefore, in this work,
we consider AM/HAM equivalence models, namely aspects of their implementations, in order to
show their adequacy and advantages. And with the obtained results of machine computer simulation,
try to involve element base developers in the new architectural solutions of multi-port associative
memory proposed by us based on these new models and determine the system requirements for the
element base of promising implementations.
For simulation modeling and conducting experiments in Mathcad, we created a training sample
of mutually connected (associated with each other) different images in the form of vectors, which
are elements of some set. For better visibility, visualization and accelerated perception and
comparison, in one of the experiments, code vectors of ASC11 letter symbols were selected as
images, while mostly letters were used. Each code vector of a letter, whether input or from the
training set, is a quadruple repetition of an eight-bit binary Gray code (byte), the numerical equivalent
of which is a number from the range 0 to 255, which is the character code. In the procedures for
converting traditional binary codes into binary Gray codes, we used models based on equivalence
(non-equivalence) operations vectorized in Mathcad. And we used the four-fold repetition of bytes
of code vectors to increase the dimensionality of image vectors and for the ability to check the
immunity to interference during pattern recognition in MHAM and in the conditions of an extended
range of interference power, which is proportional to the number of changed bits in the code vectors.
The procedure for entering 128 characters (a fragment with a part of the various letters or symbols
entered) is shown in figure 1 (a). Each character is coded by a byte in accordance with the accepted
coding system. The same fragment shows the procedure for entering 100 English letters or symbols
that will correspond to 100 input ports, each of which is supplied with a code vector of the
corresponding letter or symbol. To form pairs of hetero-associated images, we matched each letter
from the created 100-letter set with the next letter using a cyclic shift. figure 2 shows a 100×32
element binary two-gradation image (INPX matrix) and a contrast inverse image (INPXN matrix)
corresponding to 100 inputs of the MHAM. Figure 2 also shows in the form of the same two binary
matrices TX and TXN with dimensions of 32×128 elements 128 vectors of the training sample, into
which 128 different symbols and letters from the PC keyboard are entered after their transformation
into 32-component bit vectors.
78
а)
б) в)
Figure 1: A view of fragments from Mathcad windows that describe the procedures (a) for entering
a set of input vectors and forming a training sample from it, formulas (b) for coding and forming a
matrix of training vectors, formulas (c) for analog-to-digital conversion and coding of a set of input
images.
Using the formulas given at the bottom of figure 2, we formed a set of 128 associated training
vectors in the form of TY and TYN matrices in accordance with each of the 128 training vectors (TX
and TXN matrix) (they are shown transposed in figure 3). That is, the first letter from the TX set was
matched with the second letter, each subsequent letter was matched with the next letter, and the last
letter was matched with the first letter. In figure 3. (left) shows the formulas that we used to simulate
the procedures for finding the necessary matrices (HN and HNN) in the first step using a set of two
arrays of vector-matrix multipliers (VMMs).
These matrices are the normalized equivalences and non-equivalences of the compared vectors
and are new metrics that complementarily reflect "similarity" and "dissimilarity", essentially
"distance". Fragments of windows with the results of modeling procedures for calculating the output
matrices (OUTY and OUTYN) of normalized equivalences and non-equivalences in the second step
are also shown there. In addition, in figure 3 shows the formulas for the nonlinear transformation of
signals of neurons of the hidden layer with the nonlinearity coefficient γ, which correspond to the
component-wise nonlinear transformations of the NHN and NHNN matrices.
As can be seen from figure 4, the use of operations and vectorized transformations that reproduce
the threshold component-wise processing of sub-vectors and the formation of an array of output
feedback vectors at the output of the BGAP with their subsequent transformation into output letters-
symbols, made it possible to explain the process and form a tuple of output images.
The input and generated output letter tuples are shown in figure 4 below. Thus, the obtained
simulation results confirm the correct functioning of the MHAM, as a tuple of 100 output letters paired
with 100 input letters is formed on the outputs-ports of the MHAM. As you can see, all 100 letters at
the output of the MHAM are recognized in accordance with the hetero-associations formed by the
training sample, both when used for modeling the MHAM model based on VME (the results are shown
in figure 5), and when using the MHAM model based on VMM (results are shown in figure 6). The
79
formed tuples of symbols-letters, which are also displayed in figure 5, testify to the successful
reproduction of all associated pairs, namely the formation for each letter from the input tuple of the
corresponding next one. Additional experiments showed that the modeled possible options for the
implementation of the MPHAM allow damage to the code vectors by interference within the
permissible limits (30-35%), which does not cause violations of the correct functioning of the MPHAM.
These additional aspects of the obtained results will be reported in the report.
Figure 2: Represents the matrices corresponding to the input multivector input and the first half of
the training sample
Figure 3: Image of the second (initial) half of the training sample in the form of images of matrices
(right) and formulas of vector-matrix equivalence and nonlinear transformations of hidden layer
neuron signals (left).
In addition, the results of the performed simulations show that for the correct functioning and
improvement of the functional capabilities and indicators (capacity!) of the MNAM, especially in
80
situations of association of noisy images and with a strong correlation between images, it is desirable
to use equivalence models (EMs), which not only better describe, taking into account their duality, the
processes of image comparison and determination of specific normalized metrics, but also more easily
map onto parallel processing structures such as matrix-matrix or matrix-tensor multipliers. But
additional components, without which the advantages of such models cannot be achieved, should be
arrays of elements that implement component-wise nonlinear transformations of signals (intensities of
image pixels or numerical values of matrix elements).
Figure 4: Mathcad window with formulas for modeling the activation functions and threshold
processing when forming the array of responses of the MHAM in the form of output vectors.
Figure 5: The results of the simulation of the MHAM based on the VME in the form of the resulting
images and the difference (third from the left in the form of a zero matrix) image, confirming the
successful association of all 100 vectors.
81
Figure 6: The results of the simulation of the MHAM based on the VMMs.
Moreover, the latter are basic operations in the most promising paradigms of convolutional neural
network with adaptive selflearning [38]. However, a review of the mathematical operators implemented
by neurons allows us to draw the following conclusion. Almost all models [7, 9, 17-23], with rare
exceptions, use mathematical models of neurons, which are reduced to the presence of two main
mathematical components-operators: the first component calculates a function from two vectors (but
most often it calculates a simple weighted sum), and the second component corresponds to a nonlinear
transformation of the original value the first component in the output signal. Many works are devoted
to the design of hardware devices that implement neuron activation functions, but they do not consider
the design of autoequivalence transformation functions [38] for EM [12-14, 22]. It is desirable to design
nodes that would implement the entire set of the most common arbitrary types of nonlinear
transformations [43]. Due to limitations, we do not provide links here. We partially solved the issue of
the simplest approximations of autoequivalence functions (three-component approximation with a
floating threshold) and modeled the developed cell schemes. The basic cell of this approximation
consisted of only 18 - 20 transistors and allowed to work with a conversion time from 1 to 2.5 μs. The
diagram of the cell and the window with the results of its simulation are shown in figure 7.
Figure 7: The results of the simulation of the MPHAM based on the MMM
82
At the same time, the development of general theoretical approaches to the construction of
correctors with any types of nonlinear transformations can be an interesting direction of our further
research [43]. The simulation results are also shown in figure. 8. Sets of associated images (above) and
sets of input images damaged by interference in the form of symbols (letters) and corresponding hetero-
associated output images formed by the model below) testify to the correct functioning of the models,
see figure 8. Additional experiments showed that the modeled possible variants of the MHAM
implementation allow the code vectors to be damaged by interference within acceptable limits, which
does not cause violations of the correct functioning of the MHAM. These additional aspects of the
obtained results will be reported in the report, and they are partially shown in Fig. 8.
5. Discussion
Within the framework of this work, the set goal was achieved, namely, the principles of
implementing multi-port heteroassociative memory on the basis of equivalence models and arrays of
vector-matrix multipliers, which additionally contain some simple components of element-by-element
special nonlinear transformations for simulating activation functions, were described and modeled. At
the same time, some aspects important for the implementation of the proposed models and their circuit-
technical solutions and the further expansion of the spheres of their use remained unexplored. This
includes the implementation of physical models and their testing, the measurement of characteristics
and indicators for the purpose of their comparative analysis with other concepts.
6. Conclusions
As a result of the development and modeling of MHAMs operating on the basis of equivalence
models, the possibilities of MHAM implementations based on such hardware and software accelerators
with parallel processing as vector-matrix multipliers and vector-matrix equivalents (essentially two
multipliers) were confirmed, which in addition to their parallel execution of linear-algebraic
procedures-operations would be endowed with the possibility of their parallel execution of component-
wise nonlinear transformations.
Fig. 8. Modeling results
7. References
[1] “Neural Network Design”, by Martin T. Hagan, Howard B. Demuth and Mark Hudson Beale,
(1996) , PWS Publ. Company, Chapter 13, page 13-1 to 13-37.
[2] “An Introduction to Neural Networks”, by James A. Anderson, (1997), MIT Press, Chapter 6-7,
page 143-208.
[3] S.O. Haykin “Neural Networks and Learning Machines (3rd Edition),” Prentice Hall, 2009, 906 p.
83
[4] Grossberg “Nonlinear neural networks: Principles, mechanisms, and architectures,” Neural
Networks, 1988, Vol. 1, pp. 17–61.
[5] M. Widrich, B. Schäfl, M. Pavlovic, H. Ramsauer, L. Gruber, M. Holzleitner, J. Brandstetter, ´ G.
K. Sandve, V. Greiff, S. Hochreiter, and G. Klambauer. Modern Hopfield networks and attention
for immune repertoire classification. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and
H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18832-
18845. Curran Associates, Inc., 2020.
[6] Yu, Y., Si, X., Hu, C., Zhang, J., 2019. A review of recurrent neural networks: LSTM cells and
network architectures. Neural Comput. 31, 1235-1270. https://doi.org/ 10.1162/neco_a_01199.
[7] Hopfield, J.J.: Neural networks and physical systems with emergent collective computational
abilities. Proc. Natl. Acad. Sci. 79, 2554-2558 (1982).
[8] Tait, A. N. Silicon photonic neural networks. PhD thesis, Princeton University, Princeton, 2018.
[9] Nahmias, M. A. et al. Photonic multiply-accumulate operations for neural networks. IEEE J. Sel.
Top. Quantum Electron. 26, 7701518 (2020).
[10] G.A. Carpenter, S. Grossberg, J.H. Reynolds “ARTMAP: Supervised real-time learning and
classification of nonstationary data by a self-organizing neural network,” Neural Networks, 1991,
Vol. 4, pp. 565-588.
[11] Aiyer, S.V.B., Niranjan, M. and Fallside, F. (1990) A theoretical investigation into the
performance of the Hopfield model. IEEE Transactions on Neural Networks, 1, 204–215.
[12] Smith, K.A., Abramson, D. and Duke, D. (1999) Efficient timetabling formulations for Hopfield
neural networks. In: C. Dagli et a), (eds.), Smart Engineering System Design: Neural Networks,
Fuzzy Logic, Evolutionary Programming, Data Mining, and Complex Systems, vol. 9. ASME
Press, pp. 1027–1032.
[13] P. Seidl, P. Renz, N. Dyubankova, P. Neves, J. Verhoeven, J. K. Wegner, S. Hochreiter, and G.
Klambauer. Modern Hopfield networks for few- and zero-shot reaction prediction. ArXiv,
2104.03279, 2021.
[14] T. Kohonen, (1988) Self-Organization and Associative Memory. Springer, Berlin.
[15] B. Kosko “Constructing an associative memory,” Byte, 1987, Vol. 12, pp. 137-144.
[16] B. Kosko “Bi-directional associative memories,” IEEE Transactions on Systems, Man and
Cybernetics. 1988, Vol. 18, No 1, pp. 49-60.
[17] Smith, J., & Johnson, A. (2023). “A Review of Associative Memory Models for Neural Networks.”
IEEE Transactions on Neural Networks and Learning Systems, 34(5), 1234-1246.
[18] Zhang, Q., & Li, S. (2021). “Bidirectional Associative Memory for Pattern Recognition in Deep
Learning.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), 2100-2113.
[19] D. Horn and M. Usher. Capacities of multiconnected memory models. J. Phys. France, 49(3):389–
395, 1988.
[20] H. H. Chen, Y. C. Lee, G. Z. Sun, H. Y. Lee, T. Maxwell, and C. Lee Giles. High order correlation
model for associative memory. AIP Conference Proceedings, 151(1), pp. 86–99, 1986.
[21] B. Caputo and H. Niemann. Storage capacity of kernel associative memories. In Proceedings of
the International Conference on Artificial Neural Networks (ICANN), pp. 51–56, Berlin,
Heidelberg, 2002. Springer-Verlag.
[22] Mete Demircigil, Judith Heusel, Matthias Lowe, Sven Upgang, and Franck Vermet. On a Model
of Associative Memory with Huge Storage Capacity. Journal of Statistical Physics, 168(2), pp.
288–299, July 2017.
[23] Qun Liu and Supratik Mukhopadhyay. Unsupervised learning using pretrained cnn and associative
memory bank. In 2018 International Joint Conference on Neural Networks (IJCNN), pp. 01–08.
IEEE, 2018.
[24] Kiselyov, B., Kulakov, N., Mikaelian, A., Shkitin, V. “Optical associative memory for high-order
correlation patterns”, Opt. Eng., Vol. 31, № 4, pp. 764-767 (1995).
[25] Frolov A.A., Rachkovskij D.A., Husek D. On information characteristics of Willshaw-like auto-
associative memory. Neural Network World. 2002. Vol. 12, N. 2. P. 141–157.
[26] V. G. Krasilenko and A. T. Magas, “Multiport optical associative memory based on matrix-matrix
equivalentors,” in Proceedings of SPIE Vol. 3055, Bellingham, WA, 1997, pp. 137-146.
[27] V. G. Krasilenko, A. I. Nikolskyy, R. A. Yatskovskaya, and V. I. Yatskovsky, “The concept
models and implementations of multiport neural net associative memory for 2D patterns,” in
84
Optical Pattern Recognition XXII, D. P. Casasent and T.-H. Chao, Eds., Proceedings of SPIE Vol.
8055, Bellingham, WA, 2011, p. 80550T.
[28] V. G. Krasilenko, A. A. Lazarev, and S. K. Grabovlyak, “Design and simulation of a multiport
neural network heteroassociative memory for optical pattern recognitions,” in Proceedings of SPIE
Vol. 8398, Bellingham, WA, 2012, p. 83980N.
[29] Demircigil M., Heusel J., Lowe M., Upgang S. Vermet F. On a model of associative memory with
huge storage capacity. J. Stat. Phys. 2017. Vol. 168, N 2. P. 288–299.
[30] Gripon V., Heusel J., Lowe M., Vermet F. A comparative study of sparse associative memories.
Journal of Statistical Physics. 2016. Vol. 164. P. 105–129.
[31] Gripon V., Lowe M., Vermet F. Associative memories to accelerate approximate nearest neighbor
search. ArXiv:1611.05898. 10 Nov 2016.
[32] D. Krotv and J. J. Hopfield. Dense associative memory for pattern recognition. In D. D. Lee, M.
Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information
Processing Systems, pages 1172–1180. Curran Associates, Inc., 2016.
[33] H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, M. Pavlovic,`
G. K. Sandve, V. Greiff, D. Kreil, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter.
Hopfield networks is all you need. ArXiv, 2008.02217, 2020.
[34] H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, M. Pavlovic,`
G. K. Sandve, V. Greiff, D. Kreil, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter.
Hopfield networks is all you need. In 9th International Conference on Learning Representations
(ICLR), 2021.
[35] Mazumdar A., Rawat A.S. Associative memory using dictionary learning and expander decoding.
Proc. AAAI'17. 2017. P. 267–273.
[36] Onizawa N., Jarollahi H., Hanyu T., Gross W.J. Hardware implementation of associative memories
based on multiple-valued sparse clustered networks. IEEE Journal on Emerging and Selected
Topics in Circuits and Systems. 2016. Vol. 6, N. 1. P. 13–24
[37] V. G. Krasilenko, “The structures of Optical Neural Nets Based on New Matrix – Tensor
Equivalental Models (MTEMS) and Results of Modeling,” Optical Memory and Neural Networks
(Information Optics), vol. 19, pp. 31-38, 2010.
[38] V. G. Krasilenko, A. A. Lazarev, and D. V. Nikitovich, “Design and simulation of optoelectronic
neuron equivalentors as hardware accelerators of self-learning equivalent convolutional neural
structures (SLECNS),” in Proc. SPIE 10689, Neuro-inspired Photonic Computing, 106890C, May
21, 2018, doi: 10.1117/12.2316352.
[39] Shafiee, A. et al. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic
in crossbars. ACM SIGARCH Computer Architecture N. 44, 14–26 (2016).
[40] Z. Sun et al., “In-memory PageRank accelerator with a cross-point array of resistive memories,”
IEEE Trans. Electron Devices, vol. 67, no. 4, pp. 1466–1470, Apr. 2020, doi:
10.1109/TED.2020.2966908.
[41] Chen, Z., et al. (2022). "Associative Memory and Pattern Recognition in Neuromorphic
Computing." IEEE Transactions on Cognitive and Developmental Systems, 14(3), 789-798.
[42] Li, M., & Wang, Y. (2021). "Associative Memory with Neural Network Architectures for Image
Recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), 2100-
2113.
[43] V. G. Krasilenko, A. A. Lazarev, and D. V. Nikitovich, “Design and simulation of array cells for
image intensity transformation and coding used in mixed image processors and neural networks,”
Proc. SPIE 10751, Optics and Photonics for Information Processing XII, 1075119
https://doi.org/10.1117/12.2322655.
85