The proposed model consists of a 2-layered neural network with feed forward connections from layer 1 (L1) to
layer 2 (L2) and the L2 neurons form a fully connected graph. Similar to biological neurons, the processing
neurons will be either in the " ring" state (yi = 1) or "not ring" state (yi = 0)[
The model uses two weight matrices W 1 = Wi1j , 8 i 2 L1, j 2 L2 and W 2 = Wi2j 8 i 2 L2, j 2 L2. 2.1
The user is required to specify the dimension d of the pattern vector space and the number of active neurons that should re for every pattern. This global count m is used as the winning criteria for the model to nd a winner set of neurons during the testing phase. This does restrict the model from using the full capacity of the d-dimensional input vector. Although letting the model run for a certain number of epochs (set by the user) may seem like a plausible strategy, it can incur excessive contamination in the output pattern depending on the number of epochs. For a d-dimensional input vector and count of active neurons m, the number of possible patterns that can be fed into the model would be md . 2.2
(a)
(b)
The training of the model is dependent upon the time duration for which the pattern associations are presented. The patterns that need to be strongly remembered are given more training time. Consider the memorization of the association Sp ! Op. During the training phase for Sp ! Op, we will assume that both the ring neurons of Sp and Op remain ring till its training duration and there is no decay with time. At the start, when there is no stored associations, the weight matrix is initialized as Wij = 0, 8 i,j.
At each training epoch, the weights between the pattern get updated as follows:
Wi1j = Wi1j + and; Wi2j = Wi2j + ui vj vi vj
Dij Dij (t) (t) (1 (1 vj ) ui
Dij vj ) vi
Dij (t) (t) where : ui, uj 2 Sp and vi, vj 2 Op and
represent the learning rate and the unlearning rate respectively (t) is the multiplier that increases at the start of each training session. The most recent training session will have the highest value of (t).
Dij here represent the inter-neuronal distance. One measure of distance could simply be Dij = ji jj + 1.
Wij 2 [
The recall of stored associations is facilitated when the pattern Sp is introduced as input into the model. The recall process involves the triggering of depolarizations in the output vector Y p. The depolarizations are recorded by the vector Z. A neuron yi in Y p is activated (yi = 1) if the depolarization for that neuron (Zi) exceeds the threshold (Ui). The d-dimensional output vector Y p is continually updated as the recall phase goes on and the recall is completed when the number of active neurons in Y p is m.
At the nth epoch: yin = (1, 0, j if Zin > Ui if Zin < Ui where uj 2 Sp and yi,yj 2 Y p j The model so far gives a single output when an input is presented. This output is determined by the winner set of neurons among the competing neurons. There seems to be a lot of wasted potential in the sense that other output patterns associated with that particular input pattern are forgotten or cannot be retrieved. To tackle this problem, we introduce an inhibition vector Ip that inhibits the set of winner neurons to let the other set of competing neurons win.
The inhibition vector Ip is implemented as a hash table. The entries of Ip are set when a recall is ended and the output is retrieved. For a set of active neurons fyi1 , yi2 , . ., yim g, the hash table entry is lled by a hash function h(i1; i2; ::; im).
During the end of a recall session i.e when Pd
i=1 yi = m, the inhibition is triggered and updates Zi as: Zi = (0,
Zi, if yi Ip[h(keys)] = 1 if yi Ip[h(keys)] = 0
The deactivation of the previously active neurons will let the other competing neurons cross the threshold and emerge as the winning set of active neurons.This algorithm when repeated over and over, facilitates a breadthwise traversal of stored patterns in the implicit pattern association graph (corresponding to the origin input pattern).
The depthwise traversal of the memory graph can be achieved by simply using the recalled output as the new input to be recalled.
Algorithm 1 Breadthwise Traversal 14: m = 5 0.7187 m = 5 0.5513 The model was trained with random d-dimensional pattern vectors of m active neurons with a training time of 500 epochs per training sample. Table 1 shows the results of the average testing accuracy over varying numbers of stored associations (n), m and d on di erent training batches. For lower-dimensional pattern inputs (d < 30), the recall is almost perfect with an accuracy of around 0.8785. This metric was further bumped up when trained and tested against a set of mutually orthogonal pattern vectors and showed a mean testing accuracy of 0.9017 (The error only stemming from multi-output associations). For higher dimensional pattern inputs (d > 30), the testing accuracy dips as low as 0.4003 and the output su ers from contamination. This is e ectively due to the saturation of the weight matrix which fails the competitive learning procedure. The accuracy, however, was high (0.8314) when a set of mutually orthogonal pattern vectors were chosen.
Further, we simulated the training of the model for a single input vector associated with multiple output vectors (l) and the results are shown in Table 2. The testing was done by utilizing breadthwise traversal to recall the multi-output pattern associations. The results were quite similar to that of the recall of random pattern associations. Lower-dimensional vectors (d < 30) achieved an average testing accuracy of 0.8785 whereas higher-dimensional vectors (d > 30) went as low as 0.4161 4
Our results showed that the proposed model was able to e ciently store and retrieve pattern associations with minimal error (for lower dimensions). Further, in combination with the traversal schemes devised, the model is a great candidate for multi-modal information retrieval. Although restricted by the neural encoding scheme presented in this paper, there is still a high storage capacity (before excessive contamination).
However, there is much to be improved. We believe that the model can be made even better by introducing
several layers of neurons and thus induce multi-stage competition among neurons that would further enhance
learning. Like [