One-Shot Rule Learning for Challenging Character Recognition

Abstract

Unlike most of computer vision approaches which depend on hundreds or thousands of training images, humans can typically learn from a single visual example. Humans achieve this ability using background knowledge. Rule-based machine learning approaches such as Inductive Logic Programming (ILP) provide a framework for incorporating domain speci c background knowledge. These approaches have the potential for human-like learning from small data or even one-shot learning, i.e. learning from a single positive example. By contrast, statistics based computer vision algorithms, including Deep Learning, have no general mechanisms for incorporating background knowledge. In this paper, we present an approach for one-shot rule learning called One-Shot Hypothesis Derivation (OSHD) which is based on using a logic program declarative bias. We apply this approach to the challenging task of Malayalam character recognition. This is a challenging task due to spherical and complex structure of Malayalam hand-written language. Unlike for other languages, there is currently no e cient algorithm for Malayalam handwritten recognition. We compare our results with a state-of-the-art Deep Learning approach, called Siamese Network, which has been developed for one-shot learning. The results suggest that our approach can generate human-understandable rules and also outperforms the deep learning approach with a signi cantly higher average predictive accuracy.

Keywords

• One-Shot Learning Rule-Based Machine Learning Induc- tive Logic Programming (ILP)
• Malayalam Character Recognition
• Computer Vision

-

Deep Neural Networks (DNNs) [ 7, 2, 15, 3 ] have demonstrated state-of-the-art results on many pattern recognition tasks, especially in image classi cation problems [ 14, 9, 23, 6 ]. However, recent studies [24, 26] revealed major di erences between human visual cognition and DNNs, and in general most of statistics-based Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). computer vision learning algorithms. For example, it is easy to produce images that are completely unrecognizable to humans, though DNN visual learning algorithms believe them to be recognizable objects with over 99% con dence [24].

Another major di erence is related to the number of required training examples. Humans can typically learn from a single visual example [ 10 ], unlike statistical learning which depends on hundreds or thousands of images. Humans achieve this ability using background knowledge, which plays a critical role. By contrast, statistics based computer vision algorithms have no general mechanisms for incorporating background knowledge.

Computer vision is a multidisciplinary eld that aims to create high-level understanding from digital images or videos. The key intention of image analysis is to bridge the semantic gap between low-level descriptions of an image and the high level concept within the image. The main objective of structural pattern analysis is to present the visual data using natural descriptions. Traditionally, this is achieving by extracting low-level visual cues from the data provided, then applying some grouping algorithm to express relationships that are then transformed into more and more complex and convoluted features that generate higher level rules [ 16 ].

Document image analysis approaches such as Optical Character Recognition (OCR) are an important part of visual arti cial intelligence with many real-world applications. The main objective of these approaches is to identify signi cant graphical properties from images. In this context, Symbol Recognition has a long history dating back to the 70's. In the current state-of-the-art, symbol recognition involves identifying isolated symbols, however this is not enough for some more challenging real-world application. As an example, consider an application where the visual data is represented as a combination of isolated symbols as well as composite symbols that are connected with other graphical elements. Then the statistical approaches which represent shapes only as low level features will have limited success.

In this paper, we present an approach for one-shot rule learning called OneShot Hypothesis Derivation (OSHD) which is based on using a logic program declarative bias. We apply this approach to the challenging task of Malayalam character recognition. This is a challenging task due to spherical and complex structure of Malayalam hand-written language. Unlike for other languages, there is currently no e cient algorithm for Malayalam hand-written recognition. The language scripts are mainly based on circular geometrical properties. We have created a dataset for Malayalam hand-written characters which includes high level properties of the language based on 'Omniglot' dataset designed for developing human-level concept learning algorithms [ 11 ]. We compare our results with a state-of-the-art Deep Learning approach, called Siamese Network [ 8 ], which has been developed for one-shot learning.

Inductive Logic Programming (ILP) and One-Shot

Hypothesis Derivation (OSHD)

Inductive Logic Programming (ILP) has been de ned as the intersection of inductive learning and logic programming [21]. Thus, ILP employs techniques from both machine learning and logic programming.

The main objective of ILP, in its simplest form, is to discover the de nition of a predicate by observing positive and negative examples of that predicate. Together with positive and negative examples of the target predicate, other background information may also be provided containing further information relevant to learning the target predicate. This background information is represented as a logic program and is called background knowledge(BK). ILP systems develop predicate descriptions from examples and background knowledge. The examples, background knowledge and nal descriptions are all described as logic programs.

The logical notations and foundations of ILP can be found in [21, 25]. The following de nition, adapted from [25], de nes the learning problem setting for ILP.

De nition 1 (ILP problem setting).

Input : Given hB; Ei, where B is a set of clauses representing the background knowledge and E is the set of positive (E+) and negative (E ) examples such that B 6j= E+.

Output : nd a theory H such that H is complete and (weakly) consistent with respect to B and E. H is complete with respect to B and E+ if B ^ H j= E+. H is consistent with respect to B and E if B ^ H ^ E 6j= 2. H is weakly consistent with respect to B if B ^ H 6j= 2.

In this de nition, j= represents logical entailment and 2 represents an empty clause or logical refutation. Note that in practice, due to the noise in the training examples, the completeness and consistency conditions are usually relaxed. For example, weak consistency is usually used and a noise threshold is considered which allows H to be inconsistent with respect to a certain proportion (or number) of negative examples.

The following example is adapted from [ 13 ].

Example 1. In De nition 1, let E+, E and B be de ned as follows: E+ = fdaughter(mary; ann); daughter(eve; tom)g E

= fdaughter(tom; ann); daughter(eve; ann)g B = fmother(ann; mary); mother(ann; tom); f ather(tom; eve); f ather(tom; ian); f emale(ann); f emale(mary); f emale(eve); male(pat); male(tom); parent(X; Y ) parent(X; Y ) mother(X; Y ); f ather(X; Y )g Then both theories H1 and H2 de ned as follows:

H1 = fdaughter(X; Y ) H2 = fdaughter(X; Y ) daughter(X; Y ) f emale(X); parent(Y; X)g f emale(X); mother(Y; X); f emale(X); f ather(Y; X)g are complete and consistent with respect to B and E. 2.1

One-Shot Hypothesis Derivation (OSHD) In this paper we adopt a form of ILP which is suitable for one-shot learning and is based on using a logic program declarative bias, i.e. using a logic program to represent the declarative bias over the hypothesis space. Using a logic program declarative bias has several advantages. Firstly, a declarative bias logic program allows us to easily port bias from one problem to another similar problem (e.g. for transfer learning). Secondly, it is possible to reason about the bias at the metalevel. Declarative bias will also help to reduce the size of the search space for the target concept or hypothesis derivation [ 1, 22 ]. We refer to this approach as One-Shot Hypothesis Derivation (OSHD) which is a special case of Top-Directed Hypothesis Derivation (TDHD) as described in [ 18 ].

De nition 2 (One-Shot Hypothesis Derivation). The input to an OSHD system is the vector ST DHD = hN T; >; B; E; ei where N T is a set of \nonterminal" predicate symbols, > is a logic program representing the declarative bias over the hypothesis space, B is a logic program representing the background knowledge and E is a set of examples and e is a positive example in E. The following three conditions hold for clauses in >: (a) each clause in > must contain at least one occurrence of an element of N T while clauses in B and E must not contain any occurrences of elements of N T , (b) any predicate appearing in the head of some clause in > must not occur in the body of any clause in B and (c) the head of the rst clause in > is the target predicate and the head predicates for other clauses in > must be in N T . The aim of a OSHD learning system is to nd a set of consistent hypothesised clauses H, containing no occurrence of N T , such that for each clause h 2 H the following two conditions hold: > j= h B; h j= e (1) (2) The following theorem is a special case of Theorem 1 in [ 18 ].

Theorem 1. Given SOSHD = hN T; >; B; E; ei assumptions (1) and (2) hold only if there exists an SLD refutation R of :e from >; B, such that R can be re-ordered to give R0 = DhRe where Dh is an SLD derivation of a hypothesis h for which (1) and (2) hold.

According to Theorem 1, implicit hypotheses can be extracted from the refutations of e. Let us now consider a simple example.

>1 = alphabet(X) $body(X) :e = alphabet(a) >2 = $body(X) property1(X)

G1 = $body(a) b1 = property1(a)

G2 = property1(a) 2 Given the linear refutation, R = h:e; >1; >2; b1i, as shown in Figure 1, we now construct the re-ordered refutation R0 = DhRe where Dh = h>1; >2i derives the clause h = alphabet(X) property1(X) for which (1) and (2) hold.

The user of OSHD can specify a declarative bias > in the form of a logic program. A general > theory can be also generated from user speci ed mode declarations. Below is a simpli ed example of user speci ed mode declarations and the automatically constructed > theory. 2.2

The OSHD Learning Algorithm The OSHD Learning algorithm can be described in 3 main steps: 1. Generate all hypotheses, He that are generalizations of e modeb(has prop1(+image)). modeb(has prop2(+image)).

8 >> >1 : alphabet(X) > > > = <> >2 : $body(X) >>> >3 : $body(X) > >: >4 : $body(X)

$body(X): :%emptybody has prop1(X); $body(X): has prop2(X); $body(X): 2. Compute the coverage of each hypothesis in He 3. Build nal theory, T , by choosing a subset of hypothesis in He that maximises a given score function (e.g. compression)

In step 1, He is generated using the OSHD hypothesis derivation described earlier in this section.

The second step of the algorithm, computing the coverage of each hypothesis, is not needed if the user program is a pure logic program (i.e. all relationships in the background knowledge are self contained and do not rely on Prolog built in predicates). This is because, by construction, the OSHD hypothesis derivation generates all hypotheses that entail a given example with respect to the user supplied mode declarations. This implies that the coverage of an hypothesis is exactly the set of examples that have it as their generalization. However, this coverage computation step is needed for the negative examples, as they were not used to build the hypothesis set.

For step 3, the compression-based evaluation function used for the experiments in this paper is:

X Covered Examples W eight

T otal Literals (3)

The weight associated to an example may be de ned by the user but by default, positive examples have weight 1 and negative examples weight -1. In general, negative examples are de ned with a weight smaller than 0 and positive examples with a weight greater than 0. 3

Siamese Neural Networks

In this paper, we use a state-of-the-art Deep Learning approach, called Siamese Network [ 8 ], which has been developed for one-shot learning. The original Siamese Networks were rst introduced in the early 1990s by Bromley and LeCun to solve signature veri cation as an image matching problem [ 5 ]. A Siamese network is a Deep Learning architecture with two parallel neural networks with the same properties in terms of weight, layers etc. Each network takes a di erent input, and whose outputs are combined using energy function at the top to provide some prediction. The energy function computes some metric between the highest level feature representation on each side (Figure 3). Weight tying guarantees that two extremely similar images could not possibly be mapped by their respective networks to very di erent locations in feature space because each network computes the same function. Also, the network is symmetric, so that whenever we present two distinct images to the twin networks, the top conjoining layer will compute the same metric as if we were to present the same two images but to the opposite twin. 4

One-Shot learning for Malayalam character recognition

We apply One-Shot Hypothesis Derivation (OSHD) as well as Deep Learning (i.e. Siamese Network) to the challenging task of one-shot Malayalam character recognition. This is a challenging task due to spherical and complex structure of Malayalam hand-written language. 4.1

Character recognition and human-like background knowledge Malayalam is one of the four major languages of the Dravidian language family and originated from the ancient Brahmi script. Malayalam is the o cial language of Kerala, a state of India with roughly forty- ve million people. Unlike for other languages, there is currently no e cient algorithm for Malayalam hand-written recognition. The basic Malayalam characters along with International Phonetic Alphabet (IPA) are shown in Figure 41.

The handwriting recognition for Malayalam script is a major challenge compared to the recognition of other scripts because of the following reasons: { Presence of large number of alphabets { Di erent writing styles { Spherical features of alphabets 1 From: https://sites.google.com/site/personaltesting1211/malayalam-alphabet. (a) Malayalam Vowels (b) Special (Chill)

Consonants (c) Consonants and Consonant Clusters

We selected the hand-written characters from 'Omniglot' dataset [ 11 ]. Sample Malayalam alphabets from our dataset are shown in Figure 5. Feature extraction is conducted utilizing a set of advanced geometrical features [27] and directional features.

Geometrical Features Every character may be identi ed by its geometric designations such as loops, junctions, arcs, and terminals. Geometrically, loop means a closed path. Malayalam characters contain more intricate loops which (a) Character 'Aha'

(b) Character 'Tha' may contain some up and downs within the loops itself. So we follow a concept as shown in Figure 6(b). If the gure has a continuous closed curve then we will identify it as a loop. Junctions may be de ned as a meeting point of two or more curves or line. It is easy for human to identify the junction from an image as shown in Figure 6(c). As per dictionary de nitions, an arc is a component of a curve. So in our case, a path with semi opening will be considered as an arc. Please refer to Figure 6(d) for more details. Terminals may be classi ed as points where the character stroke ends, i.e. no more connection beyond that point. Figure 6(e) is a self-explanatory example for the de nition.

We have included the visual explanation for the geometrical feature extraction in Figure 6. We have selected two characters to explicate the features as shown in Figure 5 and marked each geometrical features as we discussed. Table 1 will give an abstract conception about the dataset we have developed for the experiments from 'Omniglot' dataset.

Directional Features Every character may be identi ed by its directional speci cations such as starting and ending points of the stroke. There are certain unwritten rules for Malayalam characters, e.g. it always commences from left and moves towards the right direction. Native Malayalam users can easily identify the starting and ending point. However, we will need to consider the starting and ending point as features so that these can be easily identi ed without semantic knowledge of a character. The starting and ending points are determined by standard direction properties as shown in the Figure 6(a). Figure 6(f, g) will give you an idea about developing the directional features from an alphabet. Character ID:13 is the corresponding entry for the character shown in gure 6(g). As we discussed, a user can identify both starting and ending point of the character displayed in Figure 6(g) easily whereas the terminus point of Figure 6(f) is arduous to determine. 4.2

Mode declarations and background knowledge representation In this section, we de ne the OSHD speci c details of the declarative bias, dened by mode declaration and background knowledge representation used in our experiments.

(a) Direction Properties (b) Feature : Loop (c) Feature : Junction (d) Feature : Arc (e) Feature : Terminals

(f) Feature : Starting Point (g) Feature : Ending Point

The rst step was to develop and represent the background knowledge based on the concepts described in Section 4.1. Table 1 shows the geometrical and directional features of 18 characters from 5 di erent alphabets used in our experiments.

Mode declaration and declarative bias In this section we describe how the declarative bias for the hypothesis space was de ned using mode declarations. Here, we use the same notations used in Progol [ 17 ] and Toplog [ 18 ]. There are two types of mode declarations. 1. modeh : de nes the head of a hypothesised rule. 2. modeb : de nes the literals (conditions) that may appear in the body of a hypothesised rule.

For example, in our experiments, alphabet(+character) is the head of the hypothesis, where +character de nes the character identi er character as an

No. Loops No. Junctions No. Arcs No. Terminals Starting Point Ending Point 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 3 1 1 0 1 1 1 3 4 3 1 1 1 0 0 0 4 4 4 2 3 1 1 1 1 3 4 4 1 1 1 2 0 2 3 3 3 3 3 3 2 2 2 4 4 3 2 1 1 1 1 1 2 2 2 2 2 2 1 2 1 1 0 0 2 2 2 2 2 2 sw sw sw null nw nw null nw null null null null sw nw sw sw sw nw null null null se se se se se se se null null se se ne ne ne ne input argument. We are using four predicates in the body part of the hypothesis as shown in the listing 1.1. Note that +, -, indicate input, output or a constant value arguments. : : : : :

Listing 1.1: Mode declarations modeh ( 1 , a l p h a b e t (+ c h a r a c t e r ) ) . modeb ( , h a s g e m p r o p e r t i e s (+ c h a r a c t e r , p r o p e r t i e s ) ) . modeb ( , h a s g e m p r o p e r t i e s c o u n t (+ p r o p e r t i e s ,

#g e o f e a t u r e n a m e ,# i n t ) ) . modeb ( , h a s d i r p r o p e r t i e s (+ c h a r a c t e r , p r o p e r t i e s ) ) . modeb ( , h a s d i r p r o p e r t i e s f e a t u r e (+ p r o p e r t i e s ,

#d i r f e a t u r e n a m e ,# f e a t u r e v a l u e ) ) .

The meaning of each modeb condition is de ned as follows: has gemproperties=2 predicate was used to represent the geometrical features as de ned in Table 1. The input argument character is the unique identi er for an alphabet, properties refers to the property name. has gemproperties count=3 predicate outlines the count of the particular feature associated with the alphabet. The properties is the unique identi er for a particular geometrical property of a particular alphabet , geo f eature name refers to the property name and int stands for the feature count. has dirproperties=2 predicate used to represent the directional features mentioned in table 1. The character is the unique identi er for the alphabet, properties refers to the property name. has dirproperties count=3 predicate outlines the count of a particular directional feature associated with the alphabet. The properties is a unique identi er for a particular property of a particular alphabet , dir f eature name refers to the property name and f eaturevalue stands for the feature vale. Background knowledge representation As de ned in De nition 1, background knowledge is a set of clauses representing the background knowledge about a problem. In general, background knowledge can be represented as a logic program and could include general rst-order rules. However, in this paper we only consider ground fact background knowledge. In the listing 1.2 we have a sample background knowledge for an alphabet.

Listing 1.2: Sample background knowledge for alphabet 'Aha' In this section we evaluate the OSHD approach for complex character recognition as described in this paper. We also compare the performance of OSHD with a state-of-the-art Deep Leaning architecture for one-shot learning, i.e. the Siamese Network approach described in Section 4. In particular we test the following null hypotheses: Null Hypothesis H1 OSHD cannot outperform Siamese Networks in one-shot learning for complex character recognition.

Null Hypothesis H2 OSHD cannot learn human comprehensible rules for complex character recognition. 5.1

Materials and Methods The OSHD algorithm in this experiment is based on Top-Directed Hypothesis Derivation implemented in Toplog [ 18 ], and uses mode declarations and background knowledge which de ned earlier in this paper. The Siamese Network used in the experiment is based on the implementation described in [ 8 ].

The challenging Malayalam character recognition dataset and the machine learning codes, con gurations and input les are available from: https://github.com/danyvarghese/One-Shot-ILP

We have selected 5 complex alphabets from the 'Omniglot' dataset [ 11 ]. Example characters used in our experiment and their visual properties (developed using the geometrical and directional concepts, as discussed in Section 4) are listed in Table 2. We have endeavoured to reiterate the same concept of working in both architectures and repeated the experiments for di erent number of folds and each fold consists of single positive example and n negative examples, where n varies from 1 to 4 and the negative examples are selected from other alphabets. In our experiment we are using the term 'number of classes' in di erent aspect. The number of classes is de ned by the total number of examples (i.e. 1 positive and n negative) used for the cross-validation.

In the following we de ne speci c parameter settings for each algorithm. OSHD parameter settings The following Toplog parameter settings were used in this experiment. clause length (value = 15) de nes the maximum number of literals (including the head) of a hypothesis. weight the weight of negative example is taken always as the default value. The weight of positive example is the number of negative examples for that class. During the cross-validation test, we add one more positive example of the same alphabet for each fold. The weight of newly added example will not be greater than the previous one included in the same fold. positive example inf lation (value = 10) multiplies the weights of all positive examples by this factor. negative example inf lation (value = 5) multiplies the weights of all negative examples by this factor. Siamese Networks parameter settings For the implementation of the Siamese Net, we followed the same setups used by Koch et al [ 8 ]. Koch et al use a convolutional Siamese network to classify pairs of 'Omniglot' images, so the twin networks are both Convolutional Neural Nets (CNNs). The twins each have the following architecture: { Convolution with 64 (10 10) lters uses 'relu' activation function. { 'max pooling' convolution 128 (7 7) lters with 'relu' activation function. { 'max pooling' convolution 128 (4 4) lters with 'relu' activation function. { 'max pooling' convolution 256 (4 4) lters with 'relu' activation function. The twin networks reduce their inputs down to smaller and smaller 3D tensors. Finally, there is a fully connected layer with 4096 units.

In most of the implementations of Siamese Network, they are trying to develop the training model from a high amount of data. Also, particularly in the case of character recognition, they compare a character from a language against the characters from other languages [ 12, 4 ]. In our experiment we have only considering alphabets from a single language. 5.2

Results and Discussions Figure 7 shows the average predictive accuracy of ILP (OSHD) vs Deep Learning (Siamese Net) in One shot character recognition with increasing number of character classes. According to this gure, OSHD outperforms the Siamese Nets, with an average di erence of more than 20%. In this gure the random curve represents the default accuracy of random guess. The accuracy for one class prediction is always 100%. Null hypothesis H1 is therefore refuted by this experiment. A better predictive accuracy of OSHD compared to the Siamese Net could be explained by the fact that it uses background knowledge.

Table 3 shows example of learned rules by OSHD generated from one positive and two negative examples. One can easily di erentiate alphabet 'Aha' against 'Eh' & 'Uh'. The unique properties of 'Ah' from others alphabets is given in the column 'Human Interpretations', which is almost similar to the learned rule in column 4. It is also clear that the rule in column 4 is human comprehensible. Null hypothesis H2 is therefore refuted by this experiment. 6

Conclusion

In this paper, we presented a novel approach for one-shot rule learning called One-Shot Hypothesis Derivation (OSHD) which is based on using a logic program declarative bias. We applied this approach to the challenging task of Malayalam character recognition. This is a challenging task due to spherical and complex structure of Malayalam hand-written language. Unlike for other languages, there is currently no e cient algorithm for Malayalam hand-written recognition. The features used to express the background knowledge were developed in such a way that it is acceptable for human visual cognition also. We could learn rules for each character which is more natural and visually acceptable. We compared our results with a state-of-the-art Deep Learning approach, called Siamese Network, which has been developed for one-shot learning. The results suggest that our approach can generate human-understandable rules and also outperforms the deep learning approach with a signi cantly higher average predictive accuracy (an increase of more than 20% in average). Its was clear from the results that deep learning paradigm use more data and its e ciency is less when dealing with a small amount of data. As future work we would like to further extend the background knowledge to include more semantic information. We will also explore the new framework of Meta-Interpretive Learning (MIL) [ 19, 20 ] in order to learn recursive rules.

Acknowledgments

We would like to acknowledge Stephen Muggleton and Jose Santos for the development of Top Directed Hypothesis Derivation and Toplog [ 18 ] which was the basis for OneShot Hypothesis Derivation (OSHD) presented in this paper. We also acknowledge the Vice Chancellor's PhD Scholarship Award at the University of Surrey. 20. Muggleton, S., Dai, W.Z., Sammut, C., Tamaddoni-Nezhad, A.: Meta-interpretive learning from noisy images. Machine Learning 107 (2018) 21. Muggleton, S., de Raedt, L.: Inductive logic programming: Theory and methods.

The Journal of Logic Programming 19-20, 629 { 679 (1994), special Issue: Ten Years of Logic Programming 22. Nedellec, C.: Declarative bias in ilp (1996) 23. Neethu, K.S., Varghese, D.: An incremental semi-supervised approach for visual domain adaptation. In: 2017 International Conference on Communication and Signal Processing (ICCSP). pp. 1343{1346 (2017) 24. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High con dence predictions for unrecognizable images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 427{436 (2015) 25. Nienhuys-Cheng, S.H., de Wolf, R.: Foundations of Inductive Logic Programming.

Springer-Verlag, Berlin (1997), lNAI 1228 26. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2014) 27. Usman Akram, M., Bashir, Z., Tariq, A., Khan, S.A.: Geometric feature points based optical character recognition. In: 2013 IEEE Symposium on Industrial Electronics Applications. pp. 86{89 (2013)

References

1. Ade , H. , Raedt , L.D. , Bruynooghe , M.: Declarative bias for speci c-to-general ilp systems . Machine Learning 20 , 119 { 154 ( 1995 )

2. Bengio , Y. : Learning Deep Architectures for AI . Found. Trends Mach. Learn . 2 ( 1 ), 1 { 127 (Jan 2009 )

3. Bennett , C.H. , Parmar , V. , Calvet , L.E. , Klein , J. , Suri , M. , Marinella , M.J. , Querlioz , D. : Contrasting advantages of learning with random weights and backpropagation in non-volatile memory neural networks . IEEE Access 7 , 73938 { 73953 ( 2019 )

4. Bouma , S. : One shot learning and siamese networks in keras . https://sorenbouma.github.io/blog/oneshot/ ( 2017 )

5. Bromley , J. , Guyon , I. , LeCun, Y. , Sa ckinger, E., Shah , R.: Signature veri cation using a \siamese" time delay neural network . In: Proceedings of the 6th International Conference on Neural Information Processing Systems . p. 737 { 744 . NIPS' 93 , Morgan Kaufmann Publishers Inc., San Francisco, CA, USA ( 1993 )

6. Dany Varghese , Viju Shankar: A novel approach for single image super resolution using statistical mathematical model . International Journal of Applied Engineering Research (IJAER) 10 ( 44 ) ( 2015 )

7. Hinton , G.: Learning multiple layers of representation . Trends in cognitive sciences 11 , 428 { 434 (11 2007 )

8. Koch , G. , Zemel , R. , Salakhutdinov , R.: Siamese neural networks for one-shot image recognition . In: Proceedings of the 32 nd International Conference on Machine Learning . vol. 37 ( 2015 )

9. Krig , S. : Computer Vision Metrics Survey, Taxonomy, and Analysis . Apress, Berkeley, CA ( 2014 )

10. Lake , B. , Salakhutdinov , R. , Gross , J. , Tenenbaum , J.: One shot learning of simple visual concepts . In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society . pp. 2568 { 2573 ( 2011 )

11. Lake , B.M. , Salakhutdinov , R. , Tenenbaum , J.B. : Human-level concept learning through probabilistic program induction . Science 350 ( 6266 ), 1332 { 1338 ( 2015 )

12. Lamba , H.: One shot learning with siamese networks using keras . https://towardsdatascience.com /one-shot-learning-with-siamese-networks-usingkeras- 17f34e75bb3d ( 2019 )

13. Lavrac , N., Dzeroski, S.: Inductive Logic Programming : Techniques and Applications . Ellis Horwood ( 1993 )

14. Le , Q.V. , Zou , W.Y., Yeung , S.Y. , Ng , A.Y. : Learning hierarchical invariant spatiotemporal features for action recognition with independent subspace analysis . In: CVPR 2011 . pp. 3361 { 3368 ( 2011 )

15. Liu , X. , He , P. , Chen , W. , Gao , J.: Multi-task deep neural networks for natural language understanding . CoRR 1901 . 11504 ( 2019 )

16. Mas , J. , Sanchez , G. , Llados , J. , Lamiroy , B. : An incremental on-line parsing algorithm for recognizing sketching diagrams . In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007 ). vol. 1 , pp. 452 { 456 ( 2007 )

17. Muggleton , S. : Inverse entailment and Progol. New Generation Computing 13 , 245 { 286 ( 1995 )

18. Muggleton , S.H. , Santos , J. , Tamaddoni-Nezhad , A. : TopLog: ILP using a logic program declarative bias . In: Proceedings of the International Conference on Logic Programming 2008 . pp. 687 { 692 . LNCS 5366, Springer-Verlag ( 2010 )

19. Muggleton , S. , Lin , D. , Tamaddoni-Nezhad , A. : Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited . Machine Learning 100 ( 1 ), 49 { 73 ( 2015 )