We convert existing training datasets into the ones closed under linguistic generalization operations to expand infrequent cases. We transfer the definition of the least general generalization from logical formulas to linguistic structures, from words to phrases, sentences, speech acts and discourse trees. The main advantage of the resultant frameworks is explainability and learnability from a small set of samples. Learning via generalization of linguistic structures turned out to be well suited for industrial linguistic applications with limited training datasets.
camera(zoom(AnyZoom), beginner), where variables (non-instantiated values, not specified in NL expressions) are capitalized. Given the above pair of formulas, unification computes their most general specialization camera(zoom(digital), beginner), and anti-unification computes their most specific generalization, camera(zoom(AnyZoom), AnyUser).
At syntactic level, we have generalization (‘^’) of two noun phrases as: {NN-camera, PRP-with, [digital], NN-zoom [for beginners]}.
We eliminate expressions in square brackets since they occur in one expression and do not occur in another. As a result, we obtain{NN-camera, PRP-with, NN-zoom]}, which is a syntactic analog as the semantic generalization above.
The purpose of an abstract generalization is to find commonality between portions of text at various semantic levels. Generalization operation occurs on the levels of Text / Paragraph / Sentence / Individual word.
At each level except the lowest one, individual words, the result of generalization of two expressions is a set of expressions. In such set, for each pair of expressions so that one is less general than other, the latter is eliminated. Generalization of two sets of expressions is a set of sets which are the results of pairwise generalization of these expressions.
Only a single generalization exists for a pair of words: if words are the same in the same form, the
result is a node with this word in this form. To involve word2vec models
For a pair of phrases, generalization includes all maximum ordered sets of generalization nodes for words in phrases so that the order of words is retained. In the following example
To buy digital camera today, on Monday
Digital camera was a good buy today, first Monday of the month generalization is {<JJ-digital, NN-camera> ,<NN- today, ADV,Monday>} , where the generalization for noun phrases is followed by the generalization by adverbial phrase. Verb buy is excluded from both generalizations because it occurs in a different order in the above phrases. Buy - digital - camera is not a generalization phrase because buy occurs in different sequence with the other generalization nodes.
At the discourse level, rhetorical relations with elementary discourse units can be generalized as well.
Only rhetorical relations of the same class (presentation relation, such as antithesis, subject matter
relation, such as condition, and multinuclear relation, such as list) can be generalized. We use N for a
nucleus or situations presented by this nucleus, and S for satellite or situations presented by this satellite.
Situations are propositions, completed actions or actions in progress, and communicative actions and states
(including beliefs, desires, approve, explain, reconcile and others). Hence we have the following
expression for Rhetoric Structure Theory- based
The texts in N1, S1 are subject to generalization as phrases. The rules for rst1^ rst2 are as follows. If relation_type(rst1 ) ! = relation_type(rst2 ) then similarity is empty. Otherwise, we generalize the signatures of rhetoric relations as sentences: sentence(N1, S1) ^ sentence (N2, S2) (Iruskieta et al., 2015).
To optimize the calculation of generalization score, we rely on a computational study which
determined the POS weights to deliver the most accurate similarity measure between sentences
possible
Result of the generalization can be further generalized with other parse trees or generalization. For a
set of sentences, totality of generalizations forms a lattice: order on generalizations is set by the
subsumption relation and generalization score
To rank answers according to the relevance to a complex paragraph-sized question, paragraph-level
generalization comes into play. It needs to involve a set of parse trees for each sentence of the
paragraph and the paragraph-level discourse information. We refer to the sequence of parse trees plus a
number of arcs for inter-sentence relations of the discourse type between the nodes for words as a
parse thicket
Our example is a web search query and its answers selected from the first page of a Google search. Although both answers A1 and A2 share very similar keywords, we show that using discourse information helps to differentiate them, relying on parse thicket representation and syntactic generalization operation with query Q.
Q: I am buying a foreclosed house. A bank offered me to waive inspection; however I am afraid I will not identify some problems in this property unless I call a specialist.
A1: My wife and I are buying a foreclosure from a bank. In return for accepting a lower offer, they want me to waive the inspection. I prefer to let the bank know that I would not waive the inspection … Instead I would agree that I would pay the costs involved for a proper and thorough home inspection… A2: I am a foreclosure specialist in a bank which is subject to an inspection. FTC offered us to waive inspection if we can identify our potential problems with customers we lent money to buy their properties.
The reader can see that A2 is totally irrelevant while A1 is relevant.
We selected the first Google search result for the correct answer and composed a totally irrelevant answer with the same keywords to demonstrate the role of discourse-level generalization.
The list of common keywords gives us a hint that both documents are about a relationship between the same entities, a house, a buyer and a bank in connection to a foreclosure and an inspection. However one can see that the relations between these entities in A1 and A2 are totally different. It is also obvious that something beyond the keyword statistics and n-gram analysis needs to be done to figure out the correspondence of the structure of these relations between A1 and Q, and A2 and Q. Buy, foreclosure , house, bank, wave, inspection..
One can see that the key for the right answer here is rhetorical (RST) relation of contrast: bank wants the inspection waved but the buyer does not. Parse thicket generalization gives the detailed similarity picture that looks more complete than both the bag-of-words approach and pair-wise sentence generalization would. The similarity between Q and A1 is expressed as a parse thicket expressed here as a list of phrases [[NP [DT-a NN-bank ], NP [NNS-problems ], NP [NN*-property ], NP [PRP-i ]], VP [VB-* TO-to NN-inspection ], VP [NN-bank VB-offered PRP-* TO-to VB-waive NN-inspection ], VP [VB-* VBidentify NNS-problems IN-* NN*-property ], VP [VB-* {phrStr=[], roles=[A, *, *], phrDescr=[]} DT-a NN-* ]]] And similarity with the invalid answer A2 is expressed as a parse thicket formed as a list of phrases [[NP [DT-a NN-bank ], NP [PRP-i ]], [VP [VB-* VB-buying DT-a ], VP [VB-* PRP-me TO-to VBwaive NN-inspection ], VP [VB-* {phrStr=[], roles=[], phrDescr=[]} PRP-i MD-* RB-not VB-* DT-* NN*-* ],
The important phrases of the Q ^ A1 similarity are VP [NN-bank VB-offered PRP-* TO-to VB-waive NN-inspection], VP [VB-* VB-identify NNS-problems IN-* NN*-property], which can be interpreted as a key topic of both Q and A1: bank and not another entity should offer to waive inspection. This is what differentiates A1 from A2 (where these phrases are absent). Although bank and problems do not occur in the same sentences in Q and A1, they were linked by anaphora and RST relations. Without any kind of discourse analysis, it would be hard to verify whether the phrases containing bank and problems are related to each other. Notice that in A2, problems are associated with customers, not banks, and different rhetoric relations from those common between Q and A1 help us figure that out. Notice the semantic role attributes for verbs such as VB-* {phrStr=[], roles=[A, *, *], phrDescr=[]} in generalization result.
Parse thickets for Q, A1 and A2 are shown in Fig. 1a, 1b and 1c respectively. Notice the similarity in discourse structure of Q, A1 and not in A2: the RST-contrast arc. Also, there is a link for a pair of communicative actions for Q, A1 (it is absent in A2): afraid-call and accept-want.
The generalization operation described earlier can be applied for the expanding of the training sets. It can multiply tail cases, make it more balanced, and eliminate noisy samples which cannot be generalized. We are planning to apply it in the number of the industrial linguistic applications with limited training datasets.
The article was prepared within the framework of the HSE University Basic Research Program and funded by the Russian Academic Excellence Project '5-100'.