The paper presents a short summary on the applications of the quantum logic categorical constructions to the natural language processing. We give a brief overview on the topic of quantum logic in general, and in natural language processing, in particular. As a result, we discuss comparison of sentences and their representation in quantum logic formalism. The examples of using quantum diagrams are considered in order to understand text analysis in terms of quantum logic techniques.
The main goal of our work is to provide an overview on combining words and grammar in the natural language processing using quantum logic categorical methods.
In [
The former one does not take into account the interaction between syntactically linked words and is based only on the meaning of words, from which the sentence was composed. The word meaning is usually represented as vector, calculated from the context, in which this word occurs. One word is considered to be in context of another word, if it is in the n-word window of this word in the text. For instance, in the sentence ’The cat ate the mouse’ words ’the’ and ’ate’ are in window of size one of the word ’cat’. The sentence meaning can be then determined as certain simple functions like addition or element-wise multiplication of vectors of constituting words. The main drawback of this method is its non-compositional nature. For example, the sentences ’The cat ate the mouse’ and ’The mouse ate the cat’ will have the same meanings in such type of models.
The latter one, in opposite, is concerned only with syntax. The meaning of a
sentence is represented as a function derived from the grammatical structure of
the sentence. The major aim of this approach is to represent natural language
sentences as logical expressions. For this reason, the only thing we can say about
the meaning of a sentence, is its truth or falsity. Thus, formal semantics deal with
qualitative scale, and we cannot numerically estimate similarity of meaning of
two sentences. In [
Each of the two approaches has its own shortcomings, which are more or less
solved in the other one. In [
One of the main applications of sentence representation models is to compute how close are the meanings of two given sentences. If a model can define, which sentences are similar and which are not, then we could apply this model for paraphrasing problem and improving search engines answers to queries taking into account answers for similar queries. We can also use similarity analysis as a model indicator: if model correctly determine meaning of any sentence, then it can identify which of them are close.
In [
In distributional models all meanings of words are represented as vectors, that is why we cannot apply the verb to its subject and object. In order for the model to take it into account, all words are divided into atomic type and compound functional type. Nouns have atomic type, and verbs, adjective phrases, prepositional phrases, adverbs have compound types.
The authors of [
Nouns are assigned to vectors with elements equal to the number of times they have been in relations with the bases in the corpus of text.
Verbs are assigned with matrices in the following way: element i, j is equal to the number of times a noun with property i has been subject of the verb, and a noun with property j has been object of the verb in the corpus of text. In a similar way they define vectors for other syntax types.
For example, basis vectors might be associated with properties such as ’arghungry’, denoting the argument of the adjective ’hungry’, ’subj-eat’ denoting the subject of the verb ’eat’, ’arg-tasty’ denoting the argument of the adjective ’tasty’. If a noun has occurred as an argument of ’hungry’ 5 times, a subject of ’eat’ 6 times and argument of ’tasty’ 4 times, its vector is (5, 6, 4). If there are 5 occasions when something hungry buys something tasty, the value in the cell (1, 3) of the ’buy’ matrix is 5.
Using the described algorithm one can compute the inner product and cosine similarity of two sentences. The main feature of this methodology consists of its ability to compare sentences with di↵ erent grammatical structure due to reduction of sentences to one type.
Let us consider some simple text as an example how to get representations of nouns and verbs: ’Hungry predators eat tasty victims. Fast predators like tasty food. Small victims like tasty food. Big predator frighten tasty victims. Small victims don’t like big predators. Fast predators eat fast victims. Hungry victims eat tasty food. Hungry predators frighten big victims. Small predators eat hungry victims.’ Let the bases are ’arg-big’, ’arg-small’, ’arg-tasty’, ’arg-hungry’, ’arg-fast’. The vector for the noun ’predator’ is (2, 1, 0, 2, 3), the vector for the noun ’victim’ is (1, 2, 2, 2, 1). The matrix representation for the verb ’eat’ with respect to a given text corpus: 2 0 0 0 0 03 6 0 0 0 1 07 66 0 0 0 0 077 64 0 0 2 0 075
0 0 0 0 1
In large text datasets, these matrices are not so sparse. Let us manually define example of non-trivial distribution for nouns and matrix for ’eat’: Predators Victims Wolves Rabbits
Cij
Big Small Tasty Hungry Fast Big Small Tasty Hungry
Fast
With the help of these representations we can compare meanings of the two sentences: hS1 | S2i = hpredators eat victims | wolves eat rabbitsi = = X {hpredators | nii · Ciejat · hnj | victimsi⇥ ij
⇥ h wolves | nii · Ciejat · hnj | rabbitsi } = 148470, (1) where ni are standard basis vectors with ni[j] = 1 if j = i and 0, otherwise.
Normalising it by the product of the lengths of both sentence vectors leads us to the cosine similarity value q hS1 | S2i
q hS1 | S1i · hS2 | S2i = 0.958, which is close to 1 and, thus, our sentences have close meanings. 4
In the proposed compositional distributional model the meaning of a sentence is obtained by applying some function of the tensor products of the words’ meaning vectors. This function is a morphism corresponding to the grammatical structure of the sentence in the category of finite dimensional vector spaces of meanings. Basically it is a linear map of sentence’s structure and words’ meanings to the meaning of the sentence.
Such a linear map for the grammatical type reduction can be represented as
a diagram. For example, lets look at the examples presented in [
The idea behind the usage of diagrams is that we treat quantum logic as a theory of types of systems, processes and their interactions, and all the properties are also specified in terms of processes and their compositions. The types of systems are the objects in consideration, e.g. words in a sentence. Processes are morphisms of types, such as compound types and type reduction. And composition of morphisms form sequential application on processes. Then the diagrammatic calculus framework can be applied to reason on information processes.
The abstract categorical model of Coecke et al.[
The authors showed that on the task with intransitive verbs the categorical method performs on par with the other approaches. The other approaches are addition an multiplicative models, which basically are applications of meaning vectors addition and multiplication, correspondingly. There are still doubts on the small size of the context to be fully representative, however, considering more complex grammatical structure showed better performance in comparison to the other models. 6
There are several directions of quantum logic study in NLP area. We aim to compare the diagrammatic reasoning versus semantic modelling.
We first plan to implement the model and run it on available corpus data
in order to evaluate performance of the model and indicate possible complexity
issues for further optimisation. As authors of [
We would also try to extend the notion of the sentence meaning to a
definition of the paragraph meaning. On can use the concept of syntactic thickets
described, for example, in [
Acknowledgements. The work is supported by the Russian Science Foundation under grant 17-11-01294 and performed at National Research University Higher School of Economics, Russia.