Data analysts have to deal with a large number of complex and nested queries to dig out hidden insights from the relational datasets, spread over multiple files. Extraction of the relevant result corresponding to a given query can be easily done through a deep learnt NLQA framework, but to detect further explanations, facts, analysis and visualizations from queried output is a challenging problem. This kind of data analysis over query's result can be handled by Formal Concept Analysis, a mathematical tool that results in a concept hierarchy, makes semantical relations during the queries, and also can find the implications as well as asociations in the given dataset, can unify data and knowledge and is capable of information engineering as well as data mining. So for enabling NL analytics over such datasets for analysts, we present in this paper, a semantically enriched deep learning pipeline that a) enables natural language question answering over relational databases using a set of deep learnt sequence tagging networks, and b) carries out regularity analysis over the query results using Formal Concept Analysis to interactively explore, discover and analyze the hidden structure in the selected data [12] [11]. The deep learnt sequence tagging pipeline extracts information from the NL sentence and constructs an equivalent intermediate sketch, and then uses that sketch to formulate the actual database query on the relevant tables and columns. Query results are used in Formal Concept Analysis to create a lattice structure of the objects and attributes. The obtained lattice structure is then used to find exceptions in the data, classification of a new object and also to find the set of steps to transform the data from one structure to another structure.
Formal Concept Analysis provides a theoretical framework for learning hierarchies of knowledge clusters called formal concepts. A basic notion in FCA is the formal context. Given a set G of objects and a set M of attributes (also called properties), a formal context consists of a triple (G, M, I) where I specifies (Boolean) relationships between objects of G and attributes of M , i.e., I ⊆ G × M .Usually, formal contexts are given under the form of a table that formalizes these relationships. A table entry indicates whether an object has the attribute, or not. Let I(g) = {m ∈ M ; (g, m) ∈ I} be the set of attributes satisfied by object g , and let I(m) = {g ∈ G; (g, m) ∈ I} be the set of objects that satisfy the attribute m . Given a formal context (G, M, I) . Two operators () define a Galois connection between the powersets (P(G),⊆) and (P(M),⊆), with A⊆G and B⊆M:
That is to say, A is the set of all attributes which is satisfied all objects in A , whereas B is the set of all objects which satisfies all attributes in B . A formal concept of (G,M,I) is defined as a pair (A,B) with A∈G , B∈ M , A =B and B =A. A is called the extent of the formal concept (A,B), whereas B is called the intent.The set of all formal concepts of (G, M, I) equipped with a subconceptsuperconcept partial order ≤ is the concept lattice denoted by L. The and is defined as:
In this case, the concept (A 1 , B 1 ) is called sub-concept and the concept (A 2 , B 2 ) is called super-concept.
Given a formal context (G,M,I) there are extracted exact rules and approximate rules (rules with statistical values, for example, support and confidence).
These rules express in an alternative way the underlying knowledge. These rules are significant as they expresses the underlying knowledge of interaction among attributes.The exact rules are classified as implication rules while the approximation rules are classified as association rules. Definition Given a formal context whose attributes set is M. An implication is an expression S =⇒ T, where S,T ⊆ M. An implication S =⇒ T, extracted from a formal context, or respective concept lattice, have to be such that S ⊆ T . In other words: every object which has the attributes of S, also have the attributes of T. If X is a set of attributes, then X respects an implication S =⇒ T iff S ⊆ X or T ⊆ X. An implication S =⇒ T holds in a set {X 1 , ..., X n } ⊆ M iff each X i respects S =⇒ T. Definition Given a threshold minsupp ∈ [0, 1], where the support
association rules are determined by mining all pairs X =⇒ Y of subsets of M such that
is above the threshold minsupp, and the confidence
is above a given threshold minconf ∈ [0, 1].
We present a novel approach where a natural language sentence is converted into the sketch (Listing 1.1) which uses deep learning models and then further using the sketch to construct the database query (SQL) and fetch the output. This output is then taken to derive some explanations or interesting facts, find outliers or exceptions and rationalize the queried data if required (fig: 1). In order to generate the query sketch, we have a pipeline of multiple sequence tagging deep neural networks: Predicate Finder Model (Select Clause), Entity Finder Model (Values in Where Clause), Meta Type Model, Operators and Aggregation Model (all using bi-directional LSTM network along with a CRF (conditional random field) output layer), where the natural language sentence is processed as a sequence tagging problem. The architecture uses an ELMO embedding that are computed on top of twolayer bidirectional language models with character convolutions as a linear function of the internal network states [16]. Also the character-level embedding is used as it has been found helpful for specific tasks and to handle the out-ofvocabulary problem. The character-level representation is then concatenated with a word-level representation and feed into the bi-directional LSTM as input.
In the next step, a CRF Layer yielding the final predictions for every word is used [8]. We have Z = (z 1 ; z 2 ; ...; z n ) as the input sentence and P to be the scores output by Bi-LSTM network. Q i,j is the score of a transition from tag i to tag j for the sequence of predictions Y = (y 1 ; y 2 ; ...; y n ). Finally the score is defined as :
To generate the query sketch we use four different models using the same architecture (BiLSTM-CRF) [17] explained above, where the natural language sentence is processed as a sequence tagging problem. The neural network then predicts the tag for each word using which predicates, entities, and values in the sentence are identified, and an intermediate Sketch (independent of underlying database) is created. The Sketch is then mapped into the columns of the tables with conditions to construct the actual SQL query. In the sketch generation process the order of the models matters as the input of the next model depends on the output of previous model. To train the models, we had to create the annotations. In the cases where predicate/entities present in the sentence got the direct match with columns or values present in the actual database, we extracted them using a script and in the rest of the cases we have manually annotated the data.
- The models are trained independently and do not share any internal representations. However, the input of one model depends on the previous. For example, once predicates are identified we replace the predicate part in the NL sentence with some token before passing it to the next model. We capture this information from the NL sentence and create an intermediate representation (Sketch) which is further passed to the query generator(neo4j knowledge graphs), to construct the SQL or another database query and yields results. Result table of the query is then converted into its equivalent formal context, which is a triplet of objects, attributes and incidence relation between them. This formal context is used to extract the implication and association rules [10] and create a concept lattice which derives all possible formal concepts from the context and orders them according to a subconcept-superconcept relationship [15]. This conceptual hierarchy of the queried output is further used for knowledge discovery that is implicitly present in it. Here we are focusing on three types of analysis over queried data from a relational database.
This is first type of analysis that could be perform in the queried output. Outliers are defined as rules that contradict common beliefs. These kind of rules can play an important role in the process of understanding the underlying data as well as in making critical decisions. Outliers Analysis is to uncover the exceptions hidden in the given query output. To perform this over the queried output, we firstly created a preliminary formal context from the given raw data. Then by using Conexp tool [13], implication and association rules are generated for complete dataset. These rules shows the correlation among different attributes.
After the query is posed, concept lattice of the queried data is created and formal concepts in the form of (extent, intent) tuple are extracted from it. Intents of these formal concepts are then compared with the implication and association rules. If an intent of the queried output is violating any of the implication and association rules, then it is considered as an outlier for that query.
This is the second type of analysis that we introduced in our framework. Transformation analysis is used to measure two queries results, where tasks such as conversion of the underlying lattice structure of one set of query results into the lattice structure of another set of query results are required. This kind of analysis is performed by finding the difference between the intents of the formal concepts of both lattices. In our framework when two semantically enriched queries are posed, lattice structures of their respective outputs are generated.
To find the possible transformation requirements, we match the intents of both concept lattices and put down the differences between them. This gives us the disparity in the kind of objects contained in both the lattices which will help in transforming one lattice to another.
Classification analysis in our framework is done to predict the category of new objects. This is carried out by defining a target attribute t in the dataset, generating concept lattices C i for each value v i where i ∈ N of the target attribute and then comparing new object's attributes with the intents of each C i . In this analysis, a query asking for object details is posed. Lattice structures C i corresponding to each v i is stored in the memory. At the run time, matching of new object's attributes set is done with intents of each C i . If the intent of new object is contained in any one of the lattice C j for some j ∈ range(i), then the new object is classified under the corresponding v j category otherwise if more than one concept lattices contains the new object's intent then our framework cannot determine its category.
Census Income dataset taken from UCI machine learning repository [14] is used. This relational database contains 906 observations and 14 features of people like age, occupation, education, salary, workclass, native country etc. We construct the Neo4j knowledge graph from the csv ad also generated the implication and association rules. In this dataset we considered people names as the set of objects and applied conceptual scaling over the multivalued features mentioned above to generate the set of attributes where the objects and the attributes has a binary relation in between them.
Snapshot of the dataset is:
Implication and association rules extracted from data are:
-Adarsh works >60 hours per week with salary ≤ $ 50 K and Bachelors Degree.
-Arbella works >60 hours per week with salary >$ 50 K and is only 10th grade.
Query: What needs to be done to transform workclass, education and salary of men in Cuba to be like men in England?
Query: Predict that whether Aarav has diabetes or not from his blood pressure, body mass index and age. Based on the features of Aarav, it is predicted that he don't have diabetes.
We have described a framework wherein the NL sentence is semantically mapped into an intermediate logical form (Sketch) using the framework of multiple sequence tagging networks. This approach of semantic enrichment abstracts the low level semantic information from sentence and helps in generalising into various database queries (e.g. SQL, CQL). Answer of these queries are then further interpreted using FCA to find out outliers, facts and explanations, classifications and transformations. Experimental results shows that how NLQA and FCA can help an analyst in discovering regularities in a complex data.