Knowledge representation of “open data” involves aggregation of disparate information in a semantically meaningful context. This task is challenging as such datasets are arbitrarily structured and fragmented with no overarching contextual framework in which the datasets are uploaded. The utility of such datasets is determined by the “context” in which they are presented and the same dataset can be viewed and consumed in various contexts depending on the consumer. We present open data from data.gov.in in 'Many Worlds on a Frame (MWF)' - a framework where knowledge is organized within one or more thematic worlds each of which in turn relate to one another to form the global knowledge frame.
A large chunk of open data is made available through open government initiatives such as data.gov1, data.gov.in2 etc mostly in the form of CSV files. Since open data is generated with no pre-conceived data models, there is no overarching data model that can be used for integrating such datasets. This is a non-trivial task and we call such problems as divergent aggregation problems. The semantic integration and aggregation process involves extracting the various contexts or themes along which these datasets can be integrated and representing the semantic integration in an appropriate framework that not only identifies different perspectives from which the data can be aggregated, but also depicts how the perspectives can be inter-related. In this paper we present a knowledge aggregation application using Many Worlds on a Frame (MWF) that allows for rich representation of data across two aspects, namely, the type hierarchy (is-a) relationship and the containment hierarchy (is-in) relationship supported by associations to transform the open datasets into a web of semantically interlinked themes and their associations.
To the best of our knowledge, our work is the first to enhance and extend the usage of LOD3 to Indian Open Government Data.
Many Worlds on a Frame (MWF) is an intuitive knowledge representation framework loosely modeled on Kripke semantics4. It allows for facts to be represented, grouped and related across many inter-connected worlds. Each world is considered a concept and concepts are organised in hierarchies, represented as rooted, acyclic graphs. Every concept belongs to two hierarchies - ‘is-a’ or concept hierarchy and ‘is-in’ or containment hierarchy. a ‘is-a’ b denotes a ‘is a kind of ’ b and a ‘is-in’ b denotes a ‘is contained in’ b. The concept hierarchy is used to inherit properties and associations and the containment hierarchy is used to manage visibility. The root of the concept hierarchy is a concept called Concept, and the root of the containment hierarchy is a concept called Universe of Discourse (UoD). A concept that cannot be subclassed using the ‘is-a’ relation, is called an Instance or a Record. Only ‘instance worlds’ store data, while ‘context worlds’ or ‘class worlds’ only manage structure and relationships. Each concept in a MWF system acts as a local ‘context world’ and hosts a set of knowledge fragments in the form of associations across concepts. ‘Class worlds’ can be imported into other worlds, so that their instances can participate as data elements. Associations are triples of the form (source, predicate, target). Here source and target are concepts in some target world say Cw and predicate is a label describing the association. In any association contained in world Cw, if the target concept is the world Cw itself, such associations are called Roles. The source concept is said to be playing a role defined by the predicate label in Cw. Roles, Associations and Worlds can be associated with zero or more attributes. An attribute is of the form (Key,Value), where Key is the name of attribute and Value holds the information regarding the attribute. Further, Value can hold literal data or a basic ‘type’. ‘type’ can be ‘String’, ‘URL’, ‘Date’ or a world. When a world is subclassed by another world, all the roles, associations and attributes are inherited by the sub classed world. 3
A separate model generates Thematic and Schematic integration outputs given a
collection of open data using heuristic algorithms over LOD [
Kripke Semantics: https://plato.stanford.edu/entries/logic-modal/
State dbo:Food
Market
Commodity
District
Food Commodity Variety
Roles
P State
District
Name
C Commodity
Date Min Price
Variety Yago/YagoPermanentlyLocatedEntity dbo:Food
State YaCgoo/mYmagoodPityermanentlyLocatedEntity
Yago/VarietiesOfRice Market
Market District :tableAids=s>ocAigamtioanrkARttircibeu2t0e1s2.csv
Yago/StatesAndTerritoriesOfIndia Yago/YagoPermanentlyLocatedEntity dbo:PopulatedPlace
Capital
Association Attributes :tableid=>IndianStates.csv Associations dbo:Food
Yago/VarietiesOfRice Capital
Variety
Roles
P
Name dbp:seat CoDnatteextVWaroierltdyMinSPtCarItimecep:otartbeledAidWs=so>orDMcAldiigaassmrttikroiaecnrttkARttiRrcibeou2let0es1s2.csv :tablAeisPds=ao>rceNinauttitAorPinsesnAottCctrioiabntVuitoaetenlnuste.csvPCahArialsdmsAeostCecsroiactiaiotionns dbr: dbpedia.org/resource dbo: dbpedia.org/ontology dbc: dbpedia.org/resource/Category Yago: dbpedia.org/class/yago dbp: dbpedia.org/property Fig. 1: Illustration of the working model of Many Worlds in a Frame (MWF) using open data tables most pertinent contexts for the collection of tables ( AgmarkRice2012.csv, NutrientContent.csv and IndianStates.csv ). These themes translate into ’context worlds’ in MWF. We have illustrated the components of the these two ‘context worlds’ in Fig. 1. Note that table AgmarkRice2012.csv constitutes a complex subject determined by the columns State and Commodity and has been consumed in two contexts. The context ‘dbo:Food’ shows the commodities that the various states sold while the context ‘YagoPermanentlyLocatedEntity’ shows the same table from the states perspective. The parent and child associations depicted by (State, Commodity) and (Commodity, State) in their respective contexts hold the complete semantics of the table AgmarkRice2012.csv. Similarly, table IndianStates.csv using the association Name, associates with the context ‘Yago/YagoPermanentlyLocatedEntity’. The table IndianStates.csv constitutes a simple subject with the anchoring column Name explaining all the columns of this table. The semantics of this table is explained by the association Name.
Currently, approximately 100 datasets from data.gov.in from various sectors such as ‘Agriculture’, ‘Health and Family Welfare’, ‘Environment’ etc have been aggregated using MWF. In the demonstration5, we will present “Sandesh” - the semantic data mesh of Indian Open Government Data. “Sandesh” seamlessly integrates the outputs from the ‘Thematic and Schematic integration’ model and populates MWF, given a collection of open data csv files. The demonstration is currently set up on a server with external IP and is powered by a SQLite database. During the demonstration, the implementation of 5
Demo: http://wsl.iiitb.ac.in/sandesh-web
Video: https://www.youtube.com/watch?v=pt1j2k1M97o MWF using datasets from data.gov.in will be presented using the ‘context worlds’ and ’instance worlds’ that have been inferenced from the datasets. Figure 1 and section 3 explain the MWF implementation in detail using a concrete example, that will be used during the demonstration.
In this paper, we have presented a rich and versatile application, that achieves semantic knowledge aggregation of open data. MWF intuitively represents the datasets as a semantic data mesh of interconnected worlds, roles, associations and attributes. The information from each table is represented in as many contexts as are applicable using ‘context worlds’. Information within each ‘context world’ is coherent and captures the various facets of an entity applicable within the boundary of that world. One can also traverse across inter-related worlds from a ‘context world’. We aim to integrate a reasoning engine in MWF to incorporate rules and reason new facts.
Other similar semantic integration efforts on open data include [