A key objective of multidimensional dataset analysis is to reveal patterns of interest to users, but can be di cult to conduct due to the challenges of both presenting and navigating large datasets. This work explores how initial summarizations of multidimensional datasets can be generated (designed to reduce the number of data points which would need to be displayed), using summarization policies based on provided dataset values. Additionally, functionality for explaining the derivation of summarizations is being designed in line with prior work on aiding analyst interactions with data processing systems. To help drive development of this work, as well as provide illustrative use cases, we are presently designing a dataset summarization generator as part of greater work being done on an infrastructure for managing evidence of technical emergence in varying research disciplines via automated review of published materials.
A key objective of multidimensional dataset analysis is to reveal patterns of
interest to analysts. In many cases, these analyses will involve navigation over
a dataset to expose content likely to have interesting patterns. However,
multidimensional analysis has been observed to be challenging to analysts for the
following reasons [
This work explores how initial summarizations of multidimensional datasets
can be generated for consuming parties (designed to reduce the number of data
points which would need to be displayed) driven by summarization policies
based on provided dataset values. Focus has been given to RDF-based dataset
encodings, due largely to RDFs exibility in linking to outside data sources
(e.g., ontologies for expressing possible data values). Finally, functionality for
explaining the derivation of summarizations is being developed - in line with
prior work for aiding analyst interactions with data processing systems [
To help drive development of this work, as well as provide illustrative use cases,
we are presently developing a dataset summarization generator for the Abductive
Reasoning Based on Indicators and Topics of EmeRgence (ARBITER) system
being jointly developed by Rensselaer, BAE Systems, NYU, Brandeis and 1790
Analytics as part of IARPA's Foresight and Understanding from Scienti c
Exposition (FUSE) program. ARBITER's design objective is to scan for signs of
technical emergence in published literature - where technical emergence is
dened in the FUSE program as [
In ARBITER, sets of one or more evidence entries are evaluated to make hypotheses about emergence-related questions for a given topic and time period. For example: Has a practical application for DNA Microarrays been established in the time period of 2006-2010, based on the document collection PubMed-42?
In this setting, evidence entries are de ned as emergence indicators, calculated based on analysis over document collections. Indicators are classi ed according to an OWL ontology of indicator types, where each indicator is de ned to have at least one RDF type, as well as a set of numerical scoring metrics to de ne relationship of evidence to hypothesis. For brevity, an example is provided with ve indicators, each with a single RDF type and two numerical properties (value and relevance to the question answer, where a higher value is better).
Currently, these evidence entries are presented as a 2-dimensional
spreadsheet. To reduce the number of rows directly presented, policy-based
summarization techniques are being explored - deriving from established navigation
techniques in OLAP [
Ultimately, the following system conditions are assumed: (i) A maximum number of summary rows will be speci ed, which will appear in the presented summary; (ii) A pre-de ned collection of policies will be accessible by ARBITER, along with a pre-de ned ordering for their execution; (iii) Policies will be sequentially applied to the evidence set until the summary row count is reached, or all policies have been applied. Initially, an evidence dataset D0 will represent content directly generated by evidence gathering routines in ARBITER. Each policy execution will yield a transformed dataset view D1:::n, up until condition (iii) is satis ed.
While initial summarization can be a powerful aid for analyst users, care has to be taken in their usage, since one summarization strategy may not be appropriate for all users and information-seeking tasks. To help analysts keep track of applied strategies, summaries will be accompanied by explanations of their derivation - accessible for individual entries. In Figure 2, an example summary view - along with a supporting explanation - is provided.
System Development: ARBITERs summary generator is being designed to take three inputs: (i) A set of ne-grained evidence; (ii) A set of SPARQLencoded preference policies, along with an accompanying execution order; and (iii) Corresponding ontologies for encoding the preference and evidence data. For encoding evidence, we are now exploring use of the RDF Datacube2 vocabulary - given its support for representing multidimensional data.
Upcoming Directions: In upcoming work, focus will be given to the
following three issues: (i) selection of summarization policies which align with an
analysts perceived preferences, (ii) based on the summarization explanations
provided, enabling analysts to tweak applied strategies to generate new
summarizations, and (iii) enabling analysts to identify source documents used to create
evidence entries (similar to e orts discussed in [
We would like to thank our collaborators at BAE Systems, Sean Stromsten, Dan Hunter and Olga Babko-Malaya for their assistance in this work. Support has been provided by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior National Business Center contract number D11PC20154. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.
Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the o cial policies or endorsements, either expressed or implied, of IARPA, DoI/NBC, or the U.S. Government.