<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Analyzing Military Intelligence Using Interactive Semantic Queries</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Rod Moten</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Data Fusion</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>USA rod.moten@soteradefense.com</string-name>
        </contrib>
      </contrib-group>
      <pub-date>
        <year>2013</year>
      </pub-date>
      <abstract>
        <p>- We describe a strategy for performing semantic searches for analyzing military intelligence. Our strategy allows the analyst and the query engine to work together to reduce a complex query into simpler queries. The answers for the simpler queries are combined into answers for the original query. The queries can be refined using rules defined by the analyst or analytics created by a data scientist. Our strategy uses an alternative approach to semantic modeling than the state-of-theart approaches based on OWL. OWL is an implementation of a branch of mathematical logics designed specifically for semantic modeling called description logics. Our strategy uses a branch of mathematical logics called type theory. We use type theory because of the long history of developing systems based on type theory for reasoning interactively. We demonstrate with an example how the strategy can be used to answer questions posed by analysts that couldn't be answered using conventional methods.</p>
      </abstract>
      <kwd-group>
        <kwd>semantic search</kwd>
        <kwd>military intelligence</kwd>
        <kwd>analytics</kwd>
        <kwd>type theory</kwd>
        <kwd>ontology</kwd>
        <kwd>semantic modeling</kwd>
        <kwd>interactive theorem proving</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        INTRODUCTION
"The Army is working closely with the intelligence
community and other Defense Department partners, including
the Navy, in developing cloud-based systems for battlefield
intelligence."[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] The goal of the U.S. Army is to fulfill theater
intelligence requirements using these systems as much as
possible [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. For example, suppose an analyst created a
hypothesis that a family within an Afghan village is
responsible for several IEDs. The analyst may use the Cloud to
determine which families have connections to hostile
organizations. The data may be in the Cloud that directly links
a family to a hostile organization. For example, suppose the
Sadat Baba family [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] is a member of the village and
intelligence data contains the triple (Sadat Baba shares-profit
Taliban). In the triple, Sadat Baba is the subject, shares-profit
is the predicate and Taliban is the object. On the other hand,
the intelligence data may only contain data that indirectly links
the family to a hostile organization. These links have to be
inferred either deductively or inductively from the data. For
example, it may be possible to infer that Sadat Baba and the
Taliban have common interests because both Sadat Baba and
the Taliban attacked the Dalazak family. This could be inferred
by applying the following rule. If a family and an external
organization attack another family in the same tribe or village,
then the external organization and the attacking family have
common interests. Or it could be inferred using network
analysis because Sadat Baba and Taliban both are linked to
Dalazak by the same relationship within the same subgraph.
      </p>
      <p>
        The current state-of-the-art for military intelligence
analysis focuses on the analyst using visual aids and various
retrieval techniques, such as faceted search and querying, to
perform this inference manually [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Military intelligence
analysis systems should focus on developing strategies for
reducing the manual work performed by analyst by
incorporating more automated methods. The effort for
searching for data can be reduced if the query engine
automated some of the work the analysts performed manually.
This means the query engine would need to have analytics that
automate some of the inductive and deductive reasoning
performed by the analyst.
      </p>
      <p>
        Some of the analytics used in a query or search may be
different from the analytics used in ETL (extraction,
transformation, and load). In ETL, analytics are used primary
for entity and feature extraction, entity resolution, and entity
fusion [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. In this case, the analytics aren't used to answer a
query posed by an analyst. The analytics we are interested in,
such as multi-relational link predication [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], occur after
ETL. The analytics will be performed after the data has been
mapped into a graph-like structure, such as RDF or DRIF [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ],
[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Therefore, the query engine will need the ability to express
the behavior of the analytic in terms of the underlying semantic
network.
      </p>
      <p>The behavior of an analytic can be specified as the logical
implication of the postcondition from the precondition. The
precondition is a logical statement that must be satisfied in
order for the analytic to produce valid output. The
postcondition is a logical statement that describes the
characteristics of the concepts and relationships produced by
the analytic. In the simplest case, the specification of an
analytic could be defined as ! → ! , where ! is the
precondition and ! is the postcondition. For example, the
precondition of the analytic for associating a family and an
external organization would be 'for any ? ! ∈ Family and
? ! ∈ National(Organization there exists a ? ! ∈ Family such
that ? !!attack!? ! and ? !!attack!? ! '. The postcondition
would be ' ? !!common!interests!? ! '.</p>
      <p>
        The examples above may mislead the reader to believe that
first-order logic would be sufficient for expressing the
precondition and postcondition. However, first-order logic
doesn't support cases when analytics can operate on a set of
concepts and roles. For example, consider an analytic that uses
numeric calculations to determine relationships, such as
common neighbor algorithms [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Such an analytic only cares
about relationships or edges between nodes. So it can operate
on any concept. For example, if the analytic uses a common
neighbors algorithm for determining the common-interests
relationship, then the precondition would be 'for any ? ! ∈
? ! and ? ! ∈? ! there exists a ? ! ∈? ! such that ? !!? !!? !
and ? !!? !!? ! '. In this precondition, we parameterized the
concepts for the attackers, Family and National Organization,
and we parameterized the attack relationship. We can also
parameterize the postcondition. For example, a data scientist
may be able to improve the analytic to infer a stronger
relationship between the attackers using additional information
about them. In this case, the postcondition would be 'there
exists a ? ! ⊆ common!interests such that ? !!? !!? ! '.
      </p>
      <p>The query engine could use the specification of an analytic
as a rule for solving an intelligence requirement. If the
specification of an analytic is ! → ! and the intelligence
requirement can be reduced to !, then the solution to the
intelligence requirement would be the concepts and
relationships that the analytic produces that satisfy !.</p>
      <p>In this paper, we present a strategy for analyzing
intelligence data using an interactive query language. When a
user specifies a query, the query engine solves the query by
refining it into new queries. If any of the new queries cannot be
answered, it asks the user to assist it. The user assists the query
engine by specifying an analytic or rule that can solve the
query or reduce the query into new queries. This process
continues until all queries are answered or until there is a query
that cannot be answered. The query engine keeps track of this
process and combines the answers from the generated queries
into an answer for the original query. At the heart of the query
language is the type theory TT-IQ, Type Theory for Interactive
Querying. The query engine for TT-IQ consists of a
framework that allows a data scientist to define analytics that
can be included in query processing and for analysts to add
new rules.</p>
      <p>This paper is outlined as follows. First, we present work
related to our strategy. Then, we use an example to
demonstrate interactive querying. Next we give an overview of
TT-IQ. Finally, we conclude the paper with a discussion of our
strategy.</p>
      <p>II.</p>
    </sec>
    <sec id="sec-2">
      <title>RELATED WORK</title>
      <p>
        Our strategy is similar to approaches that use a semantic
network or ontology for refining queries. These approaches,
such as QUICK [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], LISQL [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], and query rewriting [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] use
semantic information to enhance a query supplied by a user.
These approaches use a semantic network and stepwise
refinement to create semantic queries. Our approach, on the
other hand, uses stepwise refinement for query execution.
      </p>
      <p>
        Researchers at GMU have spent over 15 years developing
strategies that could be used for interactive querying [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]–[
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
Their approaches use heuristics to perform inductive and
deductive reasoning. They also use machine learning to find
new rules to add to the knowledge base. Our strategy support
inductive and deductive reasoning except we use
prooftheoretic methods used in interactive theorem provers.
Theorem provers, such as NuPrl [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ], Coq [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], and Isabelle
[
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], use interactive methods for developing formal
mathematical proofs. In these systems, the assertions are type
judgments. Type judgments are logical statements that ask
which objects belong to a specific type. The types can be
defined to resemble logical statements, such as ! ∧ ! → !. We
use this same technique in our query language. However, our
type theory differs from the state-of-the-art in order to support
semantic modeling.
      </p>
      <p>
        K-DTT [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] and S-DTT [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] are type theories that use an
extensional approach to semantic modeling. Description logics
also use an extensional approach to semantic modeling. This
means that A-Box statements or an external source, such as a
database, has to be used to determine that an object belongs to
a concept. In TT-IQ, concept membership is intensional. In
other words, we determine whether an object belongs to a
concept based on how the object was constructed. Therefore
there is no need for A-Box statements or external sources to
determine concept membership.
      </p>
      <p>
        Type theory isn't the only method of using higher-order
logic for semantic modeling. Classical higher-order logics have
been used for semantic modeling as well [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]–[
        <xref ref-type="bibr" rid="ref26">26</xref>
        ]. However,
the query languages used in these approaches do not allow a
user to define analytics to be incorporated into the query
engine.
      </p>
      <p>
        Rule-based approaches for semantic search, such as
TupleGenerating Dependencies [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ], have the capability to
incorporate rules specified by a user into the query answering
process. However, these approaches are implemented using
first-order reasoning techniques from logic programming. As a
result, they will not be able to support domain metamodeling.
Our strategy, on the other hand, will support finding concepts
and relationships that meet specific criteria.
      </p>
      <p>III.</p>
    </sec>
    <sec id="sec-3">
      <title>INTERACTIVE QUERYING</title>
      <p>In this section, we give an overview of how interactive
querying is performed. Interactive querying is analogous to
proving a type judgment. Informally, a typing judgment
consists of a goal and a set of assumptions. The goal is the
assertion we want to prove. The assumptions represent facts
and statements from the knowledge base. Therefore, it consists
of logical statements about the semantic meaning of concepts
and relationships.</p>
      <p>Fig. 1 shows an example proof created from an interactive
query for attacks in a region that may have involved a specific
person. We assume the analyst only knows that the person has
a set of features observed by an interrogator, such as facial
marks, height, and the number of a cell phone belonging to the
detainee. This query is stated as a judgment that appears as the
root of the proof tree. Here we state the judgments informally
in natural language. The assumptions in the judgments in Fig. 1
contain a definition of a type representing the concepts Attack
and Person; an object representing the features of the detainee;
features of the detainee; declarations of the predicate symbols,
such as involving; and definitions of data types that represent
cell phone numbers and regions. The assumptions also contain
a taxonomy of properties and features. The taxonomy is
defined as a partial ordering on predicate symbols and attribute
names.</p>
      <p>
        The query engine determines which tactics can be applied
based on whether the conclusion of a tactic matches the goal of
a judgment. The query engine uses the type compatibility
relationship [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] of TT-IQ to perform matching. When two
types !and !′!are compatible, it means that an object in ! can
be converted to an object in !′ and vice versa. Type
compatibility is an extension of the subtyping relationship in
TT-IQ. We give an overview of subtyping in "Overview of
TTIQ".
      </p>
      <p>Each tactic has a conclusion and zero or more antecedents.
When a tactic matches a judgment, the query engine creates
new judgments for each antecedent. For example, the Narrow
Concept tactic is a built-in tactic that replaces a type ! in a
term with a subtype of !. This tactic has two antecedents. One
antecedent is a judgment that asserts the replacement type is a
subtype of the substituted type. The other antecedent is the
same as the conclusion except all occurrences of the supertype
are replaced with the subtype. Fig. 1 only shows the second
antecedent because the first antecedent can be proved
automatically using TT-IQ's subtyping relationship.</p>
      <p>In practice, the query engine will only show the judgments
that require assistance from the user. There may be multiple
tactics that can be applied to a judgment. The compatibility
relationship can rank the tactics that match best. However, if
two tactics have the same rank, then the user will need to select
the tactic to apply.</p>
      <p>Narrow Predicate is similar to Narrow Concept, except
Narrow Predicate replaces a predicate symbol with another
predicate symbol. Therefore, Narrow Predicate has antecedents
to prove that the replacement predicate is a sub-property of
substituted predicate. This requires two antecedents: an
antecedent to prove that the replacement property is a
subproperty of the existing property and an antecedent to prove
that the type of the sub-property is a subtype of the type of the
super-property. The first antecedent has to be proved using the
taxonomy of the properties and features. The tactic also has a
third antecedent that contains the sub-property instead of the
super-property. The proof tree in Fig. 1 shows the judgment
produced from the third antecedent. In particular, it shows
"involving" replaced with "near".</p>
      <p>The judgments produced by Concept Introduction illustrate
the need for dependent judgments. Normally in type theory and
in sequent calculus, judgments of the same parent are
independent of each other. However, when using interactive
proofs to query for data, terms created on one branch could be
used in a judgment on a different branch. Notice that two of the
goals produced by Concept Introduction contain ≔ . This
special constructor informs the query engine to create a
reference to the term that satisfies the type on the right hand
side of≔. At some point, a tactic will be invoked that uses an
analytic to create objects or find objects in the knowledge base
to bind to the reference.</p>
      <p>The Geo Search tactic uses an analytic to bind a reference
to a collection of terms that are within a specific region. Geo
Search uses the goal as the criteria to search for objects within
a specific region in the knowledge base. More specifically, Geo
Search has a conclusion that has a type that contains one
attribute, location. The type of location is the supertype of all
types that could be used as the criteria for a geospatial search,
such as KML. This type matches any type that has an attribute
that is compatible with KML and the name of the attribute is
interchangeable with location. The taxonomy determines which
attributes are interchangeable with location. The tactic uses this
attribute as the search criteria. In practice, the tactic may also
require a time range.</p>
      <p>Person Observation is a tactic that is created by a data
scientist. In other words, it is an analytic that creates concepts
inductively. This means it creates the definition of a type by
generalizing existing data. Let's assume that Person
Observation examines SIGINT data for calls originating form
or made to the telephone number of the cellphone in the
possession of a detainee. The time and location of the each call
is used as an observation point of the person. In theory, a new
triple is added to the knowledge base that links the person to
the time and location of the call. In practice, the query engine
may not create the triples. Instead, the query engine may define
a way to create the triples on demand without altering the
knowledge base unless directly instructed to do so. This
approach is essential for cloud-scale data because it doesn't
perform any destructive modifications if the analyst wants to
back out changes. Instead the query engine could have a local
cache containing the new triples. The modifications could be
made permanent only when explicitly specified by the analyst
or a data scientist.</p>
      <p>The antecedent and conclusion of a tactic based on an
analytic is determined by the precondition and postcondition of
the analytic. The system uses the precondition as the
antecedent and the postcondition as the conclusion. For Person
Observation the postcondition states that there exists a concept
! where each object is a Person that has the features of the
detainee and whose locations correspond to the locations of the
cell phone. The precondition requires that there exists a person
in the knowledge base who has the features of the detainee. To
satisfy the precondition, the analyst will need to add the
detainee to the knowledge base. The precondition also requires
that we can determine the locations of the cell phone. We
assume another analytic will determine this location. For
brevity, assume that the query engine can prove this
automatically. As a result, it isn't included in Fig. 1.</p>
      <p>We envision a suite of tactics that discover relationships
between an entity and an event. These tactics use analytics that
can create new relationships. In other words, they can add new
triples to the knowledge base. Nearness Predicate belongs to
this suite. Nearness Predicate uses an analytic that creates new
triples using the near predicate. In other words, it creates triples
of the form (! near !) where ! is an entity and ! is an event.</p>
      <p>IV.</p>
      <p>OVERVIEW OF TT-IQ</p>
      <p>In this section, we give the formal definition of TT-IQ. Due
to space limitations, we omit some rigor that would be found in
a normal presentation of type theory.</p>
      <p>Both types and objects are terms. We define the terms !
and ! as follows.
!, ! ≔
⊥ !! !|! ! ! !!, … , !! !. ! !.size
tt|ff !! = !!, … , !! = !! |[!!, … , !!!]!!
!! ∧ !! ! ∨ !! ! → !!|¬!|! !!, … , !! prop
!! ℛ !!: !!× …×!!!: !! !∗| !: !|!′
|∀!: !. !′|∃!: !. !′!|∀! ≤ !. !!|!∃! ≤ !. !!|!!</p>
      <p>In the definition of terms, ! and ! range over strings and
numbers respectively; ! ranges over attribute names; and !
and ! range over variables. The term ⊥ represents null. The
terms tt and ff represent true and false, respectively. The terms
of the form !! = !!, … , !! = !! represent records. Each
!! = !! in a record represents an attribute where !! is the name
of the attribute and !! is the value of the attribute. The terms of
the form !. ! represent selecting the value of an attribute whose
name is ! from a record ! . Terms of the form [!!, … , !!!]
represent lists. Terms of the form !.size represent the number
of elements in the list !. ! and ! range over predicate symbols
and function symbols, respectively. ! is the type of strings and
ℛ is the type of numbers. We call the terms ! , ! , ! ,
!(!!, … , !!) , !. ! , !.size , tt , ff , !! = !!, … , !! = !! , and
[!!, … , !!!] objects. We call all of the other terms, such as
!: !|!′ and ∃!: !. !′, types.</p>
      <p>Terms of the form !!: !!× ⋯×!!!: !!represent record types
and terms of the form !∗represent list types. The type p ro p
represent the type that contains types that represent logical
formulas, such as ! ∧ !′ and ! !!, … , !! . Any type created
using terms in p ro p will also be in p ro p . For example
grt! 5,4 ∧ !grt! 7, 3 is a member of p ro p . Terms of the form
!: !|!′ represent set types. Intuitively a set type
!: !|!′ !represents a list of objects of type ! where each
member of the list makes the type representing the logical
formula !′ true. We call ! a reference. We use ! to support
injecting terms created by an analytic running in a separate
subsystem. Notice there are two kinds of quantifiers, those that
range over objects,!∀!: ! and ∃!: !, and those that range over
types, ∀! ≤ ! and ∃! ≤ !. In the quantifiers that range over
objects, !: ! means ! is a member of type !. So, ∀!: ! means
for all terms ! that are members of the type ! . In the
quantifiers that range over types, ! ≤ ! means ! is a subtype
of !. So, ∀! ≤ ! means for all types ! that are subtypes of !.</p>
      <p>Notice that types may contain objects. For example, if grt
is a predicate symbol that represents greater than equal to and
abs is a function symbol that represents absolute zero, then
!: ℛ|grt abs ! , 0 is a type that contains the objects 0 and
abs(!).</p>
      <p>Given any two terms ! and !′ and a variable !, ![!!/!] is
the term produced by replacing all free occurrences of ! in !
with !′. For example, ! ! !, ! , ! 3/! is ! ! 3, ! , 3 . For
two terms ! and !, we write !: ! to mean that ! inhabits the
type ! or ! is a member of the type !. For example, 3: ℛ and
3,4,5 : ℛ∗ . We give some of the rules of TT-IQ for
determining which terms inhabit a type in Fig. 4.
!! ⇀ !!!
⋯</p>
      <p>!! ⇀ !!!
(!! = !!, … !! = !!) ⇀ (!! = !!! , … , !! = !!!) (REC. ⇀)
! ⇀ (!! = !!, … , !! = !!)</p>
      <p>! = !!
!. ! ⇀ !!</p>
      <p>(FIELD!SELECTION ⇀)
!! ⇀ !!!
⋯</p>
      <p>!! ⇀ !!!
[!!, … , !!] ⇀ [!!! , … , !!!] ! (LIST ⇀)!
! ⇀ [!!, … , !!] (LIST!SIZE ⇀)!</p>
      <p>!.size ⇀ !
ℑ!!(!!, … , !!)! ↦ !</p>
      <p>!(!!, … , !!) ⇀ !
ℑ!!(!!, … , !!)! ↦ !
!(!!, … , !!) ⇀ !
(FUNCTION!APPLICATION ⇀)!
(PREDICATE!APPLICATION ⇀)
ℑ!!!! ↦ !
!! ⇀ !</p>
      <p>(REFERENCE ⇀)!
Fig. 2. Evaluation rules for terms that do not evaluate to themselves.</p>
      <p>The specification of an analytic can be generalized as
follows. The specification will need to contain universal
quantifiers to allow the query engine to pass in types and
objects to the analytic. If the analytic takes in ! types and !
objects, then the specification will need the quantifiers
∀!! ≤ !!. ⋯ . ∀!! ≤ !! and ∀!!: !!. ⋯ . ∀!!: !! . We
abbreviate these as ∀! ≤ ! and ∀!: ! , respectively. An
analytic may output types and objects. If an analytic generates
! types and ! objects, then the specification will need to
contain existential quantifiers ∃!! ≤ !!! ⋯ ∃!! ≤ !!! and
∃!!: !!!. ⋯ . ∃!!: !!! . We abbreviate these as ∃! ≤ !! and
∃!: !!, respectively. The specification will also contain the
precondition and the postcondition of the analytic. We denote
these respectively as ! and !. The general term that represents
the specification of an analytic is specified in (1).</p>
      <p>∀! ≤ !. ∀!: !. ! → ∃! ≤ !!. ∃!: !!. !</p>
      <p>In practice, the number of inputs and outputs of an analytic
will be small. For example, the specification of Geo Search, an
analytic that finds objects that occur within a region, is as
follows.</p>
      <p>∀! ≤ location:KML . ∀!:KML.true</p>
      <p>→ ∃!: !∗. ∀!: !.inRegion !, !</p>
      <p>In the specification of an analytic, the !'s and !'s in (1)
represent concepts and the !'s and !'s represent individuals. An
analytic may also take relationships as input and output
relationships. The relationships an analytic takes as input are
defined by !, and the relationships an analytic produces are
defined by !. For example, the Geo Search analytic outputs a
relationship inRegion !, ! . The domain is defined by the type
of ! which is KML and the range is all subtypes of
(1)
(2)
location:KML . Geo Search doesn't take any relationships as
input.</p>
      <p>Traditionally, the definition of a type theory includes rules
for evaluating terms. As a result, most type theories in the
literature are definitions of statistically typed functional
programming languages. TT-IQ has rules for evaluation. We
list a subset of the rules in Fig. 2. TT-IQ should not be
considered a functional programming language. TT-IQ does
not contain a construct for creating functions. Instead, TT-IQ
defines a means for injecting functions and relations into it that
are executed by an external subsystem or programming
language. We represent the external subsystem or
programming language as an interpreter. For any term !, an
interpreter maps ! to a term !′ . We use ℑ to denote an
interpreter. We write ℑ ! ↦ !′ to mean that ℑ interprets ! as
!′ . The interpreter is used to evaluate the application of
function symbols and predicate symbols. For a term !, we
write ! ⇀ !′! to mean ! evaluates to !′ . The rules
FUNCTION!APPLICATION ⇀ and PREDICATE!APPLICATION ⇀ indicate
that the application of a function symbol or a predicate symbol
to a sequence of terms is equal to the interpretation of the
application of the function symbol or predicate symbol. These
rules do not require that the arguments of the function symbol
and predicate symbol be evaluated before passing them to the
interpreter. As a result, the interpreter can determine the mode
of evaluation, such as lazy evaluation or eager evaluation. The
evaluation rule REFERENCE ⇀ illustrates the ability of the
interpreter to manage storage of instance data. Intuitively
REFERENCE ⇀ means that a reference to a concept evaluates to
the set of individuals in the concept. The concept is defined by
the type ! and ! is a pointer to an index, graph or other data
structure that contains members of the concept.</p>
      <p>A paramount feature of TT-IQ is subtyping. TT-IQ uses a
unique feature to typing that is required for semantic modeling.
Traditionally, subtyping on record types is the same as class
inheritance. In other words, a record type ! is a subtype of !′ if
all the attributes in !′ are also attributes in !. This means that
for every attribute named ! in !′ there is attribute ! in T.
However, this definition of subtyping ignores the semantic
meaning of attribute names. Two attributes can have different
names, but mean the same thing. As a result, TT-IQ defines
subtyping so that attribute names do not have to be
syntactically the same, but semantically the same. Actually, in
TT-IQ attribute names do not need to be equivalent, but one
attribute name has to be subsumable by the other. The
definition of subtyping in TT-IQ uses a partial ordering on
attribute names to define subtyping. Due to space limitations
we are unable to define the complete definition of subtyping.
We do show the subtyping rule for record types below.
! ≥ !
for!! = 1, … , !</p>
      <p>′
!! ⊑ !!
!! ≤ !′!
!1: !1× ⋯×!!: !! ≤ !′1: !′1× ⋯×!′!: !′!</p>
      <p>In "Interactive Querying" we showed examples of tactics.
A tactic is a program or script that returns a proof tree. A proof
tree represents a proof of a type judgment. Intuitively a type
judgment asserts that a term belongs to a type. A type
judgment has the form ! ⊢ !: !′ where ! represents an
environment and !: !′ represents the assertion that ! inhabits
!′. The environment, !, of a type judgment consists of type
assignments, terms, predicate symbols and function symbols. It
also consists of pairs of predicate symbols and attribute names.
These pairs form a partial ordering, ⊑. If !!, … , !! are type
assignments of terms, predicate symbols, and function
symbols, then ⊑; !!, … , !! is an environment.</p>
      <p>Only the rule engine can execute a proof rule. The rule
engine is a subcomponent of the query engine. Given a rule, a
proof tree, and judgment in the proof tree, the rule engine will
apply the rule to the judgment. The result will be a proof tree
that uses the antecedent of the rule to create children of the
judgment. Fig. 3 illustrates the relationship between the query
engine, the rule engine and the interpreter.</p>
      <p>TT-IQ does not define the language in which tactics are
written. An interpreter performs evaluation of tactics. Detailed
discussion of the language for creating tactics is outside the
scope of this paper. In this paper it suffices to say that a tactic
takes as input a typing judgment and outputs a proof tree. The
root of the tree has to be the input judgment. The tactic has to
use proof rules to create the proof tree.</p>
      <p>Fig. 4 contains a few proof rules for TT-IQ. We use
∃!!ELIMINATION to specify that an analytic will retrieve the
individuals of a concept represented as !!or a subtype of !.
The rule uses a concept reference so that we can postpone
selection of the analytic until we have a judgment whose type
matches the postcondition of an analytic. We use ANAYLIC!EVAL
to prove judgments that require an analytic. ANAYLIC!EVAL
generates concepts and individuals that satisfy a condition
specified as a type. The condition can represent a relationship
or the search criteria for a query. The condition is specified as
the type ! in ANAYLIC!EVAL. An analytic whose postcondition
matches ! is used to generate the concepts and individuals that
satisfy the condition specified as !. In the rule, the analytic is
! and its postcondition is !. The top-right hypothesis is used to
establish that ! matches ! if replacing the free variables in !
produces a term that is a supertype of !. The terms that
replace the free variables are the inputs to the analytic, ! and
!, and the outputs of the analytic, ! and !. ! is a sequence of
types !!, … , !! and ! is a sequence of objects !!, … , !! .
Likewise ! is a sequence of types !!, … , !! and ! is a
sequence of objects !!, … , !! . Intuitively, ! and ! are the
concepts and individuals required by the analytic to produce
the concepts and individuals ! and ! that satisfy the
relationship defined by !. All free variables in each !! and !!
are declared in !. Each !! and each !! do not have any free
variables.</p>
      <p>Since ! and ! represent the input to the analytic, we need
to verify they satisfy the precondition !. The three hypotheses
of ANAYLIC!EVAL on the left achieve this. The top two
hypotheses are judgments to verify that the inputs to the
analytic have the correct types. The last of the three hypotheses
verifies that ! is true with all of its free variables replaced with
inputs to the analytic. The top hypothesis on the right is used to
verify that the postcondition is true for the inputs and outputs
of the analytic. The second hypothesis from the top on the right
indicates that the output of ! on ! and ! produces ! and !.
Recall from FUNCTION!APPLICATION ⇀ in Fig. 2 that an
interpreter is used to produce ! and !. The hypothesis on the
bottom-right indicates that each !! evaluates to a list of terms
of type !! and that each !! evaluates to a term that is of type
!!. This hypothesis shows that we intend to represent concepts
!
as list of terms of a specific type. The ∃{}!REWRITE rule makes
use of the fact that for any term !, if !: !: !|!! then !: ! and
! ⊢ !!!⃗ ≤ !!!!⃗
! ⊢ !!⃗: !!!⃗
!, !! ≤ ! ⊢ ![!!!/!]:prop
! ⊢ ∃! ≤ !:prop</p>
      <p>! (∃!!ELIMINATION)
! ⊢ ∃! ≤ !. (∀!: !. !!) ∧ !!!:prop
! ⊢ ∃! ≤ !!: !|!!!!′!. !!!:prop
(∃{}!REWRITE)
! ⊢ !! ≤ ![ !!!!⃗/ !!⃗][!!!⃗/!⃗][!⃗/!⃗][ !!⃗/!⃗]
!( !!⃗, !⃗) ⇀ ! !!!!⃗, !!!⃗!
! ⊢ ![!⃗/!⃗][ !!⃗/!⃗]:prop !!!! ⇀ !!!!!"#!! ⊢ !!: !!∗
!! ⇀ !!!!!!"#!! ⊢ !!!: !!!
!, !: ∀!⃗ ≤ !!⃗. ∀!⃗: !!⃗. !! → ∃ !!⃗ ≤ !!!!!⃗!. ∃!⃗: !!!!⃗!. !!, !! ⊢ !:prop
! ⊢ ! ≤ ! ! ⊢ ∃! ≤ !. !!:prop
! ⊢ ∃! ≤ !. !′:prop!
! (NARROW!TYPE)
(ANAYLIC!EVAL)
!! !!!"#$%&amp;'!!"#$%&amp;'(!!"#$$%&amp;' ! !! ! !!!"#!!"#$%!&amp;" !! ! ! !!
! ! !!!"#$%&amp;! ! !!!"#$%&amp;"'! ! ! !"#$ !! ! ! !!! !! !!! !!!"#$!!! !!!
!
!! !!!"#$%&amp;'!!"#$%&amp;'(!!"#$$%&amp;' ! !! ! !!"#!!! !! !"#$%!&amp;" !! ! ! !!
! !!!"#$%&amp;! ! !!!"#$%&amp;"'! ! ! !"#$ !! ! ! !!! !! !!! !!!"#$!!! !!!</p>
      <p>Recall in Fig. 1, we used the Narrow Concept tactic to
replace Attack with IED. Narrow Concept contains code to
select the appropriate concept to use as the subtype for the
Narrow Type rule. After it finds a subtype, it asks the rule
engine to apply Narrow Type to a target type judgment in a
proof tree. The rule engine creates a new judgment using the
antecedent of Narrow Type as a template. The new judgment is
added as a child of the target type judgment. The rule engine
returns the new proof tree to Narrow Concept and Narrow
Concept returns the proof tree to the query engine. Fig. 5
contains a portion of the proof tree created from the Geo
Search and Concept Introduction tactics in Fig. 1.</p>
      <sec id="sec-3-1">
        <title>A. Believability</title>
        <p>
          A query engine for intelligence data needs to support
various levels of believability. We can support believability in
TT-IQ by using type modality [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]. In other words, we can
annotate type with a modality operator that represents a level
of believability, such as certain, lik e ly , n o t!lik e ly , and
im p o ssib le . Then an analyst could prove a hypothesis
represented by the type ! is likely to occur as the judgment
! ⊢ !!"#$!%!: !"#! or disprove it by proving the judgment
! ⊢ !!"#$%%!&amp;'(!: !"#!.
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>B. Too complex for an analyst</title>
        <p>
          The formalism of TT-IQ and the method of reasoning
employed by TT-IQ may be too complicated for an analyst.
We don't expect the analyst to specify queries in the formal
language of TT-IQ, but in natural language similar to that used
in Fig. 1. We could employ a technique similar to that used in
[
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] to allow end users to specify queries using natural
language.
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>C. Non-determinism and Subtyping</title>
        <p>Since a type may have multiple subtypes, a tactic that finds
or creates a subtype of a type could be nondeterministic. In
other words, the tactic may not produce the same subtype for
the same supertype. As a result, the query engine could
produce different results for the same query over the same data.
We can resolve non-determinism by asking the user to select
the appropriate subtype. This approach would be similar to
faceted search. The query engine would require the end user to
select from a list of subtypes to use as a candidate to narrow
the search space.</p>
      </sec>
      <sec id="sec-3-4">
        <title>D. Implementation of TT-IQ</title>
        <p>Currently, we are in the planning stage of creating an
implementation of TT-IQ. We plan to create an implementation
of TT-IQ using Coq and R. Coq will provide the interactive
reasoning capability. R will be used as the language and
runtime for defining and executing analytics. We anticipate
having to add a library to R to provide a more seamless
interaction with RDF than the existing R libraries.</p>
        <p>We consider this implementation of TT-IQ a
proof-ofconcept implementation. We plan to use this implementation to
conduct research to address usability issues and determine
strengthens and weaknesses of interactive semantic querying
over automatic semantic querying.</p>
        <p>The analyst workstation will contain an analytic framework
that would provide an interface to support contribution of
analytics written in a wide range of languages, such as
MATLAB, C, Java, and Python.</p>
        <p>VI.</p>
        <p>CONCLUSION</p>
        <p>In this paper, we showed how to apply techniques from
ITPs (interactive theorem provers) to analyze military
intelligence. Users of ITPs apply small programs called tactics
in an iterative fashion to construct a proof. We demonstrated
how tactics could be used to answer semantic queries
interactively. Furthermore, we showed how to incorporate
analytics that use machine learning, knowledge discovery, or
network analysis into the querying process.</p>
        <p>A. Future Work</p>
        <p>
          In "Believability", we eluded to an approach to handle
uncertainty. Future work should investigate this approach.
Also, we should consider how to incorporate the approach in
[
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] into our type system.
        </p>
        <p>In the future, we would like to investigate how to
implement interactive querying in an existing military
intelligence cloud system, such as the DCGS-A Cloud. We
believe our approach to querying would be a good fit for the
semantic enhancement approach adopted by the DCGS-A
Cloud.</p>
        <p>ACKNOWLEDGMENT</p>
        <p>Rod Moten thanks Andy Zhao of Sotera Defense Solutions
and Dr. Aaron Stump of the University of Iowa on their
valuable feedback on the ideas expressed in this paper.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Wylie</given-names>
            <surname>Wong</surname>
          </string-name>
          , “
          <article-title>The Army Brings the Cloud to the Battlefield,”</article-title>
          <source>FedTech Magazine</source>
          ,
          <fpage>31</fpage>
          -Jul-
          <year>2013</year>
          . [Online]. Available: http://www.fedtechmagazine.com/article/2013/07/army-brings-cloudbattlefield. [Accessed:
          <fpage>22</fpage>
          -Aug-2013].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S.</given-names>
            <surname>Miakhel</surname>
          </string-name>
          , “
          <article-title>Understanding Afghanistan: The importance of tribal culture and structure in security and governance,” Asian Survey</article-title>
          , vol.
          <volume>35</volume>
          , no.
          <issue>7</issue>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>B.</given-names>
            <surname>Ulicny</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. M.</given-names>
            <surname>Powell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. J.</given-names>
            <surname>Matheus</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Coombs</surname>
          </string-name>
          , and
          <string-name>
            <surname>M. M. Kokar</surname>
          </string-name>
          , “
          <article-title>Priority Intelligence Requirement Answering and Commercial Question-Answering: Identifying the Gaps,”</article-title>
          <source>DTIC Document</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>LTG</given-names>
            <surname>Keith</surname>
          </string-name>
          <string-name>
            <surname>Alexander</surname>
          </string-name>
          , BG Mike Ennis, Robert L. Grossman, James Heath, Russ Richardson, Glenn Tarbox, and Eric Sumner, “
          <article-title>Automating Markup of Intelligence Community Data: A Primer,” Defense Intelligence Journal</article-title>
          , no.
          <issue>12-2</issue>
          , pp.
          <fpage>83</fpage>
          -
          <lpage>96</lpage>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>D.</given-names>
            <surname>Davis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Lichtenwalter</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N. V.</given-names>
            <surname>Chawla</surname>
          </string-name>
          , “
          <article-title>Supervised methods for multi-relational link prediction</article-title>
          ,
          <source>” Soc. Netw. Anal. Min.</source>
          , vol.
          <volume>3</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>127</fpage>
          -
          <lpage>141</lpage>
          , Jun.
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>D.</given-names>
            <surname>Davis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Lichtenwalter</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N. V.</given-names>
            <surname>Chawla</surname>
          </string-name>
          ,
          <article-title>“Multi-relational Link Prediction in Heterogeneous Information Networks</article-title>
          ,” in
          <source>2011 International Conference on Advances in Social Networks Analysis and Mining (ASONAM)</source>
          ,
          <year>2011</year>
          , pp.
          <fpage>281</fpage>
          -
          <lpage>288</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>David</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tatiana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Alan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Shaun</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Barry</surname>
          </string-name>
          , “
          <article-title>Integration of Intelligence Data through Semantic Enhancement,”</article-title>
          <source>in Proceedings of the Sixth International Conference on Semantic Technologies for Intelligence</source>
          , Defense, and
          <string-name>
            <surname>Security</surname>
            , Fairfax,
            <given-names>VA</given-names>
          </string-name>
          , USA, pp.
          <fpage>6</fpage>
          -
          <lpage>13</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Malyuta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. S.</given-names>
            <surname>Mandrick</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Fu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Parent</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Patel</surname>
          </string-name>
          , “
          <article-title>Horizontal Integration of Warfighter Intelligence Data: A Shared Semantic Resource for the Intelligence Community</article-title>
          ,”
          <source>in Proceedings of the Seventh International Conference on Semantic Technologies for Intelligence</source>
          , Defense, and
          <string-name>
            <surname>Security</surname>
          </string-name>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>D.</given-names>
            <surname>Liben-Nowell</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Kleinberg</surname>
          </string-name>
          , “
          <article-title>The link-prediction problem for social networks,” Journal of the American society for information science and technology</article-title>
          , vol.
          <volume>58</volume>
          , no.
          <issue>7</issue>
          , pp.
          <fpage>1019</fpage>
          -
          <lpage>1031</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>G.</given-names>
            <surname>Zenz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Minack</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Siberski</surname>
          </string-name>
          , and W. Nejdl, “
          <article-title>Interactive Query Construction for Keyword Search on the Semantic Web,” in Semantic Search over the Web</article-title>
          , R. De Virgilio,
          <string-name>
            <given-names>F.</given-names>
            <surname>Guerra</surname>
          </string-name>
          , and Y. Velegrakis, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg,
          <year>2012</year>
          , pp.
          <fpage>109</fpage>
          -
          <lpage>130</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>S.</given-names>
            <surname>Ferré</surname>
          </string-name>
          ,
          <string-name>
            <surname>A</surname>
          </string-name>
          . Hermann, and
          <string-name>
            <given-names>M.</given-names>
            <surname>Ducassé</surname>
          </string-name>
          , “
          <article-title>Combining Faceted Search and Query Languages for the Semantic Web</article-title>
          ,” in Advanced Information Systems Engineering Workshops,
          <string-name>
            <given-names>C.</given-names>
            <surname>Salinesi</surname>
          </string-name>
          and
          <string-name>
            <given-names>O.</given-names>
            <surname>Pastor</surname>
          </string-name>
          , Eds. Springer Berlin Heidelberg,
          <year>2011</year>
          , pp.
          <fpage>554</fpage>
          -
          <lpage>563</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>H.</given-names>
            <surname>Pérez-Urbina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Rodrıguez-Dıaz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Grove</surname>
          </string-name>
          , G. Konstantinidis, and E. Sirin, “
          <article-title>Evaluation of query rewriting approaches for OWL 2,”</article-title>
          <source>in Proc. of the Joint Workshop on Scalable and High-Performance Semantic Web Systems (SSWS+ HPCSW</source>
          <year>2012</year>
          ),
          <year>2012</year>
          , vol.
          <volume>943</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>G.</given-names>
            <surname>Tecuci</surname>
          </string-name>
          ,
          <article-title>Building intelligent agents: an apprenticeship multistrategy learning theory, methodology, tool and case studies</article-title>
          . San Diego: Academic Press,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>G.</given-names>
            <surname>Tecuci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Boicu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Boicu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Marcu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Stanescu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Barbulescu</surname>
          </string-name>
          , “
          <source>The Disciple-Rkf Learning and Reasoning Agent,” Computational Intelligence</source>
          , vol.
          <volume>21</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>462</fpage>
          -
          <lpage>479</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Boicu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Marcu</surname>
          </string-name>
          , G. Tecuci, and
          <string-name>
            <given-names>D.</given-names>
            <surname>Schum</surname>
          </string-name>
          , “
          <article-title>Cognitive Assistants for Evidence-Based Reasoning Tasks</article-title>
          ,” in
          <source>2011 AAAI Fall Symposium Series</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>G.</given-names>
            <surname>Tecuci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schum</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Boicu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Marcu</surname>
          </string-name>
          , and
          <string-name>
            <surname>Hamilton</surname>
          </string-name>
          , Benjamin, “
          <source>TIACRITIS System and Textbook: Learning Intelligence Analysis through Practice,” in Proceedings of the Fifth International Conference on Semantic Technologies for Intelligence</source>
          , Defense, and
          <string-name>
            <surname>Security</surname>
          </string-name>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>G.</given-names>
            <surname>Tecuci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Marcu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Boicu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schum</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Russell</surname>
          </string-name>
          , “
          <source>Computational Theory and Cognitive Assistant for Intelligence Analysis,” in Proceedings of the Sixth International Conference on Semantic Technologies for Intelligence</source>
          , Defense, and Security - STIDS
          <year>2011</year>
          ,
          <article-title>Fairfax</article-title>
          ,
          <string-name>
            <surname>VA</surname>
          </string-name>
          , USA,
          <year>2011</year>
          , pp.
          <fpage>66</fpage>
          -
          <lpage>75</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>S. F.</given-names>
            <surname>Allen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bickford</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. L.</given-names>
            <surname>Constable</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Eaton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Kreitz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Lorigo</surname>
          </string-name>
          , and E. Moran, “
          <article-title>Innovations in computational type theory using Nuprl,”</article-title>
          <source>Journal of Applied Logic</source>
          , vol.
          <volume>4</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>428</fpage>
          -
          <lpage>469</lpage>
          , Dec.
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <article-title>Interactive Theorem Proving and Program Development - Coq'Art: The Calculus of Inductive Constructions</article-title>
          . .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>T.</given-names>
            <surname>Nipkow</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. C.</given-names>
            <surname>Paulson</surname>
          </string-name>
          , and M. Wenzel, Isabelle/HOL:
          <article-title>A Proof Assistant for Higher-Order Logic</article-title>
          . Springer,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>R.</given-names>
            <surname>Dapoigny</surname>
          </string-name>
          and
          <string-name>
            <given-names>P.</given-names>
            <surname>Barlatier</surname>
          </string-name>
          , “
          <article-title>Using a Dependently-Typed Language for Expressing Ontologies,” in Knowledge Science, Engineering</article-title>
          and Management,
          <string-name>
            <given-names>H.</given-names>
            <surname>Xiong</surname>
          </string-name>
          and
          <string-name>
            <given-names>W. B.</given-names>
            <surname>Lee</surname>
          </string-name>
          , Eds. Springer Berlin Heidelberg,
          <year>2011</year>
          , pp.
          <fpage>257</fpage>
          -
          <lpage>268</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>R.</given-names>
            <surname>Dapoigny</surname>
          </string-name>
          and
          <string-name>
            <given-names>P.</given-names>
            <surname>Barlatier</surname>
          </string-name>
          , “
          <article-title>Formal foundations for situation awareness based on dependent type theory,” Information Fusion</article-title>
          , vol.
          <volume>14</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>87</fpage>
          -
          <lpage>107</lpage>
          , Jan.
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>G. De Giacomo</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Lenzerini</surname>
            , and
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Rosati</surname>
          </string-name>
          , “
          <article-title>Higher-Order Description Logics for Domain Metamodeling</article-title>
          .,
          <source>” in Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence</source>
          ,
          <year>2011</year>
          , pp.
          <fpage>183</fpage>
          -
          <lpage>188</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>F. A.</given-names>
            <surname>Lisi</surname>
          </string-name>
          , “
          <article-title>A Declarative Modeling Language for Concept Learning in Description Logics,” in Inductive Logic Programming</article-title>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Riguzzi</surname>
          </string-name>
          and
          <string-name>
            <given-names>F.</given-names>
            <surname>Železný</surname>
          </string-name>
          , Eds. Springer Berlin Heidelberg,
          <year>2013</year>
          , pp.
          <fpage>151</fpage>
          -
          <lpage>165</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Abedjan</surname>
          </string-name>
          and
          <string-name>
            <given-names>F.</given-names>
            <surname>Naumann</surname>
          </string-name>
          , “
          <article-title>Improving RDF Data Through Association Rule Mining,” Datenbank Spektrum</article-title>
          , vol.
          <volume>13</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>111</fpage>
          -
          <lpage>120</lpage>
          , Jul.
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>C.</given-names>
            <surname>Benzmüller</surname>
          </string-name>
          and
          <string-name>
            <given-names>A.</given-names>
            <surname>Pease</surname>
          </string-name>
          , “
          <article-title>Higher-order aspects and context in SUMO,”</article-title>
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          , vol.
          <volume>12</volume>
          -
          <issue>13</issue>
          , pp.
          <fpage>104</fpage>
          -
          <lpage>117</lpage>
          , Apr.
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>M.-L. Mugnier</surname>
          </string-name>
          , “
          <article-title>Ontological query answering with existential rules</article-title>
          ,”
          <source>in Proceedings of the 5th international conference on Web reasoning and rule systems</source>
          , Berlin, Heidelberg,
          <year>2011</year>
          , pp.
          <fpage>2</fpage>
          -
          <lpage>23</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Moten</surname>
          </string-name>
          , Rod, “
          <article-title>Using a Type-theoretic Approach to Resolve Heterogeneity in Data Fusion,”</article-title>
          <source>in Proceedings of the National Symposium on Sensor and Data Fusion</source>
          <year>2012</year>
          , Washington, D.C.,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>A.</given-names>
            <surname>Nanevski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Pfenning</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Pientka</surname>
          </string-name>
          , “
          <article-title>Contextual modal type theory,”</article-title>
          <source>ACM Trans. Comput. Logic</source>
          , vol.
          <volume>9</volume>
          , no.
          <issue>3</issue>
          , pp.
          <volume>23</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>23</lpage>
          :
          <fpage>49</fpage>
          ,
          <string-name>
            <surname>Jun</surname>
          </string-name>
          .
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>