<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Effects-Based Air Operations Planning Framework: A Knowledge-Based Simulation Approach</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>André N. Costa</string-name>
          <email>pcosta@gmu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>C4I &amp; Cyber Center George Mason University Fairfax</institution>
          ,
          <addr-line>VA</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute for Advanced Studies Brazilian Air Force São José dos Campos</institution>
          ,
          <addr-line>SP</addr-line>
          ,
          <country country="BR">Brazil</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2016</year>
      </pub-date>
      <fpage>5</fpage>
      <lpage>12</lpage>
      <abstract>
        <p>- Planning air warfare operations has always been a complex endeavor. However, as technology evolves at an increasingly fast pace, so does the complexity of managing its resources. In modern air operations, planners have to deal with a highly changing environment influenced by enemy air defenses, weather forecasts, among many other factors, demanding much effort to handle the great number of constraints and uncertainties presented by them. As a result, a number of decision-support systems have emerged attempting to facilitate the planning of air warfare operations. These systems usually rely on a wide variety of methodologies, which sometimes present a challenge in themselves when it comes to assessing the feasibility and effectiveness of the produced plans. Computer simulations are a practical way of providing this assessment, usually by running the resulting plans multiple times and checking the results against key criteria. Yet, establishing the right criteria, properly accounting for the “fog of war,” and avoiding impractical simulation run times and costs are still major challenges. This paper addresses such challenges by proposing the development of a decisionsupport framework that combines ontology-based agile knowledge and a simulation-based mission planning methodology that accounts for the inherent uncertainties that air operations face. We avoid costly computation times required by simulationintensive course-of-action analyzers by initially pruning the solution space through ontological reasoning. Moreover, the approach complies with the Effects-Based Approach to Operations, having a clear correspondence of processes with it. The explanations are focused on a specific scenario concerning intelligence, surveillance, and reconnaissance operations.</p>
      </abstract>
      <kwd-group>
        <kwd>ontologies</kwd>
        <kwd>effects-based planning</kwd>
        <kwd>modeling and simulation</kwd>
        <kwd>semantic matchmaking</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>The fast pace by which complexity of current military
operations is increasing has become a major challenge for
mission planners, requiring a much more meticulous planning
process to handle all the factors that might influence the
outcomes of an operation. Several planning methodologies have
been observed in the last years, yielding a number of systems to
support air operations planning. To deal with complexity, these
systems usually rely on heavy computing power, as well as on
specialized operators that must be highly trained in the
methodology associated with the system. Such requirements
make the planning process brittle, as both the hardware and the
Paulo C. G. Costa2</p>
      <p>
        The planning knowledge to be captured is based on the
Effects-Based Approach to Operations (EBAO). This paradigm
has been extensively sought by several research and
development efforts in the last decades, paving the way to its
fruition and further application on the field [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. According to [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ],
“EBAO informs every aspect of how the Air Force designs,
plans, executes, assesses, and adapts operations.” Therefore, it
should guide any framework that proposes to aid the planning
process of air operations. Following this premise, the backbone
of the matchmaking process within this work is the
EffectsBased Approach to Planning (EBP), which ultimately defines
the semantic description of the domain and how it relates to the
planning procedure.
      </p>
      <p>Once the initial states of the EBAO knowledge is made
explicit through ontology engineering, the focus of our
development becomes to provide a solution that does not require
large amounts of computing power and time. Rather, it may be
done using portable computers by the operations planners. We
achieve that by leveraging the EBAO mission concepts via a
logical engine that pre-selects the possibilities given the
planning data provided, greatly limiting the solution search
space. This way, optimization methods can be used in a much
more effective way, applied to scenarios that are simulated in a
simplified fashion, and allowing for a quick means of assessing
the optimization parameters. In a second step, these generated
low-resolution solutions are evaluated based on criteria derived
from the EBAO ontology. The most promising ones then go
through a more complex simulation environment that, through
entity-level simulation, would provide a much more detailed
outcome that includes mission-specific prognostics.</p>
      <p>
        For providing a clearer view of how this framework can be
operationalized, an ISR (intelligence, surveillance, and
reconnaissance) scenario is built with unclassified data from the
Brazilian Air Force. The database includes a number of
platforms and sensors that have to be assigned to tasks, leading
to actions that generate the desired operational effects. Since not
all sensing sources can provide the needed information to task
requirements, because the sources are context sensitive [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], this
assignment can be very challenging to the planning staff.
      </p>
      <p>ISR operations proved to be a good choice for this initial
scenario, since they contain multiple factors that directly affect
the planning process, and also make its optimization very
important. Also, since ISR assets are oftentimes highly complex
and valuable, a less than adequate planning will lead to the loss
of costly flight hours and very specialized crews work.
Nevertheless, the application on the planning process of other
air operations can be made based on the same framework
structure, converting the sensor matchmaking phase to a weapon
to target matching in the case of airstrike or managing electronic
warfare (EW) measures on a suppression of enemy air defense
(SEAD) mission, both sharing many complexities with ISR
operations.</p>
      <p>At this stage of our development, we did not yet reach full
circle or obtained conclusive results of a detailed simulation.
Thus, our focus on this paper is to provide the long-range vision
of the framework, its goals, and an overview of the technical
approaches determined by our preliminary research efforts to
solve the challenges encountered so far.</p>
      <p>The paper is organized as follows. Section II gives a brief
overview of previous research on operation planning
frameworks like the one proposed, as well as on semantic
matchmaking efforts. Section III provides the main concepts
involved on EBAO, emphasizing the EBP process. Section IV
presents the framework, including the description of the
software applications to be used on its implementation. Section
V focuses explicitly on the semantic part of the framework.
Section VI displays the considered scenario, describing
resources and critical conditions that may influence how the
image requirements can be met. Finally, Section VII
summarizes the paper, pointing to the next steps to be taken.</p>
    </sec>
    <sec id="sec-2">
      <title>II. BACKGROUND</title>
      <sec id="sec-2-1">
        <title>A. Planning Simulation Framework</title>
        <p>Several planning frameworks are available within the
military research context. However, much of the work available
is either too complex or remains inaccessible (e.g., classified).
The complexity is often directly related to the high resolution
required to generate reliable results.</p>
        <p>While trying to present an alternative to this complexity
problem, some authors have proposed the use of lower
resolution simulation and optimization methodologies, which
deal with less factors at a time.</p>
        <p>
          Rosenberg et al. [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] suggest a collection of decision-support
tools for planning generation that consists of “a method to
define an operational scenario, an optimization engine to
generate a diverse set of solutions, and a suite of visualization
and analysis tools to review, analyze, and visualize generated
plans.” To provide a solution in a timely manner, the authors
propose a rapid evaluation of candidate solutions through
agentbased modeling and simulation (ABMS). They leveraged the
same software used in our proposed framework.
        </p>
        <p>
          Similarly, [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] – also using the same software application –
focuses on a Joint Suppression of Enemy Air Defenses (JSEAD)
scenario in which plans are generated, optimized, and simulated.
The authors also rely on an ABMS of two sides containing
different types of entities, usually targets and air defenses for the
opposing side and strikers and JSEAD units for the friendly side.
Their results illustrated the potential of low-resolution
simulation as a rapid evaluation tool of generated plans, which
will be in time described within our own approach.
        </p>
        <p>
          Unlike in our approach, these research efforts do not apply
semantic methods as a form of structuring the modeling and
simulation process (e.g. [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]), or as a conceptual basis for the
framework as those in the subsection below.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>B. Semantic Matchmaking Framework</title>
        <p>
          The literature on assigning sensors to missions or tasks is
vast, but the use of semantic techniques for this purpose is rather
limited. Therefore, it is worth pointing out [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ], which advocates
for an ontological problem-solving architecture to facilitate
automated inference of assigning sensors to missions. This work
limits the solution domain as a means of including a
coordination system to emulate the assets and complete bears
similarities with the aforementioned planning simulation
frameworks and the one we propose in this paper.
        </p>
        <p>
          One of the most productive solutions is sponsored by the
U.S. Army Research Laboratory and the United Kingdom
Ministry of Defence ([
          <xref ref-type="bibr" rid="ref8">8</xref>
          ], [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ], [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ], [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ], [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ], [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]). Its authors
conceive a system that relies on a series of ontologies for
assigning sensors to missions. The backbone of this process is
the “Mission and Means Framework” that is claimed to “provide
a model for explicitly specifying a military mission and
quantitatively evaluating the mission utility of alternative
warfighting solutions” [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. The three basic elements of their
methodology are [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]:
•
•
•
        </p>
        <p>Top-to-bottom solution to the problem of deploying
sensors to meet the information needs of tasks in a
mission context;
Combination of reasoning at mission-planning time, and
optimization algorithms at mission execution-time; and
Dynamic deployment configuration of selected sensor
instances by means of a sensor infrastructure.</p>
        <p>
          The work includes modular ontologies that cover task
requirements, sensor capabilities, and a structured framework to
associate tasks with sensors. The ontologies specify the
requirements of the missions and the capabilities of the sensors
so that the framework is able to decide between combinations of
sensors to satisfy the requirements of a given mission [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ].
        </p>
        <p>
          Even though providing a proven assignment system [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], with
very well-structured ontologies, this work does not focus on
dealing with the uncertainties that a planning scenario presents.
This is due to the use of logical reasoners and mostly
deterministic functions. Stochasticity is not considered, just
comparisons between deterministic possibilities. In addition, the
Mission and Means Framework ontology focuses on tasks
instead of effects. Thus, although providing a direct and clear
way of breaking down missions [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ], the approach does not
emphasize the holistic view advocated in our work. Finally, [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]
also provides more details on how this framework may be
structured as an ontology.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>III. EFFECTS-BASED PLANNING</title>
      <p>
        Even though utilizing the Mission and Means Framework,
[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] states that alternative mission planning approaches, such as
effects-based planning, may be structured in a similar way, with
the goal of assigning resources to missions.
      </p>
      <p>
        “Planning to achieve an effect” has been used naively as a
rather straightforward definition of EBP. However, the vast
majority of planners would argue that any previous approach to
military planning would include this asseveration [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
Therefore, it is imperative to clearly define this concept upfront.
      </p>
      <p>
        The US Air Force doctrine [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] holds that “there is no single
‘effects-based planning’ methodology or process. Rather,
understanding the principles of an effects-based approach to
operations should yield certain insights and enhance
comprehension of many general planning concepts”. This is the
reason why it is important to first understand what EBAO
means.
      </p>
      <p>
        Reference [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] presents the US Joint Forces Command
definition of EBAO as “a process for obtaining a desired
strategic outcome or effect on the enemy through the synergistic
and cumulative application of the full range of military and
nonmilitary capabilities at all levels of conflict”. Another
definition presented on [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] is that “effects-based operations are
operations conceived and planned in a systems framework that
considers the full range of direct, indirect, and cascading effects,
which may—with different degrees of probability—be achieved
by the application of military, diplomatic, psychological, and
economic instruments”.
      </p>
      <p>What both of these definitions emphasize is that the process
of planning has to be much more intentional on the pursuit of a
holistic view of the operation. There is a focus on addressing not
only direct physical effects, but several types of indirect effects,
which are influenced by each other. Planners are encouraged to
maintain a very broad view of the “big picture”, especially
during execution, not being caught up in details that can tarnish
the end state visualization.</p>
      <p>
        A better understanding of our approach requires exploring
EBO’s main concepts, which are described in the next Section.
However, our framework greatly relies on the EBO principles
listed below, which were suggested by [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <sec id="sec-3-1">
        <title>A. Uncertainty</title>
        <p>The first principle says that effects-based operations (EBO)
planners have to rely on methods that explicitly deal with
probabilities and randomness to properly address the inherent
uncertainties contained in the air operations. EBP has to fully
confront the scope and magnitude of these uncertainties,
especially when dealing with outcome predictions.</p>
      </sec>
      <sec id="sec-3-2">
        <title>B. Qualitative modeling</title>
        <p>Secondly, in this uncertainty-sensitive framework it is
imperative to possess a trustworthy qualitative modeling,
including frictional, credibility and cognitive factors that are
oftentimes closely related to indirect effects. This is highly
dependable on the availability of subject matter experts (SME)
to provide information about systems and operations.</p>
      </sec>
      <sec id="sec-3-3">
        <title>C. Agent-based modeling</title>
        <p>The qualitative modeling also requires a focus on
decisionmaking, which can be addressed by agent-based modeling
approaches, accurately depicting the C4ISR aspects of the
operations. Cognitive models may be housed in agent
architectures, allowing analyzes of emerging scenarios closer
to the reality and with a clear focus on the command and control
structure.</p>
      </sec>
      <sec id="sec-3-4">
        <title>D. Capability planning</title>
        <p>Is expected from EBP to determine a range of circumstances
that provides degrees of confidence towards the meeting of the
conditions that characterize a desired end state. These
operational circumstances have to be linked to the necessary
capabilities to provide this confidence, not only the necessary
means.</p>
      </sec>
      <sec id="sec-3-5">
        <title>E. Empirical information</title>
        <p>As stated when speaking of the qualitative modeling,
empirical information provided by SMEs is extremely important
for a successful EBP. In addition to that, information from
history, war-gaming, simulations and experiments should be
strongly pursued so that the complex models can be modelled
and uncertainties reduced.</p>
      </sec>
      <sec id="sec-3-6">
        <title>F. Adaptation</title>
        <p>The last principle relates to planning for adaptation. Since a
lot of uncertainties are present and the scenarios may present
emergent behavior, it is very important to be able to adapt and
dynamically change plans even during execution time.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>IV. FRAMEWORK</title>
      <p>Before presenting our framework itself, we must first
provide the necessary context, which is conveyed in Fig. 1. In
EBAO, effects are defined as results of actions. These actions
are simply assigned tasks. The ontology described in Section V,
is used to support a matching process between effects and
resources. The resources in the analyzed scenario are platforms
and sensors, which may be mounted to the platforms or not. The
objectives that defined the desired effects are then translated to
fitness values within the simulation, providing a means for the
plans optimization. On the tactical level, these objectives form
a specific mission that, on the operational level, leads to the
desired end state.</p>
      <p>ABMS is used to represent this mission, possessing
cognition models that encompass the available expert
information as well as showing the interaction and coordination
between the agents, representing the C4ISR processes involved.
With several runs of the simulation scenario, uncertainties can
be added mostly on the hostile units’ locations, on available
capabilities, and on different behavior patterns employed. Also,
time issues may be initially addressed, since the agents’
interactions allow for identifying some of the interferences they
generate on each other through the simulation run.</p>
      <p>Generating cognition models can be a strenuous process,
especially when dealing with rule-based scripts for the agent
behavior definition, which, besides being hard to implement, do
not capture the uncertainties present on military operations.
This is why the approach for the friendly and enemy forces’
threat assessment process relies on probabilistic models, such
as Bayesian networks, capable of representing the dependencies
between the entities’ actions and the evidence accrued from the
C4ISR sources available.</p>
      <p>
        From the ABMS, each of the generated plans can be
properly simulated and consequently evaluated based on
criteria originated from the three superior goals defined by [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]:
flight safety, combat survival, and mission accomplishment.
The first goal relates to the need of the pilot to concentrate on
flying the aircraft in a safe way, for instance assuring that it has
the necessary amount of fuel, that it flies through proper fly
zones, avoiding collisions with other aircrafts and the terrain.
The second focuses on the chances of enduring through the
mission, considering the capabilities of the hostile forces and
the exposure to them. Lastly, the third goal illustrates the
original objective of the mission performed, such as gathering
intelligence information, striking a ground target, or
suppressing the enemy’s air defenses.
      </p>
      <p>During the optimization parameters definition there is a need
of defining a prioritization of these three basic goals. This
process depends on several factors, such as rules of engagement,
value of the assets, and criticality of the mission. These factors
have to be properly valued by the leadership and then
parametrized by the analysts to correctly represent the
commander’s intent (CI) on a top-to-bottom fashion.</p>
      <p>At the end, the framework consists of a deeper and more
thorough entity-level simulation with the goal of determining if
the conditions that define the end state are met within a feasible
timeframe by the previously selected best plan. Also, this phase
allows for mission rehearsal and order generation.</p>
      <p>
        To summarize, as extracted from [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] and [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], the
righthand side terms of Fig. 1 can be individually defined as:
•
•
•
•
•
•
      </p>
      <p>Resources: all the available assets to generate the desired
effects;
Tasks: an action or actions that have been assigned to
someone to be performed;</p>
    </sec>
    <sec id="sec-5">
      <title>Actions: result of assigned tasks;</title>
      <p>Effects: all the physical, functional or psychological
outcomes, events or consequences that results from
specific military or nonmilitary actions;
Objectives: the clearly defined, decisive, and attainable
goals towards which every operation is directed; and
End state: the set of required conditions that defines
achievement of the commander’s objectives.</p>
      <p>As one can notice, the elements presented in the previous
Section are met, since the resources are approached as
capabilities and contain several qualitative and empirical
information, which also permeates the other concepts of the
framework. Uncertainty is handled through simulation layers,
with the ABMS suggestion alongside. Lastly, the design for
adaptation is taken in consideration through the process of
generation of multiple plans, and mostly by the ontological
reasoning that can quickly change the initial constraints,
leading to a faster plan evaluation during dynamic re-planning.</p>
      <p>Each of the last four boxes on the left-hand side of Fig. 1 is
performed by a different software application that are
respectively described as follows:</p>
      <sec id="sec-5-1">
        <title>A. Semantic Modeling: Protégé</title>
        <p>
          Protégé is one of the most popular knowledge-modelling
environments. It not only allows users to interactively edit
knowledge-bases within its graphic user interface, but also
presents a series of plugins that add a number of functionalities
and services, such as ontology management tools, multimedia
support, querying and reasoning engines, and problem solving
methods. Also, it has experienced several actualizations in the
last decades and has a vast user community, featuring high
stability and usability ratings. As well as the two following
applications, it is written in Java, allowing for a smoother
integration in the future ([
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ], [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]).
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>B. Optimization: ECJ</title>
        <p>ECJ is a general-purpose evolutionary computation and
genetic programming framework designed for large,
heavyweight experimental needs. It is a free open-source application
developed by the Department of Computer Science of George
Mason University. In spite of being more than 10 years old, it
shows great stability and an optimized design, attested by a
large number of users in the genetic programming community.</p>
        <p>
          Besides its main goal of attempting to permit as many valid
combinations as possible of individual representation and
breeding method, fitness and selection procedure, evolutionary
algorithm, and parallelism, it contains multi-objective
optimization algorithms, island models, master/slave
evaluation facilities, coevolution, steady-state and evolution
strategies methods, parsimony pressure techniques, and various
individual representations ([
          <xref ref-type="bibr" rid="ref23">23</xref>
          ], [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ]).
        </p>
      </sec>
      <sec id="sec-5-3">
        <title>C. Agent-Based Simulation: MASON</title>
        <p>
          MASON is a single-process discrete-event multi-agent
simulation toolkit written in Java that comprises a fast core
engine and a fully separated visualization display. It is very
versatile and easily expandable, providing friendly licensing
options and excellent performance. In addition, it is designed to
support large numbers of agents relatively efficiently on a single
machine in models that are entirely encapsulated. Even the
elements of the system itself are highly independent, providing
a modular and consistent way to combine its different parts in
various ways. Some of these parts form a large set of utilities
that has the goal of supporting model design. Finally, as well as
ECJ, it is developed and maintained by a research group from
George Mason University ([
          <xref ref-type="bibr" rid="ref26">26</xref>
          ], [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ], [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ], [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ]).
        </p>
      </sec>
      <sec id="sec-5-4">
        <title>D. High-Resolution Simulation: VR-Forces</title>
        <p>
          VR-Forces is a simulation environment created by VT MÄK
for scenario generation [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ]. The platform is widely used
throughout the industry, and provides a well-engineered basis
for integrating CGFs with urban, battlefield, maritime, and
airspace activity. Apart from the graphical interface (front-end),
VR-Forces consists of a back-end application, which is its actual
simulation engine. As such, VR-Forces scenarios can be scaled
up by running multiple front-ends and/or back-ends,
communicating through its networking toolkit. Moreover, both
the VR-Forces front-end and back-end can be extended either by
being embedded into another application or through plug-ins,
using the C++ API provided.
        </p>
        <p>
          Reference [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ] provides a study comparing several CGF
simulation software in terms of autonomy, learning and
adaptation, organization, realism, and architecture. VR-Forces
was considered to be the most suitable as a development
platform, mostly because its AI capability built-in, very good
documentation and technical support, and support for data
logger export. The same conclusion was drawn by [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ] in a
much more thorough analysis.
        </p>
        <p>
          One of the main reasons why this work advocates for the use
of ontologies for EBP planning is that they can contain detailed
information about the military domain in a very structured way.
This is made very formally, explicitly expressing clear and
precise definitions of concepts and relationships [
          <xref ref-type="bibr" rid="ref33">33</xref>
          ]. Besides,
it provides a domain conceptualization of EBAO as expressed
on Fig. 2, allowing for a better understanding and application of
its features.
        </p>
        <p>Some of the information contained on our knowledge-base
will be presented on the next section, but Fig. 3 shows a
screenshot taken from the proposed ontology on Protégé. It is
structure from three basic domain concepts: capabilities, sensors
and platforms. The two latter are the resource descriptions, with
their properties and limitations taken into account. The first, is
related to the actions that can generate the desired effects, but
also to some of the constraints that can influence the mission, as
discussed in the next section.</p>
        <p>After properly modeling the domain with the most
significant parameters and properties, the semantic component
of the framework needs to perform the matchmaking between
resources and the required tasks for effect generation. This calls
for a semantic breakdown of the effects, so that the available
capabilities may be used as generation factors for them. After
that, the matchmaking methods are able to assign the proper
resources as follows.</p>
      </sec>
      <sec id="sec-5-5">
        <title>B. Semantic matchmaking</title>
        <p>
          The notion of matchmaking consists of a procedure to find
correspondences between entities in ontologies [
          <xref ref-type="bibr" rid="ref34">34</xref>
          ]. Whereas
process is made by several existing techniques, this work will
focus on a description logic approach as advocated by [
          <xref ref-type="bibr" rid="ref35">35</xref>
          ].
        </p>
        <p>First, it is important to define that matchmaking takes place
as a process in which a requester party triggers the mechanism
of finding resources relevant to the request., while the provider
party describes the available resources in advance. With that, the
matchmaking is made through automated analysis and
comparisons of the semantic descriptions of the involved
resources.</p>
        <p>For doing so, the entities of an ontology and their
relationships have to be carefully modelled, representing the air
operations domain as a set of concrete resources that vary on
several properties. This variance is intended to allow the
specification of the resources, having different parameters.
However, due to incomplete information, these specifications
not necessarily describe all the parameters completely. To deal
with that, the notions of entailment and satisfiability back the
testing if all request formulas hold in all models of a knowledge
base and if these formulas are logical consequences of it.</p>
        <p>
          There are several matching infrences that are able to account
for this variance and that can be directly realised by description
logic, ranked in the following way according to their degrees of
matching [
          <xref ref-type="bibr" rid="ref36">36</xref>
          ]:
1) No match: empty intersection between two descriptions;
2) Intersection: non-empty intersection between two
descriptions;
        </p>
        <p>3) Non-Disjointness: non-empty intersection between two
descriptions in every possible world;</p>
        <p>4) Specialisation: subsumption between two descriptions
holds from right to left;</p>
        <p>5) Generalisation: subsumption between two descriptions
holds from left to right; and</p>
        <p>6) Equivalence: subsumption between two descriptions
holds in both directions.</p>
        <p>
          As stated in section VII, our next step is to test this
implementations to verify if the limitations of classical
description logic matchmaking are significant for in this context,
generating undesired matching behaviors. If so, other
methodologies may be embraced, such as the use of
nonmonotonic formalisms, such as terminological defaults,
autoepistemic and circumscriptive description logic [
          <xref ref-type="bibr" rid="ref35">35</xref>
          ].
        </p>
        <p>The proposed scenario represents the definition of a desired
effect yielding intelligence requirements. These requirements
are influenced by several factors including the resource
availability, environment conditions, and hostile activity. Each
factor imposes restrictions on the matching process. The
availability is directly related to the instantiation, the
environment produces constraints for some sensors and
platforms, and the opposing forces impact on the survivability
probabilities as well as on the mission success measurements.</p>
        <p>
          To illustrate the EBP focus, the chosen scenario contains the
requirement of an effect of assessing, gaining, and maintaining
air superiority in support of land and maritime schemes of
maneuver, as proposed by [
          <xref ref-type="bibr" rid="ref37">37</xref>
          ]. The author already exemplifies
how this effect yields ISR actions, such as: detect, discover, and
degrade key components of defense systems; confirm damage to
target acquisition radars and height-finding radars.
        </p>
        <p>With these actions in hand, the system needs to take into
consideration the available sensors and platforms to determine
which ones are capable of properly performing them. However,
their characteristics have also to be confronted with the instance
availability, environment conditions and the hostile threats
expected, since this will directly influence the matchmaking
process, as depicted in Fig. 4.</p>
      </sec>
      <sec id="sec-5-6">
        <title>A. Sensors</title>
        <p>Aircraft mounted sensors present different characteristics
that may also require different flight altitudes and visibilities in
order to properly work. Besides, depending on the applications
the image demands may be different, providing alternative
information from various sensors. The available sensors
considered in this work are:</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>OPT: Optical sensors;</title>
    </sec>
    <sec id="sec-7">
      <title>FLIR: Forward-looking infrared cameras;</title>
    </sec>
    <sec id="sec-8">
      <title>MSS: Multispectral Sensors;</title>
    </sec>
    <sec id="sec-9">
      <title>SAR: Synthetic Aperture Radar;</title>
      <p>NCES: Non-communications exploitation systems; and</p>
    </sec>
    <sec id="sec-10">
      <title>CES: Communication exploitations systems.</title>
      <sec id="sec-10-1">
        <title>B. Platforms</title>
        <p>As showed in Fig. 4 the considered platforms are some of
the aircrafts used by the Brazilian Air Force as ISR assets. These
platforms mount the aforementioned sensors according to
TABLE I. Besides, each one presents different values for range
and average speed, which may considerably influence the
operations. The aircrafts are:</p>
        <p>Elbit Systems Hermes 450 (RQ-450): medium size
unmanned aerial vehicle (UAV);
Lockheed P-3 Orion (P-3AM): four-engine turboprop
maritime surveillance aircraft;
Embraer EMB-111 Bandeirante Patrulha (P-95BM):
twin-turboprop maritime patrol aircraft;
•
•</p>
        <p>Embraer EMB-145 RS (R-99): twinjet remote sensing
aircraft; and
AMX International AMX-R (RA-1): ground-attack
aircraft for reconnaissance.</p>
      </sec>
    </sec>
    <sec id="sec-11">
      <title>VII. CONCLUSION AND FUTURE WORK</title>
      <p>The main goal of this paper was to provide an analysis of the
problem and a preliminary structure of the framework advocated
to solve it. The work focused on establishing a theoretical basis
for delineating this solution, adapting it to effects-based
approach to operations concepts. An added constraint was to
utilize free and open source applications to form the framework,
at least on its initial phases (the only exception being VR-Forces,
because of the lack of open alternatives that would provide
similar simulation capabilities). Moreover, these applications
should be light enough to allow for the execution of the
framework on a single machine, what they arguably are.</p>
      <p>The development of the framework not only justifies itself
as being an explicit representation of EBAO, but also on the
combination of simulation methods with an initial semantic
matchmaking process that reduces the solution space, allowing
for a potentially more agile way of determining operational
plans. Additionally, the ABMS phase allows for numerous and
fast simulation runs, acting as a fitness evaluation tool for the
optimization process as well as an analyzer of emerging
behaviors and complex C4ISR interactions.</p>
      <p>At the time of this writing, only the initial implementations
of the ontology described have been performed. Next steps
include the full development and implementation of the
matchmaking process. This step is needed so the optimization
can be executed giving continuity to the proposed methodology.</p>
      <p>Finally, more information regarding the scenario has to be
gathered, also allowing for an expansion of its scope, including
gradually more Air Force related activities, for instance airstrike.</p>
    </sec>
    <sec id="sec-12">
      <title>ACKNOWLEDGMENT</title>
      <p>The authors thank Major Breno Leite, from the Brazilian Air
Force, for providing data for the scenario construction as well as
serving as a subject matter expert, contributing with empirical
information and insights for the modeling. Our gratitude is also
extended to Danny Williams, from VT MÄK, who was
instrumental in providing technical expertise in the VR-Forces
suite.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>P. K.</given-names>
            <surname>Davis</surname>
          </string-name>
          ,
          <article-title>Effects-based Operations: A Grand Challenge for the Analytical Community</article-title>
          . RAND,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Joint</given-names>
            <surname>Staff</surname>
          </string-name>
          , “ANNEX 3-0 Operations &amp; Planning,” Washigton,
          <string-name>
            <surname>DC</surname>
          </string-name>
          , Aug.
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>R.</given-names>
            <surname>Ganger</surname>
          </string-name>
          , G. de Mel, T. Pham,
          <string-name>
            <given-names>R.</given-names>
            <surname>Rudnicki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Schreiber</surname>
          </string-name>
          , “
          <article-title>Sensor assignment to mission in AI-</article-title>
          TECD,”
          <year>2016</year>
          , p.
          <fpage>98310E</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>B.</given-names>
            <surname>Rosenberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Richards</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. T.</given-names>
            <surname>Langton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Tenenbaum</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D. W.</given-names>
            <surname>Stouch</surname>
          </string-name>
          , “
          <article-title>Applications of multi-objective evolutionary algorithms to air operations mission planning</article-title>
          ,”
          <year>2008</year>
          , p.
          <year>1879</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J. P.</given-names>
            <surname>Ridder and J. C. HandUber</surname>
          </string-name>
          , “
          <article-title>Mission Planning for Joint Suppression of Enemy Air Defenses Using a Genetic Algorithm,”</article-title>
          <source>in Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation</source>
          , New York, NY, USA,
          <year>2005</year>
          , pp.
          <fpage>1929</fpage>
          -
          <lpage>1936</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Tolk</surname>
          </string-name>
          , Ed.,
          <string-name>
            <surname>Ontology</surname>
          </string-name>
          , Epistemology, and
          <article-title>Teleology for Modeling and Simulation</article-title>
          , vol.
          <volume>44</volume>
          . Berlin, Heidelberg: Springer Berlin Heidelberg,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>J.</given-names>
            <surname>Qualls</surname>
          </string-name>
          and
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Russomanno</surname>
          </string-name>
          , “
          <article-title>Ontological problem-solving framework for assigning sensor systems and algorithms to high-level missions</article-title>
          ,
          <source>” Sensors</source>
          , vol.
          <volume>11</volume>
          , no.
          <issue>9</issue>
          , pp.
          <fpage>8370</fpage>
          -
          <lpage>8394</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>A.</given-names>
            <surname>Preece</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Norman</surname>
          </string-name>
          , G. de Mel,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pizzocaro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sensoy</surname>
          </string-name>
          , and T. Pham, “
          <article-title>Agilely Assigning Sensing Assets to Mission Tasks in a Coalition Context,” IEEE Intell</article-title>
          . Syst., vol.
          <volume>28</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>57</fpage>
          -
          <lpage>63</lpage>
          , Jan.
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Preece</surname>
          </string-name>
          et al., “
          <article-title>Matching sensors to missions using a knowledgebased approach</article-title>
          ,”
          <year>2008</year>
          , p.
          <fpage>698109</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A.</given-names>
            <surname>Preece</surname>
          </string-name>
          et al.,
          <article-title>“Reasoning and resource allocation for sensor-mission assignment in a coalition context,”</article-title>
          <source>in MILCOM 2008 - 2008 IEEE Military Communications Conference</source>
          ,
          <year>2008</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>7</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>G. de Mel</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Sensoy</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          <string-name>
            <surname>Vasconcelos</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A. D.</given-names>
            <surname>Preece</surname>
          </string-name>
          , “
          <article-title>Flexible Resource Assignment in Sensor Networks : A Hybrid Reasoning Approach</article-title>
          ,” presented at the 1st
          <source>International Workshop on the Semantic Sensor Web (SemSensWeb)</source>
          , Heraklion, Crete, Greece,
          <year>2009</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M.</given-names>
            <surname>Gomez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Preece</surname>
          </string-name>
          , and G. de Mel, “
          <article-title>Towards Semantic Matchmaking in Sensor-Mission Assignment: Analysis of the Missions</article-title>
          and Means Framework,” International Technology Alliance,
          <source>Technical Report 1</source>
          .3,
          <string-name>
            <surname>Mar</surname>
          </string-name>
          .
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A.</given-names>
            <surname>Preece</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pizzocaro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Borowiecki</surname>
          </string-name>
          , G. de Mel,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gomez</surname>
          </string-name>
          , and W. Vasconcelos, “
          <article-title>Sensor Assignment to Missions in a Coalition Context: The SAM Tool</article-title>
          ,” presented at the 28th Conference on Computer Communications, Rio de Janeiro, Brazil,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Gomez</surname>
          </string-name>
          et al.,
          <article-title>“An Ontology-Centric Approach to Sensor-Mission Assignment,” in Knowledge Engineering: Practice and</article-title>
          <string-name>
            <surname>Patterns</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Gangemi</surname>
          </string-name>
          and J. Euzenat, Eds. Springer Berlin Heidelberg,
          <year>2008</year>
          , pp.
          <fpage>347</fpage>
          -
          <lpage>363</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>P. H.</given-names>
            <surname>Deitz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. E.</given-names>
            <surname>Bray</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Michaelis</surname>
          </string-name>
          , “
          <article-title>The missions and means framework as an ontology</article-title>
          ,”
          <year>2016</year>
          , vol.
          <volume>9831</volume>
          , pp.
          <fpage>983109</fpage>
          -
          <lpage>983109</lpage>
          -12.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>B.</given-names>
            <surname>Bullen</surname>
          </string-name>
          , “
          <article-title>A Bayesian Methodology for Effects Based Planning</article-title>
          ,” Dec.
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>I.</given-names>
            <surname>Duyvesteyn</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Angstrom</surname>
          </string-name>
          ,
          <article-title>Modern War and the Utility of Force: Challenges, Methods and Strategy</article-title>
          . Routledge,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>A.</given-names>
            <surname>Schulte</surname>
          </string-name>
          , “
          <article-title>Cognitive Automation for Tactical Mission Management: Concept and Prototype Evaluation in Flight Simulator Trials</article-title>
          ,” Cogn. Technol. Work, vol.
          <volume>4</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>146</fpage>
          -
          <lpage>159</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Joint</surname>
            <given-names>Staff</given-names>
          </string-name>
          , “Joint Publication 5-0: Joint Operation Planning,” Washigton,
          <string-name>
            <surname>DC</surname>
          </string-name>
          , Aug.
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20] “Protégé.” [Online]. Available: http://protege.stanford.edu/. [Accessed:
          <fpage>14</fpage>
          -Aug-2016].
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>J. H.</given-names>
            <surname>Gennari</surname>
          </string-name>
          et al., “
          <article-title>The evolution of Protégé: an environment for knowledge-based systems development,”</article-title>
          <string-name>
            <given-names>Int. J.</given-names>
            <surname>Hum</surname>
          </string-name>
          .-Comput. Stud., vol.
          <volume>58</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>89</fpage>
          -
          <lpage>123</lpage>
          , Jan.
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>R.</given-names>
            <surname>Sivakumar</surname>
          </string-name>
          and
          <string-name>
            <given-names>P. V.</given-names>
            <surname>Arivoli</surname>
          </string-name>
          , “
          <string-name>
            <surname>Ontology Visualization Protégé Tools - A Review</surname>
          </string-name>
          ,
          <source>” Int. J. Adv. Inf. Technol. IJAIT</source>
          , vol.
          <volume>1</volume>
          , no.
          <issue>4</issue>
          ,
          <string-name>
            <surname>Aug</surname>
          </string-name>
          .
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23] “ECJ.” [Online]. Available: http://cs.gmu.edu/~eclab/projects/ecj/. [Accessed:
          <fpage>14</fpage>
          -Aug-2016].
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>D. R.</given-names>
            <surname>White</surname>
          </string-name>
          , “
          <article-title>Software review: the ECJ toolkit,”</article-title>
          <string-name>
            <given-names>Genet. Program. Evolvable</given-names>
            <surname>Mach</surname>
          </string-name>
          ., vol.
          <volume>13</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>65</fpage>
          -
          <lpage>67</lpage>
          , Aug.
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>S.</given-names>
            <surname>Luke</surname>
          </string-name>
          , “
          <article-title>The ECJ Owner's Manual: A User Manual for the ECJ Evolutionary Computation Library</article-title>
          ,” George Mason University, Version 23,
          <string-name>
            <surname>Jun</surname>
          </string-name>
          .
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>“MASON Multiagent Simulation Toolkit</surname>
          </string-name>
          .” [Online]. Available: http://cs.gmu.edu/~eclab/projects/mason/. [Accessed:
          <fpage>14</fpage>
          -Aug-2016].
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>S.</given-names>
            <surname>Luke</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
            Cioffi-revilla, L. Panait,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Sullivan</surname>
            , and
            <given-names>G.</given-names>
          </string-name>
          <article-title>Balan, MASON: A Multi-Agent Simulation Environment</article-title>
          . .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>S.</given-names>
            <surname>Luke</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
          </string-name>
          <article-title>Cioffi-revilla, L. Panait, and</article-title>
          K. Sullivan, “
          <article-title>MASON: A new multi-agent simulation toolkit</article-title>
          ,” in University of Michigan,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>S.</given-names>
            <surname>Luke</surname>
          </string-name>
          , “
          <article-title>Multiagent Simulation And the MASON Library</article-title>
          ,” George Mason University, Version 19,
          <string-name>
            <surname>Jun</surname>
          </string-name>
          .
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>“VR-Forces: Computer Generated Forces - VT MÄK</surname>
          </string-name>
          .” [Online]. Available: http://www.mak.com/products/simulate/vr-forces. [Accessed:
          <fpage>14</fpage>
          -Aug-2016].
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>N.</given-names>
            <surname>Abdellaoui</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Taylor</surname>
          </string-name>
          , and G. Parkinson, “
          <source>Comparative Analysis of Computer Generated Forces' Artificial Intelligence</source>
          ,” Oct.
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>G.</given-names>
            <surname>Parkinson</surname>
          </string-name>
          , “
          <article-title>AI in CGFs Comparative Analysis</article-title>
          ,”
          <string-name>
            <surname>Defence</surname>
            <given-names>R</given-names>
          </string-name>
          &amp;D, Ottawa, Canada,
          <source>Summary Report, Dec</source>
          .
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>M.</given-names>
            <surname>Hadzic</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Wongthongtham</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Dillon</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E.</given-names>
            <surname>Chang</surname>
          </string-name>
          , OntologyBased
          <string-name>
            <surname>Multi-Agent</surname>
            <given-names>Systems</given-names>
          </string-name>
          , vol.
          <volume>219</volume>
          . Berlin, Heidelberg: Springer Berlin Heidelberg,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>J.</given-names>
            <surname>Euzenat</surname>
          </string-name>
          and
          <string-name>
            <given-names>P.</given-names>
            <surname>Shvaiko</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Ontology</given-names>
            <surname>Matching</surname>
          </string-name>
          . Secaucus, NJ, USA: Springer-Verlag New York, Inc.,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>S.</given-names>
            <surname>Grimm</surname>
          </string-name>
          ,
          <article-title>Studies on the Semantic Web : Semantic Matchmaking with Nonmonotonic Description Logics</article-title>
          . Burke, NL: IOS Press,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>R.</given-names>
            <surname>Studer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Grimm</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Abecker</surname>
          </string-name>
          ,
          <source>Semantic Web Services: Concepts</source>
          ,
          <source>Technologies, and Applications</source>
          . Springer Science &amp; Business
          <string-name>
            <surname>Media</surname>
          </string-name>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>D. R.</given-names>
            <surname>Johnson</surname>
          </string-name>
          , “Enabling Intelligence, Surveillance, and
          <article-title>Reconnaissance Effects for Effects-Based Operations Conditions (Maxwell Paper</article-title>
          ,
          <source>Number</source>
          <volume>34</volume>
          ),” Jun.
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>