Effects-Based Air Operations Planning Framework: A Knowledge-Based Simulation Approach André N. Costa1,2 Paulo C. G. Costa2 1 Institute for Advanced Studies 2 C4I & Cyber Center Brazilian Air Force George Mason University São José dos Campos, SP, Brazil Fairfax, VA, USA anegraoc@c4i.gmu.edu pcosta@gmu.edu Abstract— Planning air warfare operations has always been a operators become scarce resources that usually are centralized complex endeavor. However, as technology evolves at an and not easily accessible by those who conduct the operations. increasingly fast pace, so does the complexity of managing its resources. In modern air operations, planners have to deal with a To put from a different perspective, this centralization of the highly changing environment influenced by enemy air defenses, planning resources results in distancing the plan development weather forecasts, among many other factors, demanding much from those who will execute it, since the required planning effort to handle the great number of constraints and uncertainties resources are hardly available on the operations commands. It presented by them. As a result, a number of decision-support also impacts the agility of the process, especially when systems have emerged attempting to facilitate the planning of air considering highly dynamic mission planning contexts that warfare operations. These systems usually rely on a wide variety usually require re-planning to address emerging situations. of methodologies, which sometimes present a challenge in themselves when it comes to assessing the feasibility and The framework presented in this paper leverages semantic effectiveness of the produced plans. Computer simulations are a technologies to formalize the knowledge required for planning practical way of providing this assessment, usually by running the air warfare operations. The reasoning behind this approach is resulting plans multiple times and checking the results against key two-fold, (1) to avoid the need for highly trained system criteria. Yet, establishing the right criteria, properly accounting planners, and (2) to decentralize the plan building and evaluation for the “fog of war,” and avoiding impractical simulation run process, thus reducing the dependence on heavy computing times and costs are still major challenges. This paper addresses power. such challenges by proposing the development of a decision- support framework that combines ontology-based agile knowledge The planning knowledge to be captured is based on the and a simulation-based mission planning methodology that Effects-Based Approach to Operations (EBAO). This paradigm accounts for the inherent uncertainties that air operations face. has been extensively sought by several research and We avoid costly computation times required by simulation- development efforts in the last decades, paving the way to its intensive course-of-action analyzers by initially pruning the fruition and further application on the field [1]. According to [2], solution space through ontological reasoning. Moreover, the “EBAO informs every aspect of how the Air Force designs, approach complies with the Effects-Based Approach to plans, executes, assesses, and adapts operations.” Therefore, it Operations, having a clear correspondence of processes with it. should guide any framework that proposes to aid the planning The explanations are focused on a specific scenario concerning process of air operations. Following this premise, the backbone intelligence, surveillance, and reconnaissance operations. of the matchmaking process within this work is the Effects- Based Approach to Planning (EBP), which ultimately defines Keywords—ontologies; effects-based planning; modeling and the semantic description of the domain and how it relates to the simulation; semantic matchmaking planning procedure. I. INTRODUCTION Once the initial states of the EBAO knowledge is made The fast pace by which complexity of current military explicit through ontology engineering, the focus of our operations is increasing has become a major challenge for development becomes to provide a solution that does not require mission planners, requiring a much more meticulous planning large amounts of computing power and time. Rather, it may be process to handle all the factors that might influence the done using portable computers by the operations planners. We outcomes of an operation. Several planning methodologies have achieve that by leveraging the EBAO mission concepts via a been observed in the last years, yielding a number of systems to logical engine that pre-selects the possibilities given the support air operations planning. To deal with complexity, these planning data provided, greatly limiting the solution search systems usually rely on heavy computing power, as well as on space. This way, optimization methods can be used in a much specialized operators that must be highly trained in the more effective way, applied to scenarios that are simulated in a methodology associated with the system. Such requirements simplified fashion, and allowing for a quick means of assessing make the planning process brittle, as both the hardware and the the optimization parameters. In a second step, these generated low-resolution solutions are evaluated based on criteria derived STIDS 2016 Proceedings Page 55 from the EBAO ontology. The most promising ones then go Rosenberg et al. [4] suggest a collection of decision-support through a more complex simulation environment that, through tools for planning generation that consists of “a method to entity-level simulation, would provide a much more detailed define an operational scenario, an optimization engine to outcome that includes mission-specific prognostics. generate a diverse set of solutions, and a suite of visualization and analysis tools to review, analyze, and visualize generated For providing a clearer view of how this framework can be plans.” To provide a solution in a timely manner, the authors operationalized, an ISR (intelligence, surveillance, and propose a rapid evaluation of candidate solutions through agent- reconnaissance) scenario is built with unclassified data from the based modeling and simulation (ABMS). They leveraged the Brazilian Air Force. The database includes a number of same software used in our proposed framework. platforms and sensors that have to be assigned to tasks, leading to actions that generate the desired operational effects. Since not Similarly, [5] – also using the same software application – all sensing sources can provide the needed information to task focuses on a Joint Suppression of Enemy Air Defenses (JSEAD) requirements, because the sources are context sensitive [3], this scenario in which plans are generated, optimized, and simulated. assignment can be very challenging to the planning staff. The authors also rely on an ABMS of two sides containing different types of entities, usually targets and air defenses for the ISR operations proved to be a good choice for this initial opposing side and strikers and JSEAD units for the friendly side. scenario, since they contain multiple factors that directly affect Their results illustrated the potential of low-resolution the planning process, and also make its optimization very simulation as a rapid evaluation tool of generated plans, which important. Also, since ISR assets are oftentimes highly complex will be in time described within our own approach. and valuable, a less than adequate planning will lead to the loss of costly flight hours and very specialized crews work. Unlike in our approach, these research efforts do not apply Nevertheless, the application on the planning process of other semantic methods as a form of structuring the modeling and air operations can be made based on the same framework simulation process (e.g. [6]), or as a conceptual basis for the structure, converting the sensor matchmaking phase to a weapon framework as those in the subsection below. to target matching in the case of airstrike or managing electronic warfare (EW) measures on a suppression of enemy air defense B. Semantic Matchmaking Framework (SEAD) mission, both sharing many complexities with ISR The literature on assigning sensors to missions or tasks is operations. vast, but the use of semantic techniques for this purpose is rather limited. Therefore, it is worth pointing out [7], which advocates At this stage of our development, we did not yet reach full for an ontological problem-solving architecture to facilitate circle or obtained conclusive results of a detailed simulation. automated inference of assigning sensors to missions. This work Thus, our focus on this paper is to provide the long-range vision limits the solution domain as a means of including a of the framework, its goals, and an overview of the technical coordination system to emulate the assets and complete bears approaches determined by our preliminary research efforts to similarities with the aforementioned planning simulation solve the challenges encountered so far. frameworks and the one we propose in this paper. The paper is organized as follows. Section II gives a brief One of the most productive solutions is sponsored by the overview of previous research on operation planning U.S. Army Research Laboratory and the United Kingdom frameworks like the one proposed, as well as on semantic Ministry of Defence ([8], [9], [10], [11], [12], [13]). Its authors matchmaking efforts. Section III provides the main concepts conceive a system that relies on a series of ontologies for involved on EBAO, emphasizing the EBP process. Section IV assigning sensors to missions. The backbone of this process is presents the framework, including the description of the the “Mission and Means Framework” that is claimed to “provide software applications to be used on its implementation. Section a model for explicitly specifying a military mission and V focuses explicitly on the semantic part of the framework. quantitatively evaluating the mission utility of alternative Section VI displays the considered scenario, describing warfighting solutions” [12]. The three basic elements of their resources and critical conditions that may influence how the methodology are [14]: image requirements can be met. Finally, Section VII summarizes the paper, pointing to the next steps to be taken. • Top-to-bottom solution to the problem of deploying sensors to meet the information needs of tasks in a II. BACKGROUND mission context; A. Planning Simulation Framework • Combination of reasoning at mission-planning time, and Several planning frameworks are available within the optimization algorithms at mission execution-time; and military research context. However, much of the work available • Dynamic deployment configuration of selected sensor is either too complex or remains inaccessible (e.g., classified). instances by means of a sensor infrastructure. The complexity is often directly related to the high resolution required to generate reliable results. The work includes modular ontologies that cover task requirements, sensor capabilities, and a structured framework to While trying to present an alternative to this complexity associate tasks with sensors. The ontologies specify the problem, some authors have proposed the use of lower requirements of the missions and the capabilities of the sensors resolution simulation and optimization methodologies, which so that the framework is able to decide between combinations of deal with less factors at a time. sensors to satisfy the requirements of a given mission [12]. STIDS 2016 Proceedings Page 56 Even though providing a proven assignment system [3], with uncertainties contained in the air operations. EBP has to fully very well-structured ontologies, this work does not focus on confront the scope and magnitude of these uncertainties, dealing with the uncertainties that a planning scenario presents. especially when dealing with outcome predictions. This is due to the use of logical reasoners and mostly deterministic functions. Stochasticity is not considered, just B. Qualitative modeling comparisons between deterministic possibilities. In addition, the Secondly, in this uncertainty-sensitive framework it is Mission and Means Framework ontology focuses on tasks imperative to possess a trustworthy qualitative modeling, instead of effects. Thus, although providing a direct and clear including frictional, credibility and cognitive factors that are way of breaking down missions [9], the approach does not oftentimes closely related to indirect effects. This is highly emphasize the holistic view advocated in our work. Finally, [15] dependable on the availability of subject matter experts (SME) also provides more details on how this framework may be to provide information about systems and operations. structured as an ontology. C. Agent-based modeling III. EFFECTS-BASED PLANNING The qualitative modeling also requires a focus on decision- Even though utilizing the Mission and Means Framework, making, which can be addressed by agent-based modeling [9] states that alternative mission planning approaches, such as approaches, accurately depicting the C4ISR aspects of the effects-based planning, may be structured in a similar way, with operations. Cognitive models may be housed in agent the goal of assigning resources to missions. architectures, allowing analyzes of emerging scenarios closer “Planning to achieve an effect” has been used naively as a to the reality and with a clear focus on the command and control rather straightforward definition of EBP. However, the vast structure. majority of planners would argue that any previous approach to D. Capability planning military planning would include this asseveration [16]. Therefore, it is imperative to clearly define this concept upfront. Is expected from EBP to determine a range of circumstances that provides degrees of confidence towards the meeting of the The US Air Force doctrine [2] holds that “there is no single conditions that characterize a desired end state. These ‘effects-based planning’ methodology or process. Rather, operational circumstances have to be linked to the necessary understanding the principles of an effects-based approach to capabilities to provide this confidence, not only the necessary operations should yield certain insights and enhance means. comprehension of many general planning concepts”. This is the reason why it is important to first understand what EBAO E. Empirical information means. As stated when speaking of the qualitative modeling, Reference [17] presents the US Joint Forces Command empirical information provided by SMEs is extremely important definition of EBAO as “a process for obtaining a desired for a successful EBP. In addition to that, information from strategic outcome or effect on the enemy through the synergistic history, war-gaming, simulations and experiments should be and cumulative application of the full range of military and strongly pursued so that the complex models can be modelled nonmilitary capabilities at all levels of conflict”. Another and uncertainties reduced. definition presented on [1] is that “effects-based operations are F. Adaptation operations conceived and planned in a systems framework that considers the full range of direct, indirect, and cascading effects, The last principle relates to planning for adaptation. Since a which may—with different degrees of probability—be achieved lot of uncertainties are present and the scenarios may present by the application of military, diplomatic, psychological, and emergent behavior, it is very important to be able to adapt and economic instruments”. dynamically change plans even during execution time. What both of these definitions emphasize is that the process IV. FRAMEWORK of planning has to be much more intentional on the pursuit of a Before presenting our framework itself, we must first holistic view of the operation. There is a focus on addressing not provide the necessary context, which is conveyed in Fig. 1. In only direct physical effects, but several types of indirect effects, EBAO, effects are defined as results of actions. These actions which are influenced by each other. Planners are encouraged to are simply assigned tasks. The ontology described in Section V, maintain a very broad view of the “big picture”, especially during execution, not being caught up in details that can tarnish is used to support a matching process between effects and the end state visualization. resources. The resources in the analyzed scenario are platforms and sensors, which may be mounted to the platforms or not. The A better understanding of our approach requires exploring objectives that defined the desired effects are then translated to EBO’s main concepts, which are described in the next Section. fitness values within the simulation, providing a means for the However, our framework greatly relies on the EBO principles plans optimization. On the tactical level, these objectives form listed below, which were suggested by [1]. a specific mission that, on the operational level, leads to the A. Uncertainty desired end state. The first principle says that effects-based operations (EBO) ABMS is used to represent this mission, possessing planners have to rely on methods that explicitly deal with cognition models that encompass the available expert probabilities and randomness to properly address the inherent STIDS 2016 Proceedings Page 57 information as well as showing the interaction and coordination During the optimization parameters definition there is a need between the agents, representing the C4ISR processes involved. of defining a prioritization of these three basic goals. This With several runs of the simulation scenario, uncertainties can process depends on several factors, such as rules of engagement, be added mostly on the hostile units’ locations, on available value of the assets, and criticality of the mission. These factors capabilities, and on different behavior patterns employed. Also, have to be properly valued by the leadership and then time issues may be initially addressed, since the agents’ parametrized by the analysts to correctly represent the interactions allow for identifying some of the interferences they commander’s intent (CI) on a top-to-bottom fashion. generate on each other through the simulation run. At the end, the framework consists of a deeper and more thorough entity-level simulation with the goal of determining if the conditions that define the end state are met within a feasible timeframe by the previously selected best plan. Also, this phase allows for mission rehearsal and order generation. To summarize, as extracted from [2] and [19], the right- hand side terms of Fig. 1 can be individually defined as: • Resources: all the available assets to generate the desired effects; • Tasks: an action or actions that have been assigned to someone to be performed; • Actions: result of assigned tasks; • Effects: all the physical, functional or psychological outcomes, events or consequences that results from specific military or nonmilitary actions; • Objectives: the clearly defined, decisive, and attainable goals towards which every operation is directed; and • End state: the set of required conditions that defines achievement of the commander’s objectives. As one can notice, the elements presented in the previous Section are met, since the resources are approached as Fig. 1. Air operations planning framework. capabilities and contain several qualitative and empirical information, which also permeates the other concepts of the Generating cognition models can be a strenuous process, framework. Uncertainty is handled through simulation layers, especially when dealing with rule-based scripts for the agent with the ABMS suggestion alongside. Lastly, the design for behavior definition, which, besides being hard to implement, do adaptation is taken in consideration through the process of not capture the uncertainties present on military operations. generation of multiple plans, and mostly by the ontological This is why the approach for the friendly and enemy forces’ reasoning that can quickly change the initial constraints, threat assessment process relies on probabilistic models, such leading to a faster plan evaluation during dynamic re-planning. as Bayesian networks, capable of representing the dependencies Each of the last four boxes on the left-hand side of Fig. 1 is between the entities’ actions and the evidence accrued from the performed by a different software application that are C4ISR sources available. respectively described as follows: From the ABMS, each of the generated plans can be A. Semantic Modeling: Protégé properly simulated and consequently evaluated based on Protégé is one of the most popular knowledge-modelling criteria originated from the three superior goals defined by [18]: environments. It not only allows users to interactively edit flight safety, combat survival, and mission accomplishment. knowledge-bases within its graphic user interface, but also The first goal relates to the need of the pilot to concentrate on presents a series of plugins that add a number of functionalities flying the aircraft in a safe way, for instance assuring that it has and services, such as ontology management tools, multimedia the necessary amount of fuel, that it flies through proper fly support, querying and reasoning engines, and problem solving zones, avoiding collisions with other aircrafts and the terrain. methods. Also, it has experienced several actualizations in the The second focuses on the chances of enduring through the last decades and has a vast user community, featuring high mission, considering the capabilities of the hostile forces and stability and usability ratings. As well as the two following the exposure to them. Lastly, the third goal illustrates the applications, it is written in Java, allowing for a smoother original objective of the mission performed, such as gathering integration in the future ([20], [21], [22]). intelligence information, striking a ground target, or suppressing the enemy’s air defenses. STIDS 2016 Proceedings Page 58 B. Optimization: ECJ various ways. Some of these parts form a large set of utilities ECJ is a general-purpose evolutionary computation and that has the goal of supporting model design. Finally, as well as genetic programming framework designed for large, heavy- ECJ, it is developed and maintained by a research group from George Mason University ([26], [27], [28], [29]). weight experimental needs. It is a free open-source application developed by the Department of Computer Science of George D. High-Resolution Simulation: VR-Forces Mason University. In spite of being more than 10 years old, it VR-Forces is a simulation environment created by VT MÄK shows great stability and an optimized design, attested by a for scenario generation [30]. The platform is widely used large number of users in the genetic programming community. throughout the industry, and provides a well-engineered basis Besides its main goal of attempting to permit as many valid for integrating CGFs with urban, battlefield, maritime, and combinations as possible of individual representation and airspace activity. Apart from the graphical interface (front-end), breeding method, fitness and selection procedure, evolutionary VR-Forces consists of a back-end application, which is its actual algorithm, and parallelism, it contains multi-objective simulation engine. As such, VR-Forces scenarios can be scaled optimization algorithms, island models, master/slave up by running multiple front-ends and/or back-ends, evaluation facilities, coevolution, steady-state and evolution communicating through its networking toolkit. Moreover, both strategies methods, parsimony pressure techniques, and various the VR-Forces front-end and back-end can be extended either by individual representations ([23], [24], [25]). being embedded into another application or through plug-ins, using the C++ API provided. C. Agent-Based Simulation: MASON Reference [31] provides a study comparing several CGF MASON is a single-process discrete-event multi-agent simulation software in terms of autonomy, learning and simulation toolkit written in Java that comprises a fast core adaptation, organization, realism, and architecture. VR-Forces engine and a fully separated visualization display. It is very was considered to be the most suitable as a development versatile and easily expandable, providing friendly licensing platform, mostly because its AI capability built-in, very good options and excellent performance. In addition, it is designed to documentation and technical support, and support for data support large numbers of agents relatively efficiently on a single logger export. The same conclusion was drawn by [32] in a machine in models that are entirely encapsulated. Even the much more thorough analysis. elements of the system itself are highly independent, providing a modular and consistent way to combine its different parts in Fig. 2. EBP concepts and relationships. V. EBO ONTOLOGY This is made very formally, explicitly expressing clear and precise definitions of concepts and relationships [33]. Besides, A. Knowledge-base it provides a domain conceptualization of EBAO as expressed One of the main reasons why this work advocates for the use on Fig. 2, allowing for a better understanding and application of of ontologies for EBP planning is that they can contain detailed its features. information about the military domain in a very structured way. STIDS 2016 Proceedings Page 59 Some of the information contained on our knowledge-base As stated in section VII, our next step is to test this will be presented on the next section, but Fig. 3 shows a implementations to verify if the limitations of classical screenshot taken from the proposed ontology on Protégé. It is description logic matchmaking are significant for in this context, structure from three basic domain concepts: capabilities, sensors generating undesired matching behaviors. If so, other and platforms. The two latter are the resource descriptions, with methodologies may be embraced, such as the use of their properties and limitations taken into account. The first, is nonmonotonic formalisms, such as terminological defaults, related to the actions that can generate the desired effects, but autoepistemic and circumscriptive description logic [35]. also to some of the constraints that can influence the mission, as discussed in the next section. After properly modeling the domain with the most significant parameters and properties, the semantic component of the framework needs to perform the matchmaking between resources and the required tasks for effect generation. This calls for a semantic breakdown of the effects, so that the available capabilities may be used as generation factors for them. After that, the matchmaking methods are able to assign the proper resources as follows. B. Semantic matchmaking The notion of matchmaking consists of a procedure to find correspondences between entities in ontologies [34]. Whereas process is made by several existing techniques, this work will focus on a description logic approach as advocated by [35]. First, it is important to define that matchmaking takes place as a process in which a requester party triggers the mechanism of finding resources relevant to the request., while the provider party describes the available resources in advance. With that, the matchmaking is made through automated analysis and comparisons of the semantic descriptions of the involved resources. For doing so, the entities of an ontology and their relationships have to be carefully modelled, representing the air operations domain as a set of concrete resources that vary on several properties. This variance is intended to allow the specification of the resources, having different parameters. However, due to incomplete information, these specifications not necessarily describe all the parameters completely. To deal with that, the notions of entailment and satisfiability back the testing if all request formulas hold in all models of a knowledge base and if these formulas are logical consequences of it. Fig. 3. Air operations planning framework. There are several matching infrences that are able to account for this variance and that can be directly realised by description VI. SCENARIO logic, ranked in the following way according to their degrees of The proposed scenario represents the definition of a desired matching [36]: effect yielding intelligence requirements. These requirements are influenced by several factors including the resource 1) No match: empty intersection between two descriptions; availability, environment conditions, and hostile activity. Each 2) Intersection: non-empty intersection between two factor imposes restrictions on the matching process. The descriptions; availability is directly related to the instantiation, the 3) Non-Disjointness: non-empty intersection between two environment produces constraints for some sensors and descriptions in every possible world; platforms, and the opposing forces impact on the survivability probabilities as well as on the mission success measurements. 4) Specialisation: subsumption between two descriptions holds from right to left; To illustrate the EBP focus, the chosen scenario contains the 5) Generalisation: subsumption between two descriptions requirement of an effect of assessing, gaining, and maintaining air superiority in support of land and maritime schemes of holds from left to right; and maneuver, as proposed by [37]. The author already exemplifies 6) Equivalence: subsumption between two descriptions how this effect yields ISR actions, such as: detect, discover, and holds in both directions. degrade key components of defense systems; confirm damage to target acquisition radars and height-finding radars. STIDS 2016 Proceedings Page 60 Fig. 4. Scenario factors of influence. With these actions in hand, the system needs to take into • Embraer EMB-145 RS (R-99): twinjet remote sensing consideration the available sensors and platforms to determine aircraft; and which ones are capable of properly performing them. However, their characteristics have also to be confronted with the instance • AMX International AMX-R (RA-1): ground-attack availability, environment conditions and the hostile threats aircraft for reconnaissance. expected, since this will directly influence the matchmaking TABLE I. SENSORS ATTACHMENTS TO PLATFORMS process, as depicted in Fig. 4. Platform Sensors A. Sensors RQ-450 FLIR, SAR Aircraft mounted sensors present different characteristics that may also require different flight altitudes and visibilities in P-3AM FLIR, NCES, SAR order to properly work. Besides, depending on the applications P-95BM NCES, SAR the image demands may be different, providing alternative R-99 CES, FLIR, MSS, NCES, OPT, SAR, information from various sensors. The available sensors considered in this work are: RA-1 FLIR, OPT • OPT: Optical sensors; VII. CONCLUSION AND FUTURE WORK • FLIR: Forward-looking infrared cameras; The main goal of this paper was to provide an analysis of the problem and a preliminary structure of the framework advocated • MSS: Multispectral Sensors; to solve it. The work focused on establishing a theoretical basis • SAR: Synthetic Aperture Radar; for delineating this solution, adapting it to effects-based approach to operations concepts. An added constraint was to • NCES: Non-communications exploitation systems; and utilize free and open source applications to form the framework, at least on its initial phases (the only exception being VR-Forces, • CES: Communication exploitations systems. because of the lack of open alternatives that would provide B. Platforms similar simulation capabilities). Moreover, these applications As showed in Fig. 4 the considered platforms are some of should be light enough to allow for the execution of the the aircrafts used by the Brazilian Air Force as ISR assets. These framework on a single machine, what they arguably are. platforms mount the aforementioned sensors according to The development of the framework not only justifies itself TABLE I. Besides, each one presents different values for range as being an explicit representation of EBAO, but also on the and average speed, which may considerably influence the combination of simulation methods with an initial semantic operations. The aircrafts are: matchmaking process that reduces the solution space, allowing for a potentially more agile way of determining operational • Elbit Systems Hermes 450 (RQ-450): medium size plans. Additionally, the ABMS phase allows for numerous and unmanned aerial vehicle (UAV); fast simulation runs, acting as a fitness evaluation tool for the • Lockheed P-3 Orion (P-3AM): four-engine turboprop optimization process as well as an analyzer of emerging maritime surveillance aircraft; behaviors and complex C4ISR interactions. • Embraer EMB-111 Bandeirante Patrulha (P-95BM): At the time of this writing, only the initial implementations twin-turboprop maritime patrol aircraft; of the ontology described have been performed. Next steps STIDS 2016 Proceedings Page 61 include the full development and implementation of the The SAM Tool,” presented at the 28th Conference on Computer matchmaking process. This step is needed so the optimization Communications, Rio de Janeiro, Brazil, 2009. [14] M. Gomez et al., “An Ontology-Centric Approach to Sensor-Mission can be executed giving continuity to the proposed methodology. Assignment,” in Knowledge Engineering: Practice and Patterns, A. Finally, more information regarding the scenario has to be Gangemi and J. Euzenat, Eds. Springer Berlin Heidelberg, 2008, pp. 347–363. gathered, also allowing for an expansion of its scope, including [15] P. H. Deitz, B. E. Bray, and J. R. Michaelis, “The missions and means gradually more Air Force related activities, for instance airstrike. framework as an ontology,” 2016, vol. 9831, pp. 983109–983109–12. [16] B. Bullen, “A Bayesian Methodology for Effects Based Planning,” Dec. ACKNOWLEDGMENT 2006. [17] I. Duyvesteyn and J. Angstrom, Modern War and the Utility of Force: The authors thank Major Breno Leite, from the Brazilian Air Challenges, Methods and Strategy. Routledge, 2010. Force, for providing data for the scenario construction as well as [18] A. Schulte, “Cognitive Automation for Tactical Mission Management: serving as a subject matter expert, contributing with empirical Concept and Prototype Evaluation in Flight Simulator Trials,” Cogn. information and insights for the modeling. Our gratitude is also Technol. Work, vol. 4, no. 3, pp. 146–159. extended to Danny Williams, from VT MÄK, who was [19] Joint Staff, “Joint Publication 5-0: Joint Operation Planning,” instrumental in providing technical expertise in the VR-Forces Washigton, DC, Aug. 2011. [20] “Protégé.” [Online]. Available: http://protege.stanford.edu/. [Accessed: suite. 14-Aug-2016]. [21] J. H. Gennari et al., “The evolution of Protégé: an environment for knowledge-based systems development,” Int. J. Hum.-Comput. Stud., vol. 58, no. 1, pp. 89–123, Jan. 2003. REFERENCES [22] R. Sivakumar and P. V. Arivoli, “Ontology Visualization Protégé Tools [1] P. K. Davis, Effects-based Operations: A Grand Challenge for the - A Review,” Int. J. Adv. Inf. Technol. IJAIT, vol. 1, no. 4, Aug. 2011. Analytical Community. RAND, 2001. [23] “ECJ.” [Online]. Available: http://cs.gmu.edu/~eclab/projects/ecj/. [2] Joint Staff, “ANNEX 3-0 Operations & Planning,” Washigton, DC, [Accessed: 14-Aug-2016]. Aug. 2011. [24] D. R. White, “Software review: the ECJ toolkit,” Genet. Program. [3] R. Ganger, G. de Mel, T. Pham, R. Rudnicki, and Y. Schreiber, “Sensor Evolvable Mach., vol. 13, no. 1, pp. 65–67, Aug. 2011. assignment to mission in AI-TECD,” 2016, p. 98310E. [25] S. Luke, “The ECJ Owner’s Manual: A User Manual for the ECJ [4] B. Rosenberg, M. Richards, J. T. Langton, S. Tenenbaum, and D. W. Evolutionary Computation Library,” George Mason University, Version Stouch, “Applications of multi-objective evolutionary algorithms to air 23, Jun. 2015. operations mission planning,” 2008, p. 1879. [26] “MASON Multiagent Simulation Toolkit.” [Online]. Available: [5] J. P. Ridder and J. C. HandUber, “Mission Planning for Joint http://cs.gmu.edu/~eclab/projects/mason/. [Accessed: 14-Aug-2016]. Suppression of Enemy Air Defenses Using a Genetic Algorithm,” in [27] S. Luke, C. Cioffi-revilla, L. Panait, K. Sullivan, and G. Balan, Proceedings of the 7th Annual Conference on Genetic and Evolutionary MASON: A Multi-Agent Simulation Environment. . Computation, New York, NY, USA, 2005, pp. 1929–1936. [28] S. Luke, C. Cioffi-revilla, L. Panait, and K. Sullivan, “MASON: A new [6] A. Tolk, Ed., Ontology, Epistemology, and Teleology for Modeling and multi-agent simulation toolkit,” in University of Michigan, 2004. Simulation, vol. 44. Berlin, Heidelberg: Springer Berlin Heidelberg, [29] S. Luke, “Multiagent Simulation And the MASON Library,” George 2013. Mason University, Version 19, Jun. 2015. [7] J. Qualls and D. J. Russomanno, “Ontological problem-solving [30] “VR-Forces: Computer Generated Forces - VT MÄK.” [Online]. framework for assigning sensor systems and algorithms to high-level Available: http://www.mak.com/products/simulate/vr-forces. missions,” Sensors, vol. 11, no. 9, pp. 8370–8394, 2011. [Accessed: 14-Aug-2016]. [8] A. Preece, T. Norman, G. de Mel, D. Pizzocaro, M. Sensoy, and T. [31] N. Abdellaoui, A. Taylor, and G. Parkinson, “Comparative Analysis of Pham, “Agilely Assigning Sensing Assets to Mission Tasks in a Computer Generated Forces’ Artificial Intelligence,” Oct. 2009. Coalition Context,” IEEE Intell. Syst., vol. 28, no. 1, pp. 57–63, Jan. [32] G. Parkinson, “AI in CGFs Comparative Analysis,” Defence R&D, 2013. Ottawa, Canada, Summary Report, Dec. 2009. [9] A. Preece et al., “Matching sensors to missions using a knowledge- [33] M. Hadzic, P. Wongthongtham, T. Dillon, and E. Chang, Ontology- based approach,” 2008, p. 698109. Based Multi-Agent Systems, vol. 219. Berlin, Heidelberg: Springer [10] A. Preece et al., “Reasoning and resource allocation for sensor-mission Berlin Heidelberg, 2009. assignment in a coalition context,” in MILCOM 2008 - 2008 IEEE [34] J. Euzenat and P. Shvaiko, Ontology Matching. Secaucus, NJ, USA: Military Communications Conference, 2008, pp. 1–7. Springer-Verlag New York, Inc., 2007. [11] G. de Mel, M. Sensoy, W. Vasconcelos, and A. D. Preece, “Flexible [35] S. Grimm, Studies on the Semantic Web : Semantic Matchmaking with Resource Assignment in Sensor Networks : A Hybrid Reasoning Nonmonotonic Description Logics. Burke, NL: IOS Press, 2009. Approach,” presented at the 1st International Workshop on the Semantic [36] R. Studer, S. Grimm, and A. Abecker, Semantic Web Services: Sensor Web (SemSensWeb), Heraklion, Crete, Greece, 2009, pp. 1–15. Concepts, Technologies, and Applications. Springer Science & Business [12] M. Gomez, A. Preece, and G. de Mel, “Towards Semantic Media, 2007. Matchmaking in Sensor-Mission Assignment: Analysis of the Missions [37] D. R. Johnson, “Enabling Intelligence, Surveillance, and and Means Framework,” International Technology Alliance, Technical Reconnaissance Effects for Effects-Based Operations Conditions Report 1.3, Mar. 2007. (Maxwell Paper, Number 34),” Jun. 2005. [13] A. Preece, D. Pizzocaro, K. Borowiecki, G. de Mel, M. Gomez, and W. Vasconcelos, “Sensor Assignment to Missions in a Coalition Context: STIDS 2016 Proceedings Page 62