Ontological Support for Living Plan Specification, Execution and Evaluation Erik Thomsen Frederick Reed William Duncan Tatiana Malyuta Barry Smith Charles River Charles River NCOR New York City College of NCOR Analytics Analytics Buffalo, NY Technology, NY, NY Buffalo, NY Cambridge, MA Cambridge, MA wdduncan@gmail.com tmalyuta@thedatascience.org phismith@buffalo.edu ethomsen@cra.com freed@cra.com Abstract—Maintaining systems of military plans is critical for II. THE IDEA OF THE LIVING PLAN military effectiveness, but is also challenging. Plans will become obsolete as the world diverges from the assumptions on which In the current state of military planning – as encapsulated in they rest. If too many ad hoc changes are made to intermeshed Joint Doctrine (JP 5.0) – a distinction is drawn between plans, the ensemble may no longer lead to well-synchronized and deliberate planning and crisis action planning. Deliberate coordinated operations, resulting in the system of plans becoming planning is supply driven. Plans are static information objects itself incoherent. We describe in what follows an Adaptive created as the outputs of a deliberative, rule-governed process, Planning process that we are developing on behalf of the Air and stored in a repository until needed. They may be created Force Research Laboratory (Rome) with the goal of addressing years ahead of actual use, or they may never be used at all. problems of these sorts through cyclical collaborative plan review Crisis actions plans are demand driven: something happened and maintenance. The interactions of world state, blue force and we need an urgent response; because the response should status and associated plans are too complex for manual adaptive involve a degree of organized action, planning is needed. Crisis processes, and computer-aided plan review and maintenance is action planning is a response to the uncertainty involved in our thus indispensable. We argue that appropriate semantic knowledge of real-world states. But even deliberate planning technology can 1) provide richer representation of plan-related rests on an institutional acknowledgement of our inability to data and semantics, 2) allow for flexible, non-disruptive, agile, accurately predict the future, in that Doctrine allows the scalable, and coordinated changes in plans, and 3) support more making of ad hoc resource requests which deviate from the intelligent analytical querying of plan-related data. deliberate plan as specified. Sometimes, on first contact with the enemy, deliberate plans break and workarounds are needed. Keywords—adaptive planning; outcomes assessment; ontology Regardless of the quality of the prior deliberation that went into the deliberate plan, the need for such corrective actions as a I. THE NEED FOR ADAPTIVE PLANNING result of the unanticipated interactions between blue forces and “No plan survives first contact with the enemy” the world make for suboptimal procedures. (Clausewitz, On War). Real world uncertainties all but The goal of the living plan is to remove this ‘breaks guarantee that even the most carefully developed plan will not because it would not bend’ feature of the deliberative plan by be carried out exactly as intended. The military response, as in minimizing the distinction between deliberate planning and the business domain, has been to increase the speed and agility crisis action planning through a new type of planning process of planning and execution [1-4]. On the strategic level, the that is marked by constant update in light of updates in our transition from the Joint Operation Planning and Execution real-world knowledge. The idea is to embed into the very System (JOPES) to an Adaptive Planning and Execution fabric of plan representation our uncertainties about the world, (APEX) system exemplifies this trend. In addition to speeding so that the activity of planning is transformed from one of the up the deliberate planning and review cycle, these efforts seek creation of plans as outputs to a process of continuous plan to increase the number of planned options and contingencies. development. The living plan itself becomes a probabilistic, According to the Adaptive Roadmap II, signed by the branching information artifact – a representation of the Secretary of Defense in March 2008, the ultimate goal is to moment-to-moment intentions not merely of single platoon provide plans that are “maintained continuously within a commanders but of the military as a whole. It incorporates at collaborative environment” to reflect any changes that impact each phase representations of multiple alternative courses of any significant aspects of a plan. Such plans will together form action which are continuously changing in light of actual and something the Adaptive Roadmap calls a “living plan.” Plans projected states of the world, adjacent plans, supporting and may need to be adjusted to maintain their relevance based on supported plans. changes in the world (e.g., weather, location of enemy troops, III. ADAPTIVE PLANNING REQUIREMENTS troop readiness, air assets). Additionally, they may need to be adjusted in order to maintain their coherence within a system We believe that any computational approach to supporting of plans, such as when the goals of supporting or supported the Secretary of Defense’s goal for living plans must meet six plans change. critical requirements. 10 First, it must be able to represent all the types of entities interactive applications that can deal with the sorts of diverse and relationships, knowledge about which is important to but integrated user environments required for living plans. maintaining a living plan. This requires a highly expressive representational capability to capture, manage, and reason Relative to the six requirements described above for over plans, plan elements (e.g., goals, available assets, weather, supporting the Secretary of Defense’s goal for living plans, our battle terrain), and their relations within a system of plans. overall approach is based on the idea that semantic representation of data by means of ontologies, combined with Second, any approach must be able to detect meaningful probabilistic classifiers operating in a transactional changes that impact plan relevance and coherence. This environment, will allow the needed representation, monitoring, requires effective monitoring and sensitivity analysis to identify analysis, sharing and querying of information at distinct levels in a reliable and scalable way those changes which are of of granularity and detail and across distinct applications. The significance to the system of plans [5,6]. Recognition of the system will be required, for example, to display a JFACC’s significant changes must then trigger processes that maintain view of ATO mission plans, a squadron Commander’s view of the relevance and coherence of this system at multiple levels the day’s mission plan, and STRATCOM’s view of a Theater. and across plan elements. As in other domains, the semantic approach is designed to reduce information siloes, and enable effective tailoring of Third, any approach requires coordinated adjustment knowledge and information to different needs. It is designed processes, which are needed to fulfill the second requirement also in such a way as to allow incremental improvements over (above). Such processes must be able to run independently, be time, as shortcomings in the framework uncovered at any given applicable (when necessary) to real-time conditions, and be stage are rectified in subsequent stages. capable of harmonizing with other large-scale plan adjustments. In what follows we focus on the first and fifth requirements described above: for rich representations of data and semantics, Fourth, any such approach requires automated information and for the capacity to use such representations in mounting extraction and routing because maintaining realistic plans queries against plan-related data. requires more information processing than can be achieved through manual methods alone. As regards the former, we describe the coverage domain of our proposed Plan Ontology (see Figure 2) in terms of how we: Fifth, whether in support of human planners, warfighters (a) model plans in terms of cyclical phase-specific attributes; during mission execution, operations assessment staff, or (b) embed metrics that relate plans to world conditions; and (c) automated systems performing the same tasks, any approach embed meta-metrics that use the metrics under (b) to create an needs to support analytical queries against the ensemble of incremental plan and plan-execution improvement process plan-related data. Since plan-related data is very hetero- across the whole system. On each level multiple families of geneous, this amounts to applying a unified structured query related terms will be required, including definitions and axioms front end to structured and unstructured data on the backend. specifying the relations between them. Sixth, joint warfighters at all levels of command will need As regards the latter, we describe how queries are passed to collaboratively plan and execute in conjunction with semi- through parts of the system in order to illustrate some of the automated adaptive planning systems. Therefore, any approach semantic relations that need to be computed in order to support for providing living plans must support extensible and versatile analytically useful queries over living plan data. Figure 1: Fragment of the draft Plan Ontology at http://ncor.buffalo.edu/plan-ontology 11 IV. REPRESENTING PLANS IN RECURRING PHASES the goal has been achieved, or because the plan is no To better understand and support the notion of continuous, longer relevant or coherent, or is being executed living plans, we require a view of planning that is more unsatisfactorily. abstract than is traditionally employed. The simplistic notion of x post-execution – This phase involves the post- ‘the plan’ created prior to ‘the execution’ is at odds with our execution processes of interpreting and judging an view of planning as a dynamic, continuous, iterative process executed plan and its outcomes relative to that not only adapts to the effects of planned actions, but also expectations. In this process, all actions taken under adapts the process of planning itself in ways designed to commitment to the plan have been taken. Thus their achieve more satisfactory outcomes over time. net effect can be assessed relative to the specified goal. Our model focuses on three primary factors in the planning The primary purpose of the processes involved in this process: post-execution phase is to enhance future planning, for example by: 1. different phases of the planning process (successive phases within a given course of planning processes), o defining new goals; 2. types of judgments within each of those phases that o clarifying existing goals; enable effective planning, and o improving effectiveness in achieving goals. 3. information, including metrics, on which these Associated with processes of each of the mentioned types judgments are grounded. are four basic planning-related judgments that enable reasoning On the traditional view, planning only happens periodically aimed at leading to the creation and selection of better plans: as a precursor to its execution. Here, in contrast, we view the x relevance – How well does the current state of total planning process computationally as forming a series of planning relate to actual or anticipated external world parallel, interacting courses or flows at a number of different conditions, such as constraints, opportunities, planned levels. These processes unfold dynamically, with changes in outcomes, unplanned side-effects, etc.? any given course being communicated to parallel and hierarchically related courses wherever changes in the latter are x coherence – How well do the processes of planning required. The system is organized in such a way that updated on-going in the current phases relate to other versions of needed plans and subplans can be generated at any synergistic planning processes. In other words, are they point in time. in conflict or coherent with other friendly force, coalition, political, etc. planning? Each parallel course is itself seen as being organized into a succession of three phases corresponding roughly to the first x planning-assessment – How well were the processes in three phases of the well-known Plan Do Check Act (PDCA) each phase of planning performed by the planner, from cycle, and similar models. A difference is that the phases in our a single person to an organization? framework are viewed as continuous and intermeshed with each other rather than discreet. Especially the Act phase, where x meta-metric learning – How well does the current set adaptive actions are taken, is distributed and continuous across of metrics support the goal of evolutionary the other phases. improvement of the entire planning process (and, as a consequence thereof, the entire process of creating and x development – This phase consists of processes of executing and evaluating plans)? identifying, considering, selecting, constructing, and/or modifying potential courses-of-action (COAs) that are V. REPRESENTING RECURRING CLASSES OF METRICS IN expected to satisfy a goal. This includes the process of SUPPORT OF CYCLICAL PLAN PHASES creating and maintaining potentially executable ‘plans sitting on the shelf’ in traditional, deliberate planning – In this section, we bring together the three factors of referred to in our ontology as ‘plan specifications’. The planning outlined above – phases, judgments, and metrics – to distinguishing feature of this phase is that there has see how they merge to form a more complete picture of a been no decision to take actual actions in conformity continuous adaptive planning process. For each combination of with and under commitment to any specific plan. planning phase and judgment we provide example metrics. These are provided here for illustrative purposes only, and x execution – This phase involves processes of planning especially as concerns plan execution our framework will draw while acting according to a particular planned COA. on the extensive list of Measures of Effectiveness and Unlike random or spontaneous actions, such planned Performance identified in salient doctrine for the tasks of the processes can be evaluated relative to the plan. For Universal Joint Task List, for example as described at: example, indicators can be used to judge whether the http://www.dtic.mil/dtic/tr/fulltext/u2/a398683.pdf intermediate effects of planned actions are consistent with expectations. But, as the plan has not yet A. Plan Development Phase terminated, the net effect of all planned actions relative to the goal set forth in the plan cannot be judged. A 1) Relevance key planning process in the execution phase is the Example metrics informing the judgment whether a making of a decision to terminate execution because potential plan will be relevant to some anticipated world state: 12 x Values and locations of relevant adversary assets (a learning often requires data over combinations of planning plan to invade a country to remove WMD stockpiles phases. would be irrelevant if there were no stockpiles, or if existing stockpiles were unreachable in a timely x Inter-phase meta-metrics deriving from manner) correlations between some earlier-phase metric with some later-phase metric relating to outcomes x Number of red operational defensive SAM sites (a (for example: if the number of options embedded in plan that did not either act to reduce this number, or COAs has historically correlated positively with post- account for blue attrition because of them, would not execution assessment metrics indicating greater be relevant) satisfaction of plan goals, then it may become a more positive metric that is given greater weight in future x Number of blue re-fueling tankers available during plans) a period (a plan with more missions than could be supported for refueling would not be relevant) x Correlation between intra-phase metrics generally considered positive (or negative) (for example: the x Network of adversary command communications (a number of COA options considered is itself to be plan that intends to cripple communications by taking viewed as a positive metric; but if this number goes up out a central node is not relevant if the network is in such a way that the time required to bring a plan to decentralized and/or has alternate paths) execution goes up at the same time (which is 2) Coherence considered negative), then this suggests an optimization is possible, or perhaps a different metric, Example metrics informing the judgment of whether a such as measuring the difference in time between potential plan will be coherent with other related planning: completing a plan and its estimated time of execution x Rates of attrition of shared assets (a plan that over- rather than total time) optimistically assumes assets will remain available x Percent of relevance and coherence metrics with after another plan executes is not coherent) measures above a certain level of belief/confidence x Times of anticipated/actual actions that are signs of (over time, the confidence in metrics should be driven intentions (a plan that assumes an element of surprise up, for example the confidence in metrics of adversary is not coherent with another plan that takes earlier state such as number of SAM sites should be actively actions that signal a shared or related intent) improved with better sensors and analysis processes) x Intentions of non-military planning in Area of x Number of corrections made to a metric Operations (a military plan that depends on large- (‘corrections’ means: substantial changes in a metric scale destruction of economic infrastructure, apparatus which are made on the basis of evidence contradictory of civil authority, etc. is not coherent with a political to the original estimate of what sort of metric would be plan that seeks to rapidly restore civil rest and order) needed; for example: contradictory evidence that the current WMD estimate, made by whatever process, is 3) Planning Assessment wrong leads to improving the process that led to this Example metrics informing the assessment of planning estimate). performance during plan development: B. Plan Execution x Time required to reach plan execution phase (compared to predicted, needed, historical, and so on) 1) Relevance Metrics informing the judgment whether an actual plan x Number of substantially different COAs and being executed remains relevant to actual conditions, such as embedded options considered (based on the constraints and opportunities: assumption that the larger the number of options the better is the understanding of the space of options) x Cloud height over intended target (may violate constraint of target visibility) x Number of relevance and coherence metrics considered (by some definition of considered and a x Number/rate of adversary unit surrenders or other procedure for counting separate metrics) change in adversary offensive activity (may indicate plan assumptions regarding adversary’s will to fight x Length of review chain prior to approval by are incorrect or not relevant) Commander (includes first-pass and re-review cycles) x Aggregate Measures of Performance (MOPs) for 4) Meta-Metric Learning current actions (low levels of mission performance Example meta-metrics describing how well the relevance, may indicate that the pre-conditions and contexts for coherence and planning assessment metrics support plan actual actions were not satisfactorily planned – for development, and enable improvement of the metrics – and example low levels of destroy, degrade, deny, disrupt thus of the total planning process – over time. Meta-metric (4Ds) may indicate poor intelligence, weaponeering, etc.) 13 2) Coherence These are metrics informing the judgment of the effects of the executed plan on world state, particularly relative to intend- Example metrics informing the judgment whether a plan ed outcomes. In addition to the more typical post-operations remains coherent over time: assessment process, there are other ways to conceptualize post- x Changes in planned asset availability committed by execution relevance. For example: do the lessons drawn from other plans (for example: there are assets which the assessment have relevance to the current or future world? Is the plan assumes other plans do not require) originally desired outcome – such as destroying (or building up) another actor’s offensive capability (for example arming x Success rate of synchronization points (if plans have the Taliban) of continued relevance? Or is it becoming less explicit specifications of COA relationships, defined- relevant, for example because they have changed sides? execution windows, handoffs, meetings, supporting events, and so on, then what is the rate at which these x Number of missiles landing in homeland (this is said relations are successfully maintained?) to have been the post-execution operations metric for the recent Gaza invasion) 3) Planning Assessment x Number of computer systems not patched for Example metrics informing the assessment of whether the exploit X (exploit X might have worked well on this plan is being executed satisfactorily: occasion, but if the adversary has since learned about it x Percent of scheduled missions flown on time and therefore patched the prior vulnerability, the (assessing compliance with plan, not outcomes) simple assessment that it worked well previously is not particularly relevant for future planning) x Rate COA modifications made per unit time (a better specified plan might require a lower rate of 2) Coherence modifications) Metrics informing the judgment how the net outcome is x Aggregate time delays of actual execution for coherent with other plans (in any phase) planned simultaneous actions (for example in mass- x Actual asset attrition (for example: achieving the ing fires in planned combined air strike and artillery) current plan objective with more or fewer bullets may x Time from a relevant change in world state to the not matter to the current plan, but it may harm/limit appropriate change in COA (for example: time from other planning. This is following the notion that when the new target location information is obtained to Relevance is assessing the relation of the outcome to time when a new mission tasking has been created that the current world state, so Coherence would be the accounts for the new information) relation between the outcome and other plans.) 4) Meta-Metric Learning x Degree to which actual net outcome facilitates or limits COAs of future plans (e.g., confident removal Meta-metrics describing how well the relevance, coherence of WMD threat makes other plans easier to develop and and planning assessment metrics support plan execution, and execute) enable improvement of the metrics: 3) Planning Assessment x Inter-phase metric correlation (for example: low correlation between missions flown on time and post- Metrics informing the judgment of how well the post- execution MOE metrics may suggest that flight execution planning process is performed: promptness is not as important as thought, perhaps x The number of indicator metrics integrated into the because late flights were able to act on better, more overall goal assessment (for example: if goal end-state recent information) is to influence future behavior, then more indirect x Intra-phase metric correlation (for example: a nega- present indicators would potentially lead to better tive correlation of rate of COA changes and aggregate inference of future behavior tendencies) time delays of planned simultaneous actions may sug- x The fraction of actually executed missions for which gest that allowing more frequent COA changes to con- a reliable measure of performance exists (for how structively maintain coherence is beneficial, notwith- many missions do we have the metrics needed to assess standing the expected disruptive effect of the changes; mission performance? for any given mission, how many better metrics might distinguish COA changes by class salient performance metrics are we actually capturing of initiating event, such as new information, command for that mission?) decision, and so on; as the framework itself becomes more sophisticated in its reasoning power, more x The number of lessons-learned distributed (clearly frequent COA changes will themselves become more depends on how lessons and distribution are counted) easily accommodated by the planning system) 4) Meta-Metric Learning C. Post Execution Meta-metrics describing how well the relevance, coherence and planning assessment metrics support plan assessment, and 1) Relevance enable improvement of the metrics: 14 x Inter-phase correlation (e.g., correlation of lessons- As stated, the metric is conditioned on a user’s learned distributed and follow-on planning preparation specification of a plan. Given a plan, the metric represents the metrics over time might suggest little relationship percentage of operational anti-aircraft missile sites by area-of- between the two. Perhaps the value of the lesson should operations for the specified plan. The query processor thus be included in the metric, or independently, whether the needs to be able to ascertain area-of-operations associated with lesson-learned changed any process) a given plan, something which could possibly vary over time. x Intra-phase correlation (e.g., no correlation between A. Indirect identification of plans asset attrition and assessment of satisfaction of goal state suggests that it might valuable to distinguish Even the identification of the plan may be a non-trivial between “productive” and “unproductive” attrition) exercise. While in theory it may be possible to use a unique plan identifier to locate the desired plan, in practice the plan may be identified indirectly in a number of ways, such as: VI. ONTOLOGY-DRIVEN QUERYING OF PLAN INFORMATION The kinds of representations described above are necessary to x Attributes: Using combinations of attributes such as support Living Plan requirements. But they are not sufficient. plan phase (development, execution or post- execution), Commander in charge of plan execution, Without the query support to populate them, the approval date, and so on. representations are vacuous. Since the underlying living plan- related data requires the inference-based identification of x Containment: Identifying related plans through objects with associated attribute and location information relations of containing or being contained within other under conditions of uncertainty, ontology-driven query plans: the AOP (Air Operations Plan) is contained mechanisms will need to include probabilistic functions in within a specified Joint Campaign Plan, or conversely, addition to more traditional deductive ones. for a Campaign Plan that contains a specified AOP. Consider the following metric where we have underlined x Assets: By relating a plan to the assets associated with ontology terms to be used by the Living Plan framework: it during a given time frame, as when an AOP is tasking Squadron X in some given week. the percentage of operational anti-aircraft missile sites by area-of-operations for some given plan specification. x Operational relation: For example, one plan precedes or succeeds another as pre-condition or sequel. Or two Such a metric would be useful in determining the progress of plans relate to each by having mutually dependent an operational objective for example related to suppression of executions. air defenses. Though seemingly straightforward, even this metric raises a number of interesting semantic challenges that need to be resolved by a query processor. Figure 2: I2WD ontologies at http://milportal.org 15 One or more of these methods could be used in the query to threat. In short, the process of identifying and counting sites identify the desired plan, requiring the query processor to apply may be substantially different according to whether they are additional knowledge of plan attributes and relations to operational or non-operational. To provide appropriate properly parse the query to eventually locate the desired plan measures of confidence in the associated metrics, the query and its area-of-operations. processor would have to know what sorts of biases to consider and their relative magnitudes in terms of attributes such as power projection capability, which will be defined in our B. Ontology-driven queries ontology framework. The complexity and dynamic nature of relationships between the plans and the involved information cannot be A likely more difficult counting complication would arise adequately represented in non-semantic technologies (for from semantic assembly of information regarding the very example in traditional databases). Moreover, direct traditional attribute of being operational as applied to sites. Whether a site querying of such representations will be difficult to automate is operational may be difficult to determine for multiple and maintain in the necessary flexible manner, and the results reasons. For example, if a site loses some part of its targeting of such querying may not be capable of the needed rapid capacity but retains ability to launch, then it is operational as a update to incorporate emerging important data. Our hypothesis, launch site, but without targeting it will pose little threat to therefore, which draws on the work described in [7,8] is that a modern aircraft. The state of the site may also be time- comprehensive and incrementally evolving set of Living Plan dependent; for example, a site that is partially degraded could ontologies, drawing on the I2WD suite of ontologies (see be anticipated to be restored at some point in the future. Such Figure 2) can provide the needed nuanced representation of the expectations would depend on the nature of the degradation plans, metrics, and of the semantics of the source data against and the resources available to make repairs and restore which the querying is performed, while taking account of operation. At any particular time, the query processor would relationships between all of these components. Such an have to combine operational state attributes based on reports approach will lay a foundation for sophisticated querying and from different times and with varying levels of confidence analytics enhanced by inference, and is designed above all, to arising from uncertainty in expectations as to whether a site enable agile changes to all components. Additionally, the will remain operational. ontology framework will have to include representations of Other complications might arise in classifying a site as complicating factors such as those described below and their functioning or not functioning as an ‘anti-aircraft missile site’. relationships with the plans and metrics. It is certainly possible that the raw intelligence information and sensor data on which counts are made will not directly and C. Probabilistic ontological classifications unambiguously classify a facility as an anti-aircraft missile site. Instead, there may be reports of a more specific nature (for One example complication concerns the identification of example, that we are dealing with a specific type of missile the location constraint for those sites that are to be considered capability) which through interaction with weapons ontology because they lie within the area-of-operations. The problem would be determined to qualify more generally as ‘anti- turns on the fact that there may be sites physically outside this aircraft’. On the other hand, some reports may refer only to a area that are identified as harboring capabilities that project ‘missile site’, which would then require further inference to into the area-of-operations. This may imply an ambiguity at the determine if the site is likely to have a more specific type of operational level. If the focus is on assessing the performance anti-aircraft capability. Such inferences generally require the of missions to disrupt or destroy sites physically within the knowledge of type-subtype relations and the attributes on area-of-operations, then the metric should be interpreted in one which such classifications are based. For example, information way. If, on the other hand, the intent of the metric is to assess about a missile site supertype could be inferred to be also of the the security of aircraft within the area-of-operations, then the anti-aircraft missile site subtype through examination of other better interpretation may extend the focus to include sites that potentially known attributes, such as size and location of the have an air defense capability that reaches into the area-of- site, imagery features, connectivity to other assets, and so on. operations from outside. In order to properly respond to a Such information will be incorporated as probabilistic query based on the latter interpretation, the system would need functions into our ontology framework. to be able to infer such projection capabilities and perform spatial reasoning to find substantial intersections with the physical boundaries of the area-of-operations. Such capabilities D. Missing, inconsistent and other invalid data may depend on the type of missiles available, requiring further Considering the fog of war, some information will at any information about specific missile capabilities and deployment. given stage be incorrect, inconsistent, or missing. Barring independent evidence to the contrary, incorrect information, Another potential complication is bias in the identification such as a site being reported as operational that is not, cannot of individual sites for counting. For example, the adversary be rectified. However, when there are multiple reports in might expend additional effort to hide remaining operational conflict, it may be possible to reach a most likely conclusion. A sites rather than sites that may have already been degraded in query processor that maintains, or has access to, meta- some way. Conversely, missile firings from operational sites information regarding the typical or historically-observed make them more difficult to hide. At the same time, own- believability of reports from various sources can combine forces may not expend as much effort in identification and conflicting reports as weighted evidence to reach a most counting of non-operational sites as those which still pose a believable conclusion. The needed provenance-related 16 attributes, too, will be incorporated into our ontology of the ratio of operational to non-operational sites over a given framework. area of interest. A conflict in evidence may be due to understandable reasons, the simplest being that they were made at different VII. CONCLUSION times in relation to something that is changing, such as the state To support the Secretary of Defense’s vision for Living of a missile site. A more complex case would involve the Plans, we believe that plan-related ontologies need to be ability of different sources to provide substantial evidence at extended into two areas: different times or under different circumstances. For example, prior to actually observing an anti-aircraft missile site launch a x A generic planning process ontology that is based on weapon, a determination of its state of operation may be the Information Artifact Ontology and that takes into difficult to establish. An intact-looking site might be non- account the cyclical process of planning. operational for reasons that are not directly observable, such as x Ontologies containing representations of each of the broken electronic or computer-based equipment. Under these kinds of attributes and relations needed to identify circumstances, direct observation might provide credible desired plans according to relevant areas-of- evidence of non-operational status (the physical structure may operations, assets, capabilities, and so forth. be visibly degraded or destroyed), without being able to provide evidence of operational status. Intelligence reports Additionally, the query processing component of any plan- from intercepted communications would be a better source of related computational framework that converts potentially information under these circumstances, but only if they are to huge stores of plan-related expressions (data types, values, be believed as genuine and not intentional misinformation. Of natural language expressions), into user-oriented actionable course, direct observation of a successful missile launch at a metrics needs to be aware not merely of the ontologies, but later point in time would over-rule any prior assertions about also of the needed types of deductive transformations and, as the site’s state, but only until contravening reports are later received indicating that its state may have changed, such as a we showed above, of probabilistic classifications. battle damage assessment that it was successfully struck and Materialized query processing tools will rely on the destroyed at an even later point. principles set forth in [7, 8] which are being used to integrate diverse data in a variety of disciplines. The approach is Such issues, related to reports of the changing state of a designed to achieve integration in an agile, flexible and missile site, may be interpreted differently depending on the incremental way, and also to incorporate into our system the purpose of the associated metric. If the intent is to assess ontology content created for related purposes by our progress of given actions toward an operational objective of collaborators in different military domains and disciplines. reducing the risk of operations in a given airspace, then the most important information is the conversion through those REFERENCES actions of known operational sites into non-operational sites. In [1] Mintzberg, H. (1994). The fall and rise of strategic planning. Harvard that case, for example, it would be less important to know Business Review, 72, 107-114. which sites were non-operational for other reasons prior to the [2] Grant, R. M. (2003). Strategic planning in a turbulent environment: start of the campaign. At the same time, the change in state of a evidence from the oil majors. Strategic Management Journal, 24, 491- particular site would presumably be the effect of some action 517. [3] Davis, P. K. (2012). Lessons from RAND’s Work on Planning Under taken, and such information would aid in the interpretation of Uncertainty for National Security. DTIC Document. the action reports. For example, if the site were observed to be [4] Boukhtouta, A., Bedrouni, J., Berger, J., Bouak, F., and Gulati, S. launching missiles prior to a kinetic strike on the facility and (2004). A Survey of Military Planning Systems. International Command no launches were observed after the strike, it would be and Control Research and Technology Symposium (ICCRTS). reasonable to believe that the strike had its intended effect in [5] Pollack, M. E. and McCarthy, C. (1999). Towards focused plan rendering the site non-operational. On the other hand, if the monitoring: A technique and an application to mobile robots, Proceedings of the IEEE International Symposium on Computational metric is being used primarily to ascertain the relative risk of Intelligence in Robotics and Automation (CIRA99), 144-149. operations in that airspace, then the numbers of operational and [6] Boutilier, C. (2000). Approximately optimal monitoring of plan non-operational sites prior to the campaign become important, preconditions. Proceedings of the Sixteenth Conference on Uncertainty as well as the previously-discussed issue of sites being restored in Artificial Intelligence, 54-62. to operation over time. [7] Smith, B., Malyuta, T., Mandric,k W.S., Fu, C., Parent, K., Patel, M. (2012). Horizontal Integration of Warfighter Intelligence Data: A Shared In addition to incorrect and conflicting information, the Semantic Resource for the Intelligence Community, Proceedings of the query processor must also deal with missing information. In Conference on Semantic Technology in Intelligence, Defense and Security (STIDS), George Mason University, Fairfax, VA, October 23- some circumstances reports may be available only for certain 25 (CEUR 996), 112-119. time periods, or concerning certain types of information. For [8] Salmen, D., Malyuta, T., Hansen, A., Cronen, S., Smith B. (2011). example we may have reports on site location without state of Integration of Intelligence Data through Semantic Enhancement, operation information, or only assertions of being operational Proceedings of the Conference on Semantic Technology in Intelligence, but not of being non-operational. Such differences in missing Defense and Security (STIDS), George Mason University, Fairfax, VA, November 16-17 (CEUR 808), 6-13. information will add complexity to making a reliable estimate 17