<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Capability-Based System-of-Systems Approach in Support of Complex Naval Ship Design:</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Jacques P. Olivier</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Santiago Balestrini-Robinson</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Royal Canadian Navy National Defence Headquarters</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ottawa</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Canada jacques.olivier@forces.gc.ca</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Georgia Tech Research Institute Georgia Institute of Technology</institution>
          ,
          <addr-line>Atlanta, GA</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Naval ship design can be understood to be a networked System-ofSystems (SoS) multidisciplinary process whereby a decision on one aspect of the design may have simultaneous, multiple order effects on other aspects of the design. Modern naval ship design should therefore consider the systems of interest as components subsumed by a holistic environment encompassing assets and capabilities inorganic to a naval platform. This position paper propose a starting point approach intended to provide a more defined means of establishing and improving the ship design process as part of a multi-layered maritime domain warfare enterprise. Fundamental is the tenet that capability levels transcend several hierarchical echelons and exist across many functional domains. The proposed methodology provides a structured and cohesive approach for identifying and assessing ship capability portfolio with traceable and better known impacts on mission effectiveness, affordability and risk, in the early stages of ship design within the scope of a naval system-of-systems.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>2</p>
    </sec>
    <sec id="sec-2">
      <title>Motivation</title>
      <p>•
•
•
•</p>
      <p>The proposed method is intended to provide a more defined means of
establishing and improving the ship design process as part of a multi-layered maritime
domain warfare enterprise. To achieve this, the design approach is dependent upon
high levels of confidence in the fidelity of the analyses, and is based on shared
understanding and a common language.</p>
      <p>
        Alike best practice in portfolio, programme and project management [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], using
such an approach should deliver a range of benefits which will be revisited throughout
the paper, these include:
• Identifying capability strengths and interests to be maintained, developed and
exploited.
• Identifying capability deficiencies (shortcomings or surpluses) to be remedied
or accepted.
      </p>
      <p>Providing a more structured and cohesive approach to identifying and
assessing ship capability portfolio.</p>
      <p>Creating a common language and conceptual framework for the way to
manage and improve capability-based planning within a ship design process.
Educating stakeholders on the fundamental elements of capability-based ship
design and how they relate to their roles and responsibilities.
• Involving more relevant stakeholders at all levels in the capability-based ship
design process.</p>
      <p>Ranking ship variants based on operational effectiveness, capability and
affordability trade-offs across a spectrum of missions’ priorities.
•
•
•
•
•
•</p>
      <p>Facilitating comparisons, identifying and allowing the sharing of best practice
across major ship acquisition projects within an organisation or a community
of practice.</p>
      <p>Assessing and presenting the findings from a variety of reviews in a format
that is easy to understand.</p>
      <p>
        The aspiration is to show how these benefits can be realised through a
combination of techniques including the adroit use of model-based systems engineering
(MBSE): the formalized application of modelling to support system requirements,
design, analysis, verification and validation activities beginning in the conceptual
design phase and continuing throughout development and later life cycle phases [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Recognizing that MBSE has as its foundation the use of models, the approach
is limited to construct an abstraction of selected aspects of the behaviour, operation,
or other characteristics of a real-world SoS [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The purpose therefore is not to
eliminate all uncertainties and cover all options related to ship conceptual design but to
circumscribe them so to distil a deeper appreciation of the critical factors.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Naval Surface Combatants as System-of-Systems</title>
      <sec id="sec-3-1">
        <title>3.1 SoS in Defence</title>
        <p>
          Applications of systems engineering (SE) and SoS principles abound in
Defence. Indeed, a growing proportion of the acquisition, sustainment, and
management of materiel and non-materiel of military capabilities is sought through a
SoS approach [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. Moreover, the adoption of enterprise architectural framework in
Defence by several nations is a definite step towards providing a more rigorous
approach to life-cycle management including governance, design, building, analysing,
and change management [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]. For instance, the UK Ministry of Defence Architecture
Framework (MODAF) offers the following benefits within the acquisition processes
[
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]:
        </p>
        <p>Improved clarity on the context within which a new capability will operate.
Clearer and more comprehensive requirements documents.</p>
        <p>Improved ability to resolve interoperability issues between systems.</p>
        <p>Better understanding of the mapping of system functions to operational needs
and hence the ability to conduct improved trade-offs.</p>
        <p>The proposed approach aims to utilize an architectural framework similar to
MODAF to embody the SoS elements, unify their capabilities at the appropriate
hierarchical levels, and define their interdependencies to provide a common picture of the
SoS measure of effectiveness (MoE).</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2 SoS in the Navy</title>
        <p>
          Basic sets of architecting principles were proposed by Maier as discriminating
factors to assist in the design of SoS [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ], which later generated five characteristics that
define SoS more appropriately [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. This useful taxonomy may be used to draw the
SoS boundaries for a naval platform, namely: operational independence of the
individual systems, managerial independence of the systems, geographic distribution,
emergent behaviour and evolutionary development.
        </p>
        <p>
          Recognizing that a single platform is a contributing element of a naval SoS, it
follows that we should attempt to define the measures of effectiveness (MoE) of that
SoS. In naval terms, models of hierarchical complexity could be translated into naval
ranks and typology such as those described in Fig. 1 [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. The legend could be used
to characterise the MoE of a naval SoS hierarchically from a naval force capable of
independently carrying out all the military roles on a global scale to that which has
minimal ships’ capabilities and is intended to only perform the most limited of
constabulary functions. Mapping against the typology levels could facilitate the ranking
of ship variants based on potential operational effectiveness and capabilities trade-offs
across a spectrum of missions’ priorities.
        </p>
        <sec id="sec-3-2-1">
          <title>Rank</title>
        </sec>
        <sec id="sec-3-2-2">
          <title>Typology Naval SoS Description 1 2</title>
          <p>3
4
5
6
7</p>
        </sec>
        <sec id="sec-3-2-3">
          <title>Complete Major</title>
        </sec>
        <sec id="sec-3-2-4">
          <title>Global Force</title>
        </sec>
        <sec id="sec-3-2-5">
          <title>Projection</title>
        </sec>
        <sec id="sec-3-2-6">
          <title>Partial Global</title>
        </sec>
        <sec id="sec-3-2-7">
          <title>Force Projection</title>
        </sec>
        <sec id="sec-3-2-8">
          <title>Medium Global</title>
        </sec>
        <sec id="sec-3-2-9">
          <title>Force Projection</title>
        </sec>
        <sec id="sec-3-2-10">
          <title>Medium Regional</title>
        </sec>
        <sec id="sec-3-2-11">
          <title>Force Projection</title>
        </sec>
        <sec id="sec-3-2-12">
          <title>Adjacent Force</title>
        </sec>
        <sec id="sec-3-2-13">
          <title>Projection</title>
        </sec>
        <sec id="sec-3-2-14">
          <title>Offshore Territorial Defence</title>
        </sec>
        <sec id="sec-3-2-15">
          <title>Inshore Territorial Defence</title>
        </sec>
        <sec id="sec-3-2-16">
          <title>Constabulary</title>
        </sec>
        <sec id="sec-3-2-17">
          <title>Defence</title>
          <p>Capable of carrying out all the military roles of naval forces on a
global scale. It possesses the full range of carrier and amphibious
capabilities, sea control forces, and nuclear attack and ballistic
missile submarines, and all in sufficient numbers to undertake
major operations independently.</p>
          <p>Possesses most if not all of the force projection capabilities of a
"complete" global navy, but only in sufficient numbers to undertake
one major "out of area" operation.</p>
          <p>May not possess the full range of capabilities, but have a credible
capacity in certain of them and consistently demonstrate a
determination to exercise them at some distance from home waters, in
cooperation with other Force Projection Navies.</p>
          <p>Possesses the ability to project force into the adjoining ocean
basin. While may have the capacity to exercise these further afield,
for whatever reason, do not do so on a regular basis.</p>
          <p>Possesses some ability to project force well offshore, but not
capable of carrying out high-level naval operations over oceanic
distances.</p>
          <p>Possesses relatively high levels of capability in defensive (and
constabulary) operations up to about 200 miles from shores,
having the sustainability offered by frigate or large corvette vessels
and (or) a capable submarine force.</p>
          <p>Primarily inshore territorial defence capabilities, capable of coastal
combat rather than constabulary duties alone. This implies a force
comprising missile-armed fast-attack craft, short-range aviation
and a limited submarine force.</p>
          <p>Not intended to fight, but to act purely in a constabulary role.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4 Capability-Based Framework</title>
      <sec id="sec-4-1">
        <title>4.1 Capability Definitions</title>
        <p>
          Military concepts generally use a lexicon of frequently interchangeable terms
with sometimes only subtle differences in meaning and often dependent entirely upon
context. For instance, words such as mission, role, function, task, activity, and
capability may have both a descriptive sense (“what”) and a process sense (“how”). The
descriptive sense defines the purpose or basic functions of an organisation and
identifies the precise nature of an operation to be conducted in pursuit of an assigned
mission or objective. The operational sense denotes the precise activities to be undertaken
or achieved which in combination contribute to mission success [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ].
        </p>
        <p>It is recognized nevertheless that there are those essential capabilities which
are common to any naval force at any time, as required to exercise any of the
missions, roles, functions or tasks that might be assigned to it. The degree to which these
core competencies are required and met is predicated upon the needs of the local and
temporal situation. Ergo, they will be considered as capability priorities summarised
by the basic naval concepts of float, move and fight for the purpose of this study.</p>
        <p>
          Of note, the United States Department of Defense (US DoD) defines a
capability as the ability to achieve a desired effect under specified standards and conditions
through combinations of “ways” and “means” to perform a set of tasks [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. This
definition joins the previous definitions in that the “ways” are the strategic and
operational methods describing “how” to conduct military operations to accomplish the
specific military objectives, the “ends”, while the “means” describe “what” resources
are adequate to achieve these objectives within an acceptable level of risk.
        </p>
        <p>
          Lastly, the level of operational capability and the potential response time
constitute the basis for the concept of readiness which is a measure of the ability to
undertake an approved task, at a given time. Four readiness levels are considered in this
study [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]:
•
•
        </p>
        <p>Extended Readiness (ER): Not operational.</p>
        <p>Restricted Readiness (RR): Transitioning between readiness levels or subject
to deficiencies in personnel, materiel and training severely limiting
employment.
• Standard Readiness (SR): Capable of conducting core naval continental and
expeditionary missions that do not entail the possibility of high intensity, full
spectrum combat.
•</p>
        <p>High Readiness (HR): Capable of conducting the full-spectrum of combat
operations.</p>
        <p>As will be seen in the next section, these definitions may be used to create a
common language and conceptual framework that may facilitate identifying capability
strengths and interests to be maintained, developed and exploited; but also identifying
capability deficiencies (shortcomings or surpluses) to be remedied or accepted.
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Cross-Functional Capability Framework</title>
        <p>
          This position paper espouses the tenet that capability levels transcend several
hierarchical echelons and exist across many functional domains. For instance, from a
marine platform systems viewpoint, a hierarchy of equipment-based capabilities
prescribe the minimum materiel standard necessary to support the intent of materiel
safety [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. That baseline level identifies the equipment that must be available for ships to
proceed and remain at sea, i.e., float and move capabilities in higher than restricted
readiness. Other equipment may now be selected to enhance the platform systems
capability levels or elevate the combat systems capabilities enabling fighting at the
standard or high readiness levels.
Fig. 2. Cross-Functional Capability Framework – Example 1.
        </p>
      </sec>
      <sec id="sec-4-3">
        <title>4.3 Capability-Based Planning</title>
        <p>
          Concepts of capability-based planning (CBP) in enterprise architecture can be
invoked to explain that capabilities can be horizontal, going against the grain of
business processes (platform and combat capabilities), or be vertical, being handled in the
context of the business organizational structure (task group, flotilla or squadron) [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ].
Applied to the military context, CBP evolved from threat-based planning, and is
envisaged as the framework that will permit the military forces to optimize their
capacity to respond to the range of plausible missions in which they may be called upon to
serve.
        </p>
        <p>
          CBP is a systematic approach for identifying the levels of capabilities needed
to meet government priorities. Using scenarios, CBP explicitly connects capability
goals to strategic requirement to develop force options more responsive to
uncertainties, economic constraints and risk [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. CBP is thus not estranged to the Defence
realm and its principles were used as a pillar to the proposed ship design
methodology.
5
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Methodology</title>
      <sec id="sec-5-1">
        <title>5.1 Hierarchical Capability Decomposition</title>
        <p>Inspired from hierarchical functional decomposition (HFD) principles, the
proposed approach suggests to decompose, prioritize and recompose capability
requirements through the strategic, operational, tactical and technical levels of
abstractions enabling both the descending “top-down” approach from political aspirations
and the ascending “bottom-up” approach from equipment-level capabilities. The
hierarchy can span any set of functional levels, but it should always include as its lowest
level a tangible set of requirements that can be mapped to physical systems and
performance constraints. The process generates upward, lateral and downward
connections to produce a collectively created and shared picture of the SoS being designed.</p>
        <p>These ship-level platform and combat systems capabilities, which correlate to
system-level key users requirements, could be mapped to tactical-level capabilities
usually pertaining to effectively conduct a combination of naval functions under
prescribed conditions, with other SoS elements. The overall achievement of naval
functions would subsequently propagate up the hierarchy to analyze the effect that a given
set of ship systems capabilities have on higher level operational and strategic
capabilities.</p>
        <p>As shown in Fig. 4, the process involves eliciting capabilities by mapping and
prioritizing strategic-level defence roles with operational-level domestic and
expeditionary missions which are in turn linked to tactical-level naval functions. These naval
functions are the bridge to ship-level capabilities where the SoS is decomposed into
its elements by systems, sub-systems and equipment.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2 Interactive and Dynamic Capability-Based Trade Studies</title>
        <p>This approach creates a dynamic SoS architecture decomposing and linking
high-level organizational goals to key performance parameters. By integrating all
design analyses, including cost models, into a single environment, probabilistic
methods and surrogate models can be used to facilitate parametric trade studies and capture
the propagated uncertainties impacts.</p>
        <p>As summarized in Fig. 5, the interactive and dynamic trade-off studies will
results in design variants at the ship-level capabilities which better define the
performance of the ship independently of mission scenarios, or as an element of a SoS, in
the early stages of ship design. It follows then that when taken as an element of a SoS,
much consideration is applied to create a solution with a higher MoE. The equipment
and systems-level study will generate better key user requirements selected on merit
because they are critical to the achievement of operational needs and the appeasement
of political pressures.</p>
        <p>The intent of the capability analysis is to capture the knowledge and
experience of the subject matter experts (SMEs), so as to allow a decision maker to assess a
large number of potential ship capability combinations without the need to query the
SMEs each time.</p>
        <p>One of the objectives is to unify the stakeholders’ community such that a naval
architect can readily understand the impact of a design configuration or equipment
selection on the effectiveness to achieve a specific mission at the SoS level.
Conversely, a strategist may better understand the technological implications of
privileging a given political defence priority. By involving more relevant stakeholders at all
levels, greater awareness and education may be reached on the fundamental elements
of capability-based ship design and how these canons relate to the stakeholders’ roles
and responsibilities.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3 Visualization</title>
        <p>
          Communicating the potentially complex fused common operating picture
encompassing the interdependencies between domains and disciplines to the stakeholder
community is essential to sharing a collective understanding of the issues. The use of
dashboards is an obvious first choice as they are visual displays that can often
communicate with greater efficiency (can be more intuitive) and have richer meaning than
text alone. Moreover, as exhibited in Fig. 6, a well-designed and customized
dashboard would summarize the information most needed to achieve specific objectives in
a single screen using clear and concise displays mechanisms that are easy to
comprehend [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ].
        </p>
        <p>Strategic C apability</p>
        <p>Geopolitical   Roles
Domestic
Continental
Region
International
73%
72%
72%
Operational C apabilities: M ilitary</p>
        <p>Domestic</p>
        <p>SAR
SOV
OPS</p>
        <p>HA
ACP</p>
        <p>ALEA
Expeditionary</p>
        <p>HUMRO</p>
        <p>MCO
UN  Ch  VI
UN  Ch  VI</p>
        <p>NRFOPS</p>
        <p>NEO</p>
        <p>COIN</p>
        <p>CANUS
STABOPS
6</p>
        <p>Capability P riorities
100%
58%</p>
        <p>29%</p>
        <p>Float Move Fight
Ta#112434656789c#38197147841753tISSSECBRAAAANMSSiCcPBUNeh4RNISSAFERWalIEouDWSRDeGWlwWcVtL Ce eadvp Naeablvi:laiStly FhuGinpacptC i fooanrps abOirlgitaineics Ai31133224332212r Save 31133224332212 31133224332212 ++++#----­­­­------‐‐‐‐­­­­­­1111#1212‐‐‐‐‐‐1
LEVEL  4:  Capability  to  maintain  and  support  multiple  air  assets  onboard  </p>
        <p>(e.g.,  CH-­‐148  +  Firescout)
Cost E stimation
Confidence
100%
80%
60%
40%
20%
0$$%   $   5   0   0                                  $    7  0  0                             11,, $119480460,,270000,,000000  $22100,1110000  ((5900%%   CCII)) $1,300
90%
8</p>
        <p>These visualization methods may assist assessing and presenting the findings
from a variety of reviews in a format that is easy to understand. Ultimately, the
visualization of these outcomes provides the catalyst for decision-makers to more
confidently consider options they would otherwise ignore and move forward based on
wellfounded assumptions.</p>
        <p>It is acknowledged that verification and validation of the characteristics and
behaviours of the SoS comply with the design intent is usually performed while the
systems are being integrated and upon completion of sea trials and ship acceptance.
But as earlier stated, correctly applying MBSE methods within an architectural
framework may improve the ability to resolve interoperability issues between
systems, and improve clarity on the context within which capabilities will operate.
6</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Conclusion</title>
      <p>This position paper proposed an initial approach intended to provide a more
defined means of establishing and improving the ship design process as part of a
multi-layered maritime domain warfare enterprise. The proposed methodology
provides a structured and cohesive initial way forward for identifying and assessing
ship capability portfolio with traceable and better known impacts on mission
effectiveness, affordability and risk, in the early stages of ship design. The epistemic
nature of the proposed process allows the collective generation and evaluation of
scenarios which challenges prevailing mind-sets and presumed correlations between
uncertainties, while reducing subjective interpretations. Again, the purpose is not to
eliminate all uncertainties and cover all options related to ship conceptual design but
to circumscribe them so as to instil a deeper appreciation of the critical factors.
7</p>
    </sec>
    <sec id="sec-7">
      <title>Disclaimer References</title>
      <p>This paper is an unclassified position paper containing public domain facts and
opinions, which the authors alone considered appropriate and correct for the subject.
It does not necessarily reflect the policy or the opinion of any agency, including the
Government of Canada, the Canadian Department of National Defence, or the
Georgia Institute of Technology.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>J.P.</given-names>
            <surname>Olivier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Balestrini-Robinson</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Briceño</surname>
          </string-name>
          , “
          <article-title>Ship Cost-Capability Analysis using Probabilistic Cost Modeling and Hierarchical Functional Decomposition Methodologies,” 11th International Naval Engineering Conference and Exhibition (INEC), Edinburgh</article-title>
          , UK: Institute of Marine Engineering, Science and Technology, May
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Her</surname>
          </string-name>
          <article-title>Majesty's Treasury, “Portfolio, Programme and Project Management Maturity Model (P3M3®) Introduction and Guide to P3M3®, London, UK: The Office of Government Commerce (OGC</article-title>
          ),
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3. International Council on Systems Engineering (INCOSE),
          <source>“INCOSE Systems Engineering Vision</source>
          <year>2020</year>
          (
          <string-name>
            <surname>INCOSE-TP-</surname>
          </string-name>
          2004
          <source>-02-02</source>
          .03),” San Diego, CA, USA,
          <year>September 2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4. Institute of Electrical and Electronics
          <string-name>
            <surname>Engineers</surname>
          </string-name>
          (IEEE),
          <source>“IEEE Standard Glossary of Software Engineering Terminology (IEEE-Std</source>
          <volume>610</volume>
          .
          <fpage>12</fpage>
          -
          <lpage>1990</lpage>
          ),” New York, NY, USA, 28
          <year>September 1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Canada</surname>
          </string-name>
          , Department of National Defence, “
          <article-title>CFCD 129 Readiness and</article-title>
          Sustainment Policy,” Halifax,
          <string-name>
            <surname>NS</surname>
          </string-name>
          : Canadian Fleet Pacific Headquarters,
          <year>October 2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Canada</surname>
          </string-name>
          , Department of National Defence, “
          <article-title>Department of National Defence and Canadian Forces Architecture Framework (DNDAF) Volume 1: Overview</article-title>
          and Definitions,
          <source>Version 1.8</source>
          .1.,” Ottawa, ON:
          <article-title>Directorate of Enterprise Architecture (DEA), 25</article-title>
          <string-name>
            <surname>January</surname>
          </string-name>
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>Her</given-names>
            <surname>Britannic</surname>
          </string-name>
          <article-title>Majesty's Government, “Ministry of Defence Architectureal Framework (MODAF) Acquisition Integrated Project Team (IPT) Community of Interest Deskbook Version 1</article-title>
          .0,” London, UK: 31
          <year>August 2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>M.W.</given-names>
            <surname>Maier</surname>
          </string-name>
          , “
          <article-title>Architecting Principles for System of Systems</article-title>
          ,” 1999 John Wiley &amp; Sons, Inc.
          <source>Syst Eng</source>
          <volume>1</volume>
          :
          <fpage>267</fpage>
          .284,
          <string-name>
            <surname>Chantilly</surname>
            ,
            <given-names>VA</given-names>
          </string-name>
          , USA,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>A.P.</given-names>
            <surname>Sage</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.D.</given-names>
            <surname>Cuppan</surname>
          </string-name>
          , “
          <source>On the Systems Engineeing and Management of Systems of Systems and Federations of Systems,” Information, Knowledge, Systems Management</source>
          <volume>2</volume>
          (
          <issue>4</issue>
          ):
          <fpage>325</fpage>
          -
          <lpage>345</lpage>
          (
          <year>2001</year>
          ), Fairfax,
          <string-name>
            <surname>VA</surname>
          </string-name>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Canada</surname>
          </string-name>
          , Department of National Defence, “
          <source>Leadmark: The Navy's Strategy for</source>
          <year>2020</year>
          ,” Ottawa, ON: Directorate of Maritime Strategy,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <article-title>Office of the Deputy Under Secretary of Defense for Acquisition and Technology</article-title>
          ,
          <source>Systems and Software Engineering, “Systems Engineering Guide for Systems of Systems,” Version 1</source>
          .0. Washington, DC:
          <string-name>
            <surname>ODUSD(A&amp;T)SSE</surname>
          </string-name>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Canada</surname>
            , Department of National Defence, “NAVORD 3000-0
            <given-names>Materiel</given-names>
          </string-name>
          <string-name>
            <surname>Baseline Standard (MBS) - Surface</surname>
          </string-name>
          Ships-Policy,” Ottawa, ON: Royal Canadian Navy, NMPRO 2/ACOS NEM,
          <volume>23</volume>
          <year>April 2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13. The Open Group Architecture Forum, “
          <source>TOGAF® Version 9</source>
          .1
          <string-name>
            <given-names>Enterprise</given-names>
            <surname>Edition - Par III32 Capability-Based</surname>
          </string-name>
          <string-name>
            <surname>Planning</surname>
          </string-name>
          ,” San Francisco, CA, USA:
          <year>December 2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <article-title>The Technical Cooperation Program (TTCP), Joint Systems</article-title>
          and Analysis Group, “
          <article-title>Guide to Capability-Based Planning,” TR-JSA-TP3-2-2004</article-title>
          . Alexandria, VA:
          <year>October 2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15. S. Few, “Information Dashboard Design:
          <article-title>The Effective Visual Communication of Data,” 1st ed</article-title>
          .,
          <source>Sebastopol: O'Reilly</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>