<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Securing Navigation of Unmanned Maritime Systems</article-title>
      </title-group>
      <pub-date>
        <year>2018</year>
      </pub-date>
      <volume>9506</volume>
      <fpage>53</fpage>
      <lpage>62</lpage>
      <abstract>
        <p>Copyright ' by the paper's authors. Copying permitted for private and academic purposes. In: S. M. Schillai, N. Townsend (eds.): Proceedings of the International Robotic Sailing Conference 2018, Southampton, United Kingdom, 31-08-2018</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Unmanned Maritime Systems (UMS), such as Unmanned Surface
Vehicles (USVs), are increasingly playing a critical role in expanding the
undersea superiority of a nation, addressing growing challenges, such
as, inter alia, in piracy, natural resource disputes, drug trafficking,
weapons proliferation, as well as being highly used for science and
survey missions. Autonomous capabilities in USVs can reduce the costs of
reaching into distant environments and using that reach to meet a
particular mission’s objectives. However, to take on increased autonomy
in unmanned systems, USVs will increasingly require the ability to be
untethered from human interaction, and a key enabler to effecting this
is accurate navigation. USVs have traditionally depended on Global
Navigation Satellite Systems (GNSS), which are known to have
security and safety vulnerabilites. Using systems-theoretic process analysis
(STPA), this paper provides systematic analyses of the attack surfaces
and of the impact of cyber attacks against the navigational aspects
of Unmanned Surface Vehicles. As part of these analyses, we identify
potential threats, vulnerabilities and attacks in the Positioning,
Navigation and Timing (PNT) functionalities of USVs. These analyses can
be used to drive a USV’s architecture, leading to the design of more
effective and secure USV operations.
1</p>
    </sec>
    <sec id="sec-2">
      <title>Introduction</title>
      <p>Over 90% of information, people, goods and services flow across the worlds oceans (Navy, ). Protecting a
country’s residents and economic prosperity is, therefore, essential and dependent on the ability to persistently
monitor ocean surface and sub-surface activities, in order to identify, classify and mitigate emerging threats.
Unmanned Maritime Systems (UMS), such as Unmanned Surface Vehicles (USVs), are increasingly playing a
critical role in expanding the surface and underwater superiority of a nation, and addressing growing challenges,
such as, inter alia, in piracy, natural resource disputes, drug trafficking and weapons proliferation. USVs are
also employed in other areas of economic life, such as (a) Maritime search and rescue, (b) Hydrologic surveys,
(c) Port surveillance, (d) Underwater Inspection, (e) Naval Defence, and their greater use could save the global
marine industry up to £80 billion per annum by “potential reductions in capital costs, manning costs and fuel
costs” (Rolls-Royce, ).</p>
      <p>Because of their position at the air-sea interface, USVs have the ability to relay radio frequency transmissions
in air and acoustic transmissions undersea. Thus they are a key piece in the vision of networked maritime space,
both in defence and civil. Figure 1 shows an example commercial USV, while figure 2 shows an example of how
BP is making use of UMS in its networked maritime space (the picture shows a number of USVs, e.g. C-Worker,
WaveGlider and Autonaut, plus some underwater vehicles, such as the Seaglider, working together to monitor
the ocean surface and the seabed).</p>
      <p>Autonomous capabilities in USVs can reduce the costs and risks of reaching into distant environments while
using that reach to meet missions’ objectives. Marine vehicles are taking on higher levels of autonomy to perform
unmanned missions, therefore securing their autonomous navigation and control modules becomes increasingly
important.
2</p>
    </sec>
    <sec id="sec-3">
      <title>Autonomy, Navigation and Control</title>
      <p>
        Autonomy incorporates “systems which have a set of intelligence-based capabilities that allow it to respond to
situations that were not programmed or anticipated in the design (i.e., decision-based responses). Autonomous
systems have a degree of self-government and self-directed behavior (with the humans proxy for decisions)”
        <xref ref-type="bibr" rid="ref2">(Air Force Research Laboratory, 2013)</xref>
        . USVs must be capable of avoiding ships, docks, floating debris, and
navigation aids must ensure that these USVs are able to avoid these obstacles and other marine assets, and remain
in navigable waters. In addition, USVs must operate in accordance with collision regulations (COLREGS)1.
Because not all maritime traffic (including military and commercial) always follow the COLREGS, therefore,
assured autonomous navigation and control are very important to develop and maintain in USVs.
      </p>
      <p>There are a wide range of definitions of autonomy depending on the field. In the maritime sector, the most
widely agreed definition is given in (UK, ), and has six levels. These levels are, in ascending order of autonomy:</p>
      <sec id="sec-3-1">
        <title>Level 0: Manned; Vessel/craft is controlled by operators aboard</title>
        <p>Level 1: Operated; Under Operated control where all cognitive functionality is within the human operator
Level 2: Directed; Under Directed control some degree of reasoning and ability to respond is implemented
into the Unmanned Vessel. However, the authority to make decisions is with the operator
Level 3: Delegated; The Unmanned Vessel is now authorised to execute some functions. The control
initiative emanates from the Unmanned Vessel and decision-making is shared between the operator and the
Unmanned Vessel
Level 4: Monitored; The Unmanned Vessel will sense environment and report its state. The operator may
monitor the events, and
Level 5: Autonomous; The Unmanned Vessel will sense environment and report its state. The operator may
monitor the events.</p>
        <p>At the very highest level, missions are specified as a series of waypoints, with functionality tags (such as
profile, station keep, dock) with the vehicle attempting to maintain a straight course between waypoints. Many
USVs have the capacity for real-time bidirectional communication between the control station and the USV,
where the USV can have the waypoints, steering, and communication commands sent to it in near-real time.</p>
        <p>During USV navigation, there are three main operations to carry out. These are: (a) Route Planning
(waypoints’ elicitation), (b) Monitoring of the Navigation, and (c) Updates to Route Planning, if and when necessary.
Once the USV starts moving, locomotion along the route has to be monitored, by the USV and/or the control
station, for various reasons, such as obstacle avoidance, asset detection, and changes in weather, thereby making
accurate and secure navigation of these autonomous vessels essential for safety.
2.1</p>
        <sec id="sec-3-1-1">
          <title>Navigating Autonomous Marine Vessels Safely and Securely</title>
          <p>USVs are required to be at least as safe as the equivalent human-operated surface vessels. Some of the safety
concerns for USVs that have navigation as core include: (a) their ability to avoid collisions with other marine
assets, such as floating objects (e.g. bouys, etc.) or other marine vessels, (b) their ability to navigate safely in
coastal areas, (c) ability to handle emergencies, such as failure recovery and repairs at sea of itself or of other
marine vessels.</p>
          <p>To be safe in its operation, a USV should endeavour not be a safety hazard to itself, other surrounding marine
assets, or the maritime environment, of which it is a part. USV navigation is usually provided by the Global
Navigation Satellite Systems (GNSS), of which the General Positioning System (GPS) is a part of. Depending on
the level of autonomy of the corresponding USV, successful navigation, and mission operations, require precise
positioning, timing and collision avoidance. All these depend on the accuracy of the GNSS values provided to
the USV.
2.2</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>Positioning And Navigating with Global Navigation Satellite Systems (GNSS)</title>
          <p>
            Global Navigation Satellite Systems (GNSS), such as the Global Positioning System (GPS), Russia’s GLONASS,
the European Union’s Galileo and China’s COMPASS, provide important positioning, navigation and timing
information to military, civilian and commercial users around the world. GNSS comprises mainly three components
            <xref ref-type="bibr" rid="ref7">(Ioannides et al., 2016)</xref>
            : (a) the User Segment, (b) the Control and Uplink Segment and (c) the Space Segment.
          </p>
          <p>The Space segment consists of a constellation of operating satellites that transmit one-way signals that give
the current GNSS satellite position and time. These signals are generated by the satellites’ payloads that also
contain one or more atomic clocks. These clocks are used to precisely time the signals and to provide good
frequency reference. The navigation signals are optimised for various applications but they share a similar
structure. For a given satellite, m, the transmitted signal, s(t), is modelled by: sm(t) = √2Pmcm(t)cos(2πfRF t),
where m denotes the satellite index, P is the transmit power, d(t) the broadcast navigation message, c(t) a
pseudo-randomly alternating chipping sequence, t denotes time, and fRF is the nominal carrier frequency.</p>
          <p>The control segment consists of a global network of ground facilities that track the satellites, monitor their
transmissions, perform analyses, and send commands and data to the constellation. As the locations of these
stations are precisely known and the orbital motion of the satellites follows Kepler’s laws, these data can be used
to determine and predict the satellite positions. The user segment consists of the GNSS receiver equipment,
which receives the signals from the GNSS satellites and uses the transmitted information to calculate the user’s
three-dimensional position, velocity and time. Figure 3 shows a schematic diagram of GPS showing the three
segments.</p>
        </sec>
        <sec id="sec-3-1-3">
          <title>GNSS Vulnerabilities</title>
          <p>
            GNSS signals are very weak, as low as −160dB W , and unencrypted. As such, the system is vulnerable to
unintentional and intentional interference. The result of such interference could be the complete failure of the
vessel’s GNSS receiver or, possibly worse, the presentation to the vehicle of hazardously misleading information for
navigation and situational awareness. In general, three attack types are distinguished
            <xref ref-type="bibr" rid="ref10">(Maarse, 2016)</xref>
            : spoofing,
jamming, and meaconing attacks. Meaconing is the interception and rebroadcasting of navigation signals in
order to confuse navigation, while jamming is the intentional interference of the GNSS signals via the emission of
radio frequency energy of sufficient power and with the proper characteristics to prevent receivers in the target
area from tracking the GNSS signals. Spoofing is the broadcast of false signals with the intent that the victim
receiver will misinterpret them as authentic signals. The victim might deduce a false position fix, a false clock
offset, or both.
          </p>
          <p>Although GNSS signal jamming has been popular in recent times, interest in GNSS spoofing has intensified
of late due to successful “spoofing in the wild” that have been reported. Examples include, the Iranian military
forcibly capturing a highly classified CIA drone in Dec. 2011. An Iranian engineer involved in the capture
claimed that they spoofed the drone into landing in Iran when it thought it was landing at its base in Afghanistan
(Rawnsley, ). A scientific satellite was reported to have received spoofing-like GPS interference over Ukraine
(Divis, 5 09). A yacht was spoofed deluding the receiver, causing the vessel’s autopilot system and crew to
navigate along a course laid out by the adversary (Bhatti and Humphreys, 7 05; ?). Spoofing could send marine
vehicles, off-course, especially in low-visibility conditions, threatening safety and security.
2.4</p>
        </sec>
        <sec id="sec-3-1-4">
          <title>Safe and Secure Navigation of Unmanned Marine Systems (UMS)</title>
          <p>The rising level of autonomy brings with it new hazards and risks that need to be handled and/or mitigated
in order to enjoy its economic benefits. Due to their importance, safety and security impose constraining
requirements that need to be fulfilled in the design and implementation of the navigation module of unmanned
marine systems, such as USVs. Safety analysis methods, such as HAZOP (Tyler et al., 2015), work on an existing
design and are ill-suited to assess the kinds of cyber-physical systems employed in UMS. Systems and system
designs have become so complex that waiting until a design is completed to perform safety and security analyses
on it is impractical. Even if by dint of sheer will it is possible to perform such analyses, changing the design after
the fact is usually impractical (financially and intellectually). Much of this effort, therefore, goes into proving
that existing designs are safe and/or secure rather than building designs that are safe from the beginning. The
only hope for practical and cost-effective safe design approaches in these systems is to design safety and security
in from the beginning.</p>
          <p>
            Due to the interactions between the software and the physical parts of cyber-physical systems, and the
ensuing emergence that are the results of these component interactions, research suggests that designing secure
safety-critical systems poses a substantial challenge
            <xref ref-type="bibr" rid="ref12">(Oates et al., 2013)</xref>
            with a view that engineering “complex
embedded and cyber-physical systems requires a holistic view on both product and process” (Schlinglof, 2016).
Security and safety need to be incorporated across the engineering life-cycle to ensure such systems are safe from
accidents and hazards, and secure from deliberate threats.
3
          </p>
          <p>
            Security and Hazard Analyses of the Navigation Component of Unmanned
Marine Systems (UMS)
Accidents have traditionally been conceived of as occurring from a sequence of directly related failure events,
each of which leads to the next event in the chain of events. Increased system complexity and interactions, and
the introduction of software, are leading to new types of accidents, accidents that are more a result of
intercomponent interactions (and not just intra-component failures). Traditional analysis methods work with accident
models that are based on the fault-error-failure chain
            <xref ref-type="bibr" rid="ref3">(Avizienis et al., 2004)</xref>
            . While these models are valid to
describe failures of single components, they are insufficient to describe system failures in complex interconnected
systems. Systems-Theoretic Accident Model and Processes (STAMP)
            <xref ref-type="bibr" rid="ref8">(Leveson, 2004)</xref>
            is an accident causality
model based on systems theory. It expands the traditional model of causality beyond a chain of directly-related
failure events or component failures to include more complex processes and unsafe interactions among system
components.
          </p>
          <p>STAMP is based on the three concepts of safety constraints, a hierarchical safety control structure and
process models. STAMP considers events leading to accidents occur because safety constraints were not successfully
enforced. Safety constraints on a system are imposed by the laws of physics, the regulatory and organisational
frameworks, the systems with which it interacts, and/or the functions it performs, and design and development
decisions. System-level constraints are first identified and responsibility for enforcing them is divided and
allocated. Then, during system design and development, system-level safety constraints are broken down and
sub-constraints are allocated to the system components.</p>
          <p>STAMP considers systems as hierarchical control structures where each level imposes constraints on the
activity of the level beneath it. The standard control structure involves four components of Controller, the
Controlled entity, Actuators and Sensors (figure 4).</p>
          <p>The controller issues control actions implemented by actuators that affect the state of the controlled
entity/process. Sensors capture changes in the state of the controlled process and transmit them to the controller
process which uses this feedback information to issue new actions to keep the controlled process in the desired
safe operational state. In STAMP, the controller has a model of the controlled process. The controller maintains
this model with the feedback information provided by the sensors and, based on this model, determines the
control actions.</p>
          <p>Accidents, in STAMP, are violations of safety constraints that were not adequately enforced by control actions
because the model of the controlled process in the controller departs from the actual behaviour of the controlled
process. This discrepancy is the source of the four possible causes of accidents: (a) A control action required for
safety was not provided; (b) An unsafe control action was provided; (c) A control action required for safety was
provided too early or too late or in the wrong sequence; and (d) A control action required for safety was stopped
too soon or applied too long.</p>
          <p>
            Based on STAMP, System Theoretic Process Analysis (STPA)
            <xref ref-type="bibr" rid="ref9">(Leveson and Thomas, 2018)</xref>
            and STPA-Sec
            <xref ref-type="bibr" rid="ref16">(Young and Leveson, 2013)</xref>
            were developed as new hazard analyses techniques to evaluate the safety and security
of a system. STPA starts from fundamental system engineering activities, including the identification of losses
or accidents to be avoided, the hazardous behaviour that could lead to these losses, safety requirements and
constraints, and the basic system control structure used to avoid these losses. STPA relies on four safety related
activities of system engineering, viz: (a) determination of unacceptable accidents. An accident, in STPA, is
defined as “an unplanned or undesired event that result in a loss of human, human injury, property damage,
environmental pollution, mission loss, etc.”; (b) determination of the system boundaries. Boundaries determine
which conditions related to accidents are considered part of the system and which are considered part of the
environment; (c) Identification of high-level system hazards. A hazard is a system state or a set of conditions
that, together with a particular set of worst-case environmental conditions, will lead to an accident; and (d)
the identification, determination and definition of system safety constraints. System safety constraints are
the conditions the system itself, its organisation and its development process must fulfil to prevent hazards
from occurring. STPA-Sec buttresses STPA in being used to analyse the security of systems. It changes the
traditional bottom-up approach to security, where threats are used to derive the security requirements, to a
top-down approach where the outcomes are more relevant.
          </p>
          <p>STPA has two steps: (1) the identification of potential inadequate control actions that can lead to hazardous
states; and (2) the determination of how these unsafe control actions can occur. Through these two steps,
STPA and STPA-Sec help to generate detailed safety and security requirements and constraints that must be
implemented in the design in order to prevent the identified unacceptable losses.
3.1</p>
        </sec>
        <sec id="sec-3-1-5">
          <title>Using STPA and STPA-Sec in the Security Analyses of UMS Navigation Module</title>
          <p>Two main steps can be identified in STPA Analysis: (1) Identification of the possible accidents in the system, of
system-level hazards leading to these accidents, and of the system constraints that can prevent and/or mitigate
these hazards; and (2) identification of the scenarios that could lead to these unsafe control actions. Table 1 shows
the results of our applying STPA Analysis on the UMS Navigation Module, helping to discern the accidents, the
system-level hazards, and the constraints of the Navigation Module (NM).
3.1.1</p>
        </sec>
        <sec id="sec-3-1-6">
          <title>Identification of accidents, system-level security hazards and security constraints</title>
          <p>In table 1, we see the accidents (AX) that may occur in the system, the hazards that may cause the accidents to
occur (HY), and the security constraints (safe control actions) (CZ) that can prevent or mitigate such hazards.
Section 3.1.2, below, expands on and identifies the unsafe actions that may be caused by these hazards.
A1
A2
A3
A4
A5
A6</p>
          <p>Unavailability of the Navigation Module (NM)
H1.1 NM receives continuous stream of un-usable GNSS signals via a jamming attack</p>
          <p>C1.1 NM shall assure assure the accurate and timely receipt of received GNSS signals
H1.2 Malevolent manipulation of hardware and software layers of NM to alter GNSS data
interpretation</p>
          <p>C1.2 NM shall guarantee the authenticity of its software and hardware components
Un-authorised disclosure of GNSS information
H2 There is un-authorised disclosure of GNSS information</p>
          <p>C2 NM shall assure GNSS data are disclosed only to authorised parties
Received GNSS information are not genuine
H3.1 GNSS information have been intentionally altered via a spoofing attack</p>
          <p>C3.1 NM shall assure assure the integrity of received GNSS signals
H3.2 NM receives replayed GNSS signals via a meaconing attack</p>
          <p>C3.2 NM shall assure the integrity of received GNSS signals
H3.3 GNSS information have been un-intentionally altered (probably due natural accident)</p>
          <p>C3.3 NM shall assure the integrity of received GNSS signals
NM physical antennae poorly installed
H4.1 Antennae installed position inhibits clear view of sky or clear signals from
satellites</p>
          <p>C4.1 NM operational staff shall assure correct installation of NM’s antennae
H4.2 Antennae improperly matched to NM receiver</p>
          <p>C4.2 NM operational staff shall assure accurate matching of antennae to receiver
Users’ Psychological Error
H5 Over-reliance of users on provided GNSS data</p>
          <p>C5 NM shall assure availability of other sources of accurate UMS positioning and timing data
Unintentional Radio Frequency (RF) interference
H6 Noise from nearby RF transmitters interferes with genuine GNSS signals</p>
          <p>
            C6 NM shall assure un-intentional RF interference
After the preliminary hazard analyses carried out in table 1, the next step is to use STAMP’s four general
categories of unsafe control actions
            <xref ref-type="bibr" rid="ref9">(Leveson and Thomas, 2018)</xref>
            of: (a) “Not providing causes hazard”, (b)
“Providing causes hazard”, (c) Wrong timing/ordering causes hazard”, and “Stopping too soon/applying too
long causes hazard”, to identify the conditions under which the hazardous controls, as enumerated in table 1,
could lead to system hazards.
          </p>
          <p>
            Tables 2 and 3 present our use of STAMP to analyse some controls and outputs issued by the NM. The first
column identifies the analysed control. The second column records the consequences of not providing a safe
control. The third column records the consequences of providing an unsafe control (i.e. the controller allows the
controlled process to perform actions in a context where hazards may occur). The fourth column records the
consequences of providing a safe control too early or too late or in a wrong order. The fifth column records the
consequences of stopping a safe control too soon or applying it too long. Every UCA must be traceable to one
or more system-level hazards
            <xref ref-type="bibr" rid="ref9">(Leveson and Thomas, 2018)</xref>
            .
          </p>
          <p>These identified unsafe control actions, with the related hazards, serve many functions. They can be used to
shape early design decisions regarding the security of the system to be built. When the conditions under which
a control action may be unsafe are stated, these help the engineers to perceive those instances, eliminate those
instances from the system design or find ways to mitigate them. When translated into requirements, they form
parts of the constraints to be enforced by the system design.
The vast majority of global trade flows across the world’s oceans. Unmanned marine systems (UMS) are
increasingly being used to facilitate and secure these trade flows. The security of the autonomous navigation of these
systems is becoming increasingly important. Therefore, accurate positioning, velocity, and timing (PVT) values
are essential to their safe navigation. These PVT values are usually provided by the Global Navigation Satellite
Systems (GNSS) signals that are received and decoded by the UMS’ receivers. These signals are very weak
and un-encrypted. Their accurate reception and decoding are, therefore, open to many vulnerabilities. This
paper has used systems, and control, theory and STPA to analyse these vulnerabilities. We identified the major
system-level security hazards and constraints, and especially of unsafe control actions. Our analyses can be used
as a springboard to drive their mitigation and/or resolution, thereby helping to designing a more effective and
secure UMS’ navigation components.</p>
          <p>In future work, we will extend these analyses with STPA causal scenarios, and using the Event-B (Abrial,</p>
        </sec>
        <sec id="sec-3-1-7">
          <title>Stopped Too</title>
        </sec>
        <sec id="sec-3-1-8">
          <title>Soon or Applied</title>
        </sec>
        <sec id="sec-3-1-9">
          <title>Too Long</title>
          <p>Same as UCA8
(Stopped Too Soon)</p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>Same as UCA9 (Stopped Too Soon)</title>
      </sec>
      <sec id="sec-3-3">
        <title>Same as UCA10</title>
        <p>(Stopped Too Soon)
2010) formalism, will develop a framework for security analysis for autonomous navigation of unmanned marine
systems.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Abrial</surname>
            ,
            <given-names>J.-R.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Modeling in Event-B: System</article-title>
          and
          <string-name>
            <given-names>Software</given-names>
            <surname>Engineering</surname>
          </string-name>
          . Cambridge University Press.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          Air Force Research Laboratory, Dayton,
          <string-name>
            <surname>O.</surname>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>U.s. air force, autonomy science and technology strategy</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Avizienis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Laprie</surname>
            ,
            <given-names>J.-C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Randell</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Landwehr</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Basic concepts and taxonomy of dependable and secure computing</article-title>
          .
          <source>In IEEE Transactions on Dependable and Secure Computing</source>
          , volume
          <volume>1</volume>
          , pages
          <fpage>11</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Bhatti</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Humphreys</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2017</year>
          -
          <fpage>05</fpage>
          ).
          <article-title>Hostile control of ships via false gps signals: Demonstration and detection</article-title>
          . Navigation.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Divis</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          (
          <year>2015</year>
          -
          <fpage>09</fpage>
          ).
          <article-title>Scientists document possible drone jamming</article-title>
          .
          <source>Inside Unmanned Systems.</source>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Humphreys</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>State of the art and future trends in radionavigation</article-title>
          . Briefing to USPTO.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Ioannides</surname>
            ,
            <given-names>R. T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pany</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Gibbons</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Known vulnerabilities of global navigation satellite systems, status, and potential mitigation techniques</article-title>
          .
          <source>In Proceedings of the IEEE</source>
          , volume
          <volume>104</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Leveson</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>A new accident model for engineering safer systems</article-title>
          .
          <source>Safety Science</source>
          ,
          <volume>42</volume>
          (
          <issue>4</issue>
          ):
          <fpage>237</fpage>
          -
          <lpage>270</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Leveson</surname>
            ,
            <given-names>N. G.</given-names>
          </string-name>
          and Thomas,
          <string-name>
            <surname>J. P.</surname>
          </string-name>
          (
          <year>2018</year>
          ). STPA Handbook.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Maarse</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>A systematic approach towards gnss receiver vulnerability analysis on remotely piloted aircraft systems</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Navy</surname>
            ,
            <given-names>U. S.</given-names>
          </string-name>
          <article-title>United states navy biography</article-title>
          . http://www.navy.mil/navydata/leadership/quotes.asp?q=253&amp;c=6.
          <fpage>2018</fpage>
          -
          <volume>06</volume>
          -28.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Oates</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thom</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Herries</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Security-aware, model-based systems engineering with sysml</article-title>
          .
          <source>In Proceedings of the 1st International Symposium on ICS &amp; SCADA Cyber Security Research</source>
          , pages
          <fpage>78</fpage>
          -
          <lpage>87</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Rawnsley</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <article-title>Iran's alleged drone hack: Tough, but possible</article-title>
          . http://www.wired.com/dangerroom/2011/12/irandrone-hack-gps.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Rolls-Royce</surname>
          </string-name>
          .
          <article-title>Rolls-royce written evidence (auv0083)</article-title>
          . http://data.parliament.uk/writtenevidence/committeeevidence.svc/evi and-technology
          <string-name>
            <surname>-</surname>
          </string-name>
          committee-lords/autonomous-vehicles/written/42075.html. M.
          <article-title>Being a responsible industry - an industry code of practice.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          https://www.maritimeuk.
          <source>org/documents/197/CODE OF PRACTICE V1</source>
          .
          <article-title>0 - Up to 24m - Final</article-title>
          .pdf.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Young</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Leveson</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Systems thinking for safety and security</article-title>
          .
          <source>In Proceedings of the 29th Annual Computer Security Applications Conference</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>