<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>An Approach for Logic-based Knowledge Representation and Automated Reasoning over Underspecification and Refinement in Safety-Critical Cyber-Physical Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Hendrik Kausch</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mathias Pfeiffer</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Deni Raco</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bernhard Rumpe</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>RWTH Aachen University</institution>
          ,
          <addr-line>Aachen</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-In this paper the extension of an intelligent compositional verification framework for cyber-physical systems is presented and the capabilities of accompanying underspecificationrefinement steps by verification are demonstrated on a representative example of a flight guidance system. Formal knowledge representation using higher-order logic and intelligent reasoning is shown to be applied to software engineering problems to perform correctness proofs, execute symbolic tests or find counterexamples. The theorem prover Isabelle is a mature and fundamental tool, which allows to represent knowledge as a collection of definitions and theorems and reason about systems. To increase the usability, an architecture description language (ADL) coupled with a code generator from the ADL to Isabelle is used. These and the rapid increase of computation capabilities suggest that a prominent application for reducing certification costs of critical systems such as intelligent flight control systems or assistance systems for air or road traffic management is not far in the future.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        The complexity of safety-critical systems is increasing in the
avionics and other fields and new technologies like unmanned
flying vehicles are rapidly introduced into the market. These
bring new challenges to certification processes. While trying
to maintain high system reliability, the scalability of traditional
quality assurance methods such as testing and reviewing is not
ideal and causes considerable amount of costs [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        If requirements were to be specified in a formal way, one
could reason about them and thereby replace or complement
many tests (please see also the conclusion for a more detailed
discussion). Abstract Interpretation [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] is a static analysis
technique which has been applied in avionics. It does acts
rather on code-level (in contrast to the approach of this paper,
which handles requirements on all abstraction levels.) While
having good automation, Abstract Interpretation is usually
targeted on very specific artifacts and usually requires some
manual expertise to discharge false positives. Model Checking
[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] also has been applied to small and middle sized systems.
But it does suffer from the well-known state-explosive problem
when trying to handle larger systems. While hardware is
better handled, the complexity of verifying software increases
exponentially with the size of the (state space of the) system
(while e.g. in theorem proving this growth is rather linear [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]).
      </p>
      <p>Meanwhile, artificial intelligence fields such as logic and
knowledge representation have also been applied successfully
in software engineering. When representing knowledge by
logic, theorem proving is then reduced to intelligent
reasoning. So by creating a knowledge base for safety-critical
systems, automatic reasoning becomes possible by reducing
verification into applying AI techniques such as metaheuristic
(proof-) search techniques. Same holds for error detection;
a counterexample-finder takes as input a system model, (the
negation of) a property, and error detection is reduced into a
search problem (and in case of an error the trace-path of this
search is returned as a malicious input). In a time of increasing
computational power, these techniques open up possibilities to
maintain manageable certification costs.</p>
      <p>
        Common ADLs used for model-based development avionics
are AADL [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], SysML [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], Simulink [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] etc. SCADE [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]
(based on the dataflow language Lustre [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]) is used by Airbus
[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] coupled with a model checker. In comparison, our ADL
presented in this paper is extended to be able to handle
not only SCADE-like time-sensitive deterministic systems
(such as our previous automotive case study in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]), but
also (higher-level predicative-oriented and lower-level
stateoriented) underspecification and refinement.
      </p>
      <p>
        For the incorporation of analysis tools in their life-cycle
processes, companies such as Airbus and Dassault Avionics
have identified a number of requirements [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]:
      </p>
    </sec>
    <sec id="sec-2">
      <title>Soundness</title>
      <p>Ability to be integrated into the certification standards
conforming process
Cost Savings
Deliver correctness by construction
Scalability: Compositional Verification
Expressivity of Specification Language
Timing aspects and underspecification refinement
Usability by normal software engineers on normal
machines</p>
      <p>
        The Chair for Software Engineering at the RWTH Aachen
University has been conducting research for many years
using its language workbench MontiCore [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] for developing
domain-specific languages (DSLs) and also performing model
transformations. A model transformation could be for example
a refinement [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] of an underspecified model into a more
specified one in the same modeling language, or a code
generator mapping a model from a domain-specific front end
language into an equivalent model in an analysis tool (e.g. a
model checker or a theorem prover).
      </p>
      <p>
        In principle, for any DSL one can create a knowledge base
and reason about it in a logic language. In this paper this is
demonstrated on the example of an architecture description
language (ADL) for cyber-physical systems (called MontiArc
[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]) coupled with a code generator from the ADL into
the knowledge base (here theorems in Isabelle). The added
value demonstrated in this paper is accompanying step-wise
refinements during the design-lifecycle of safety-critical systems
by verification to reduce certification costs. The knowledge
base consists in the encoding of generic dataflow-oriented data
structures, functions and theorems for reasoning in the theorem
prover Isabelle [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. The semantical [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] underpinning of
MontiArc is FOCUS [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ], a dataflow-based methodology for
the stepwise development and refinement of systems. If the
behavior of a component is described by a MontiArc automata
with input/output, it is implementable (also called realizable)
per construction. For further discussions about foundations
of our methodology, please read also [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. The semantics of
(non-deterministic) components are (sets of) stream processing
functions [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. The unique selling point of FOCUS compared
with other concurrents such as -calculus [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], CSP [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]
or Petri nets [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] is that refinement is fully compositional.
Thus, one can decompose a system, refine each component
separately e.g. until an implementation, and then be sure that
after composing back the new system will be a refinement
of the old one, thus sparing associated testing and integration
costs. It also allows leveraging proof reuse (as will be shown
in the running example section II-G or section II-I).
      </p>
      <p>The contribution of this work is the extension of
logicbased a knowledge representation for an automatic reasoning
approach aiming at reducing certification costs of
safetycritical systems. This is demonstrated through accompanying
the refinement of different abstraction levels of
underspecification (see fig. 1) by verification. The full compositionality of
refinement of FOCUS is leveraged for the step-wise refinement
of a representative example of a pilot flying system (in this
cyber-physical system the bus component is hardware and the
flight guidance component is software).</p>
      <p>On a more detailed level, this includes:</p>
      <p>Extending an ADL and its knowledge base for enabling
higher-level history-oriented requirement specification.
Extending an ADL and its knowledge base for enabling
refinement of history-oriented requirement specifications
into lower-level (closer to implementation, yet still not
necessarily deterministic) state-based requirement
specifications.</p>
      <p>High-Level
B Requirements‘
-Compliance
-Traceability</p>
      <p>D
-Verifiability
-Accuracy and
Consistency
-Verifiability
-Accuracy and
Consistency</p>
      <p>C</p>
      <p>System</p>
      <p>A Requirements
-Verifiability
-Accuracy and
consistency
-Verifiability
-Accuracy and
consistency
-Compliance
-Traceability</p>
      <p>H
I -Compliance</p>
      <p>-Traceability</p>
      <p>High-Level</p>
      <p>Requirements
-Compliance
-Verifiability
-Accuracy and
consistency J</p>
      <p>K --TCroamcepalibainlictye
Source
Code</p>
      <p>F</p>
      <p>E
-Verifiability
-Accuracy and
consistency</p>
      <p>G
Software
Architecture</p>
      <p>Low-Level
Requirements</p>
      <p>Software
Architecture</p>
      <p>Low-Level
Requirements‘
Fig. 1. Certification Activities (Letters = Sections in Chapter 2). The denoted
activities are shown to be handled by our methodology and thus save a lot of
test and review costs.</p>
      <p>Extending an ADL and its knowledge base for
enabling refinement of lower-level non-deterministic
stateoriented requirement specifications into another
stateoriented specification by means of transition-refinement
and state refinement.</p>
      <p>Extending counterexample-finding capability for design
error detection.</p>
      <p>
        Optimizing signatures of key structures in the knowledge
base to increase the automation degree and reduce the
needed user-expertise (consisted in rewriting the encoding
with total functions instead of partial functions (such as in
[
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]), thus making automatic proof finding much easier)
Integration of an abstraction mechanism called locals,
which improved the efficiency of the code generator
about 10 times and enables thus an easier-qualifiable code
generator from the ALD to the theorem prover.
      </p>
      <p>Discussing at what extend such a knowledge
representation approach can scale to meet industrial requirements,
documented in a table in the conclusion after the
evaluation and consideration of the lessons learned.</p>
      <p>Providing an Integrated Verification Environment created
with Visual Studio Code for the integration of all the
artifacts of the framework (see Appendix).</p>
      <p>
        The rest of the papers consists in the following: The next
chapter describes a typical avionics software development
activity, where step-wise refinements of underspecification
accompanied by verification is presented on the example
of a flight guidance system (adapted from a collaboration
between NASA and Rockwell Collins [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], we handle the
more involved asynchronous case).
      </p>
    </sec>
    <sec id="sec-3">
      <title>II. RUNNING EXAMPLE</title>
      <sec id="sec-3-1">
        <title>General Approach</title>
        <p>
          In this section we present the development of a pilot
flying system (PFS) with two flight guidance systems (FGSs)
in a refinable, decomposable fashion, adapted from [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ].
While in [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] different levels of abstractions are handled
with different techniques (theorem proving, model checking,
abstract interpretation), we demonstrate to be able to handle all
abstraction levels by our framework. The red-colored letters
(fig. 1 extracted from certification activities of DO-178C)
correspond to the sections of the next chapter. During the
formalization of informal requirements, errors in a first version
of the higher-level requirements are detected (red-letters B, C),
and then repaired (letters I, H). The system to be developed
is depicted in fig. 3. The tool chain is described in fig. 2.
The components of the system transmit potentially infinite
sequences of messages called streams over its channels [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ].
Time-synchronous streams are used to model time-sensitive
behavior. As described previously in [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ], this can be realized
through extending the message alphabet by s to denote that
no data is transmitted in a time slot. Timing granularity can
be set at wish depending on the system to be modeled (e.g.
in the case of the flight guidance system below, one time unit
represents 1 millisecond).
        </p>
        <p>
          The semantics of components are sets of stream processing
functions mapping input channel histories to output channel
histories and are hierarchically decomposable [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ].
        </p>
        <p>One-directional channels connect components by
transmitting input and output messages. In the PFS, messages sent
between the FGSs are transmitted by hardware buses. Each
FGS outputs a pair of booleans where the first boolean denotes
the liveness of the system and the second boolean is used as
an acknowledgement bit. Clocks restrict the other components
behavior by sometimes forcing them to output the last message
and not reacting to its input in any way. The system is
visualized in fig. 3.</p>
        <p>For each component (hardware and software), HLRs and
LLRs were created. When verifying compliance (say between
HLR and SysReqs), the HLRs of all components in the
software architecture are proven to fulfill all SysReqs. We
demonstrate to handle in the following a few
representative certification activities. In our knowledge representation
approach, certification credits concerning ”‘verifiability”’ of
requirements (see fig. 1) are generally claimed by having used
a formal language for the specification, and ”‘accuracy”’ is
claimed by having formal proofs.</p>
        <sec id="sec-3-1-1">
          <title>A. System Requirements (SysReqs)</title>
          <p>
            The system requirements (SysReqs) are largely adapted
from [
            <xref ref-type="bibr" rid="ref23">23</xref>
            ]. By formulating them as Object Constraint Language
(OCL) (see [
            <xref ref-type="bibr" rid="ref11">11</xref>
            ], [
            <xref ref-type="bibr" rid="ref24">24</xref>
            ]) constraints, these can be inputted to the
tool chain together with the MontiArc system model. It is then
checked whether the system model fulfills the requirements.
The first 5 SysReqs are similar to the 5 SysReqs from [
            <xref ref-type="bibr" rid="ref23">23</xref>
            ] (see
also Appendix). We give the first SysReq in natural language.
          </p>
          <p>SysReq1: At least one side is the active pilot flying side.</p>
          <p>Below is the first SysReq also as an OCL expression. It is
formulated in the context of the general component structure
from fig. 3 and addresses the streams by their port name:
f s t c 1 [ n ] o r f s t c 3 [ n ]</p>
          <p>The OCL expression consists of elements known from
first-order logic. Ports are named after the connected channels
from fig. 3. A syntactic extension allows to obtain the nth
element (point in time) of the stream flowing in the channel
c1 as c1[n]. The translation as Isabelle theorem looks alike
(variables are per default universally quantified). The first
item of a tuple is returned by f st.</p>
          <p>Furthermore, we define a safety-critical sixth SysReq
informally.</p>
          <p>SysReq6: Pressing and holding the transfer switch c5 for
10 milliseconds always switches the inactive flying side, if
the inactive component received the last transfer switch input
and the system is in a stable state (see also section C for the
architecturial contex).</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>B. High-Level Requirements’ (HLR’)</title>
          <p>The engineer now tries to develop high-level requirements
for each sub-component, so that the overall system
requirements can be fulfilled. First, we introduce the high-level
requirements of the clocks (corresponding in fig. 1 to HLR’):
component C l o c k U n f a i r S p e c f
t i m i n g s y n c ;
s p e c f
/ / o u t p u t s an i n f . l o n g b o o l s t r e a m
c l k 1 . l e n g t h ( ) = INF
g
g</p>
          <p>The bits on the clocks output stream determine whether
its associated component is active (executing) or not. The
ADL MontiArc is used to define the high-level requirements
(HLRs)of the clock component. This is a component with
one output and no input channel. First, the timing of the
clock is defined. The keyword sync indicates weak-causal
and causalsync strong-causal time-synchronous components
(causality captures realizability in time sensitive modeling,</p>
          <p>Clock 1
clk1
c5</p>
          <p>L-FGS
c4
c1</p>
          <p>Clock 4</p>
          <p>clk4
RL-Bus
LR-Bus</p>
          <p>clk2
Clock 2
c3
R-FGS
c2 clk3</p>
          <p>
            Clock 3
Fig. 3. An architecture of the pilot flying system (PFS), sufficient for
correctly synchronizing the pilot flying sides and the other mentioned system
requirements. clk1 etc. denote the channels that have clocks as sources, c1
etc. refer to the other communication channels, L-FGS denotes the left flight
guidance system, RL-Bus denotes the bus from right to left.
some technical details concerning the knowledge base see
[
            <xref ref-type="bibr" rid="ref25">25</xref>
            ]). The following composition correctly defines the system
model because the composition operator is commutative and
associative (thus making the composition order irrelevant)
[
            <xref ref-type="bibr" rid="ref26">26</xref>
            ].
          </p>
          <p>F GSL</p>
          <p>F GSR</p>
          <p>BusLR</p>
          <p>BusRL
clk1
clk2
clk3</p>
          <p>clk4</p>
          <p>
            After defining the Pilot Flying System as a composition
of its sub-components, we define the low-level requirements
of the sub-components. The unfair clock has to output
nondeterministicially true or false (input and transition guards can
be modeled in MontiArc [
            <xref ref-type="bibr" rid="ref15">15</xref>
            ], but are not present in this
component). A graphical representation of the automata is
given in fig. 4 and the textual version is:
component C l o c k U n f a i r A u t o m a t a f
t i m i n g s y n c ;
[
            <xref ref-type="bibr" rid="ref11">11</xref>
            ]). Weak-causality enforces that the production of an output automaton f
at a point in time t does not depend from an input arriving after / / one s t a t e
the point in time t. Strong-causal components have to define s t a t e S i n g l e ;
an initial output and delay interaction by 1 time unit. They i n i t i a l S i n g l e ;
are needed for well-defined semantics in feedback loops. The / / o u t p u t s t r u e o r f a l s e i n e v e r y s t e p
behavior is specified as a high-level predicate (spec stands S i n g l e &gt; S i n g l e / f c l k 1 = t r u e g ;
for specification). Inside the spec section, requirements to S i n g l e &gt; S i n g l e / f c l k 1 = f a l s e g ;
the boolean output stream clk can be formulated as OCL g
expressions. The length function obtains the length of a g
stream and IN F represents infinity. Timing and interface of the component are defined similar
          </p>
          <p>The HLRs for the Buses and FGSs are defined similarly. to section II-B. The automata then specifies the behavior in
This paper lists only the LLR’ as supplementary material in a lower-level, yet still non-deterministic fashion. States are
fig. 7. defined in the first line of the automata body. The second
line sets the initial state of the automata. Following the state
C. Software Architecture and Lower-Level Requirements’ specifications, the automata transitions and the output and
(LLR’) state-change behavior are defined on a step-by-step basis.</p>
          <p>In fig. 3 each component is represented by a named box.</p>
          <p>The channels describe the internal interaction between the
components. The incoming and outgoind channels are the
global in- and outputs of the PFS. Small boxes represent the
ports of a component. Our scalable vector graphics (SVG)
generator draws figures based on MontiArc models. Such
visualization is a powerful tool to gain overview and improve
the communication, especially in the early stages of the
development process.</p>
          <p>Here, each of the 4 clocks is connected to one of the
nonclock components to model their non-deterministic behavior.</p>
          <p>On a ”‘False”’ clock output signal, each non-clock component
does not react to its input and simply repeats the last output.</p>
          <p>Input channel c5 transmits the transfer switch status, c1 is the
output of the L-FGS system and c4 its input from the RL-Bus
etc.</p>
          <p>
            The complete system can be decomposed into
subcomponents hierarchically [
            <xref ref-type="bibr" rid="ref15">15</xref>
            ]. After the translation into
Isabelle code, the composition operator connects the
inand output channels of two components by its names (for
          </p>
        </sec>
        <sec id="sec-3-1-3">
          <title>D. Compliance of LLR’ to HLR’ and HLR’ Consistency</title>
          <p>The consistency of a specification is defined as the existence
of at least one implementation that satisfies the specification.</p>
          <p>The consistency of our HLR’ can be shown as follows: First
prove the compliance of LLR’ to HLR’ (i.e., LLR’ refines
HLR’). Then show that an implementation of LLR’ exists.</p>
          <p>For the later, consider that LLR’ (e.g. of the clocks) are
nondeterminsitic (total) automata, and can easily be converted
to an implementation (deterministic automata) by removing
transitions responsible for non-determinism. Next, we focus
on compliance of LLR’ to HLR’.</p>
          <p>
            For this, the LLR’ MontiArc automata is translated to an
Isabelle automata and further transformed to (sets of) stream
processing functions. The HLR’ spec is directly transformed
to (a set of) stream processing functions. The transformation
from automata to stream processing function (SPF) is defined
as a (greatest) fixed point calculation of the corresponding
functional [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ]. Now, a theorem can be formulated in OCL by
the user, stating: LLR’ refines HLR’. This is translated into
/True
/False
reason over not-necessarily-executable specifications [
            <xref ref-type="bibr" rid="ref29">29</xref>
            ]. A
counterexample for the 6th SysReq would be e.g. the following
streams on channels c5 and clk3:
          </p>
          <p>streamc5 = F alse1
streamclk3 = T rue1</p>
          <p>T rue11
F alse11</p>
          <p>
            F alse1
T rue1
Single The infix is used for stream concatenation, whereas i-fold
repetition is formalized as i. The stream T rue1 for example
is an infinitely long stream of messages T rue. Obviously, the
assignment does not contradict the HLR’ of the clock (random
trues and f alses). But the system does not change the flying
Fig. 4. The unfair clock outputs non-deterministically booleans side after pressing the button for 10 milliseconds (SysReq 6),
as the clock blocks any reaction for 11 time-units. Thus, a
the following Isabelle theorem, where the Isabelle-semantics new and sufficient HLR has to be formulated.
of a MontiArc construct is abbreviated by J:::K): chaTihnehiansfrbaseternucetuxrteenfdoerdfibnydinidgenctoiufynitnegrexaanmdpilnesseritninogur(dtoonoel
theorem as described e.g. in [
            <xref ref-type="bibr" rid="ref28">28</xref>
            ]) fitting lemmas into the transformed
”J C l o c k U n f a i r A u t o m a t a K J C l o c k U n f a i r S p e c K” Isabelle encoding. These lemmas are essential not only to
So refinement is reduced to a set-inclusion of SPFs. counterexample-finders, but also facilitate automatic proofs
This theorem is proven by showing that every SPF from and general transparency of the encoding.
JClockUnfairAutomataK outputs a bool stream that is infinitely F. Refinement of HLR’ to HLR
long, thus satisfying the OCL requirement in the spec-behavior
of ClockUnfairSpec. All other components consistencies are As one may notice, one does not have to come up with an
proven similarly and follow the overall tactic to show the entirely new HLR. With only a slight refinement of HLR’,
fulfillment of all specification properties. Since the other the 6th SysReq can be fulfilled. The new clock specification
components are deterministic, their semantics contains exactly (HLR) is obtained by adding the following OCL-predicate into
one SPF. In order to increase automation of reasoning over the spec-body of the ClockUnfairSpec:
(sets of) stream processing functions, abstract (case-study- f o r a l l n i n n a t .
independent) theorems have been proven in the knowledge e x i s t s m i n n a t .
base [
            <xref ref-type="bibr" rid="ref25">25</xref>
            ]. n &lt; m &amp;&amp; m &lt; n +10 &amp;&amp; c l k [m ] ;
          </p>
          <p>
            It might be interesting to note that compared to [
            <xref ref-type="bibr" rid="ref11">11</xref>
            ], the
largest part of the generator has been moved inside Isabelle. The new HLR restricts the clocks to output at least one
This keeps the inter-language transformation from architecture true every 10 milliseconds. The new specification is called
description language (ADL) to Isabelle minimal (about 10 f air clock and also translated to Isabelle. After generating
times less compared to [
            <xref ref-type="bibr" rid="ref11">11</xref>
            ]). This was achieved by integrating the corresponding set of stream processing functions for HLR,
the concept of locales [
            <xref ref-type="bibr" rid="ref27">27</xref>
            ]. The whole tool chain becomes thus ”HLR refines HLR’” is reduced to a set-inclusion proof.
easier qualifiable due to the axiomatic and conservative nature
of Isabelle (please note that the translation from a DSL into
a logical language needs in general to be qualified, and is
usually performed by testing it for a representative collections
of models).
          </p>
        </sec>
        <sec id="sec-3-1-4">
          <title>E. HLR’ Compliance with SysReq</title>
          <p>
            The next important step is to check the fulfillment of the
system requirements. HLR’ fulfills the first 5 SysReqs, but
not the 6th. The proofs that HLRs comply with the 5 SysReqs
is generally history-oriented (reasoning over infinite streams).
Meanwhile, tools like quickcheck and nitpick were integrated
in our tool chain and can be used to automatically search
for counterexamples. Quickcheck originates from Haskell and
is a test-based tool that needs an executable implementation
from which it generates Haskell code and is fast [
            <xref ref-type="bibr" rid="ref28">28</xref>
            ]. Nitpick,
on the other hand, is a SAT-solving based tool that is able
to find even infinitely long stream-counterexamples, and also
theorem
”J C l o c k F a i r S p e c K J C l o c k U n f a i r S p e c K”
The signatures of the involved functions are identical, and
the predicate of HLR is a slight sharpening of the predicate
of HLR’ (by the added OCL-requirement in section II-F),
and thus the refinement proof is easily found automatically.
Since all SPFs from ClockFairSpec output an infinite stream
of booleans, the only requirement of the ClockUnfairSpec is
already met.
          </p>
        </sec>
        <sec id="sec-3-1-5">
          <title>G. Refinement of LLR’ to LLR</title>
          <p>Similar to above, one does not need to come up with an
entirely new LLR (for the clocks) out of their HLR, but
can instead refine the almost-successful LLR’ just enough to
comply with the HLR. A new LLR for the fair clock is defined
as a non-deterministic automata (see fig. 5) using a finite timer
to force at least one true every 10 milliseconds:
proving the remaining 6th SysReq. This reduces certification
costs of HLR compliance.</p>
          <p>Compliance: The compliance with the last SysReq follows
straightforwardly (proof is thus easily found automatically)
from the added OCL-predicate in the spec-body of HLR in
section II-F).</p>
          <p>Traceability: Certification credits for the traceability of
HLR is claimed by deleting HLR-predicates (OCL-expressions
from the spec body) one-by-one and proving that each deletion
in isolation already leads to an incompliant system requirement
(one of these cases led to the above mentioned HLR’).</p>
        </sec>
        <sec id="sec-3-1-6">
          <title>J. Source Code</title>
          <p>
            The non-deterministic clocks can be refined to a
deterministic behavior by deleting transitions that offer non-deterministic
choices (e.g. by deleting one of the transitions such as
those occurring in ClockUnfairAutomata). Apart from
the non-deterministic clocks, any LLR of other components
is a deterministic MontiArc model and can be interpreted
as an implementation, since its semantic is a single stream
processing function [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ].
          </p>
        </sec>
        <sec id="sec-3-1-7">
          <title>K. Compliance of Source Code to LLR</title>
          <p>One can save certification costs (of proving compliance
of LLR with HLR and SysReq all over again) by proving
that LLR is a refinement of LLR’, and then reasoning that
the already proven properties (LLR’ refines HLR’ and HLR’
fulfills almost all SysReq) will also hold for LLR. This means
one only needs to prove the missing 6th SysReq.</p>
          <p>The refinement of LLR’ into LLR is a refinement between
non-deterministic automata. Similar to the refinement between
HLR and HLR’, a theorem in Isabelle can be formulated over
the automata’s semantics:</p>
          <p>After translating the deterministic components (representing
the source code) to Isabelle, the compliance of source code
with LLR can be proven. The proof is reduced to showing that
the resulting stream processing function (i.e. the semantics of
the deterministic automata) is an element of the LLR semantics
theorem (a set of SPFs). Accuracy and consistency are correct per
”J C l o c k F a i r A u t o m a t a K J C l o c k U n f a i r A u t o m a t a K” construction in the case of a deterministic component. The</p>
          <p>
            This is proven by two simple refinement steps. The first one compliance of the source code to the software architecture
is a semantics-preserving state-refinement (through splitting and to the system requirements holds without further
efthe one state of LLR’ into 10 by introducing a counter fort by the unique property of FOCUS, that refinement is
variable. The set of behaviors remains after this step the fully compositional. Since the other (non-clock) components
same (does not become strictly smaller yet) [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ] (section already were deterministic, their compliance with their LLRs
5.3.5 and 6.4.3). The state refinement is usally a prelude for is proven directly.
subsequent underspecification-reducing refinement steps, such III. CONCLUSION
as the following. The second and final step is a
transitionrefinement [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ] (section 6.3.1) by which transitions
(responsible for non-determinism), which lead to ”unfair” behavior
(infinite consecutive inactivity clock-signals) are removed.
          </p>
        </sec>
        <sec id="sec-3-1-8">
          <title>H. Compliance of LLR to HLR and HLR Consistency</title>
          <p>As LLR’ refines HLR’ (section II-D) and LLR refines LLR’
(section II-G), it follows that LLR refines HLR’. Now all that
is left to prove is that LLR satisfies the additional constraints
of HLR (compared to HLR’). This reduces certification costs
for the HLR consistency proof significantly. The property that
the state with counter = 0 is reached after maximal 9 steps
(visible in fig. 5) is sufficient to prove the additional HLR
property easily automatically.</p>
        </sec>
        <sec id="sec-3-1-9">
          <title>I. HLR compliance with SysReq and Traceability of HLR</title>
          <p>As HLR’ is already proven to satisfy all but one SysReq and
HLR is a further refinement of HLR’, we can directly conclude
that HLR also satisfies the first 5 SysReq. We thus focus on
In conclusion, the methodology in this paper demonstrated
handling representative scenarios for a verified development
and compositional refinement of safety-critical systems. A
developer can specify either directly using a logic language
(e.g. Isabelle), or using an ADL for distributed systems as
frontend to describe interfaces of the components, their
behavior and their interaction in a comfortable way. Then the
system model and all desired properties can be then translated
into an equivalent specification in a knowledge base created
in a logic language.</p>
          <p>The developed knowledge base for MontiArc is very general
and can be largely reused for creating knowledge bases for
other modeling languages (such as AADL, SysML, SCADE,
Simulink etc.) Keeping FOCUS as semantical underpinning
has the already mentioned advantage that refinement is fully
compositional.</p>
          <p>This kind of approach can replace a lot of tests and reviews.
This helps also with requirements involving always/never,
which cannot be exhaustively verified in general by tests.
Please note though that a certain group of tests and reviews can
only be complemented hereby, but not completely replaced, for
instance checking whether:
requirements formalization is correct (in general
compliance between informal and formal models),
the methodology is justified and appropriate,
requirements and software architecture are compatible
with the target computer (unless the target environment
is formally modeled),
a requirement has not been forgotten,
there is no unidentified dead or deactivated code.</p>
          <p>
            As can be seen in fig. 1, a lot of development and
certification costs can be saved by omitting coming up with Source
Code’ out of LLR’, since it would not be correct anyway,
because HLR’ are not good enough. One could have even
spared the effort of creating LLR’ by the following reasoning:
HLR’ has to fulfill SysReqs and to be consistent. In our case
we went first for consistency. Instead, by leveraging the
higherlevel history-oriented spec-infrastructure of our methodology,
before one takes care of consistency, one could have shown
first that high-level spec’s and the architecture violate SysReqs
(and these kind of proofs are generally history-oriented, thus
much easier than induction-based proofs of inductively,
stateoriented LLR automata proofs [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ]). Then one does not need
to prove HLR consistency (which involves coming up with
LLR automata and showing their compliance to HLR), since
they will be not good enough.
          </p>
          <p>Concerning lessons learned, an interesting observation is
that there are a couple of reasons for preferring the
specification of components by means of an ADL ideally in a
statebased fashion (coupled with a generator), rather then giving
the user just an encoding of the stream data type and set of
theorems over streams in Isabelle:</p>
          <p>
            The ADL is for a user more comfortable than writing
recursive (specified typically as least fixed points [
            <xref ref-type="bibr" rid="ref30">30</xref>
            ])
stream processing functions in Isabelle.
          </p>
          <p>The input/output automata of this work are designed
to describe realizable components per construction (and
only these).</p>
          <p>
            The automata of our methodology are general enough
to represent every realizable stream processing function
(proof in [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ]).
          </p>
          <p>Finally, the table in fig. 6 summarizes how the methodology
presented in this paper handles the industrial requirements
described in the introduction. The biggest challenge
encountered is having a proof being found automatically in a large
model. The mitigation of this is an ongoing work consisting
in increasing the number of general theorems over
dataflowbased systems (which are identified intellectually during
verifying case studies), and also exploiting the rapid increase of
computational capability by using a central web-based
highperformance service to perform the proof search.</p>
          <p>However the application of reasoning over knowledge bases
is still not mature enough and further research and time will
Soundness
Ability to be integrated into
the DO-178x conforming
process
Cost savings</p>
          <p>Established usually by peer reviews, from 1980-now there
are over 200 papers about FOCUS.</p>
          <p>The methodology replaces or complements many tests and
supports fulfilling certification requirements. By relocating
the majority of the code generator internally in Isabelle (we
aim over &gt;99%), the tool chain becomes easier qualifiable
due to Isabelle’s axiomatic and conservative nature.</p>
          <p>Replaces traditional verification methods throughout the
development life cycle, as seen in the running example (also
see the point below)
Automata Language restricts the user into specifying only
well-behaved implementable functions (vs expect user to
write realizability proofs)
Correctness by construction Underlying methodology FOCUS: Refinement fully
compositional – After decomposing a system, refining the
components separately, and composing back, the new
system is a refinement of the old one (no new (unwanted)
behaviors are added, thus sparing test and integration costs)
Scalability: Compositional
verification
Expressivity of specification
language
Timing aspects and
underspecification
refinement</p>
          <p>
            Verification: Compatibility of composition with refinement
allows modularizing and breaking down the proof complexity
of representative industrial-sized models (as in the running
example)
Automata language can represent all implementable Stream
Processing Functions (semantical mapping is surjective,
proof in [
            <xref ref-type="bibr" rid="ref9">9</xref>
            ])
Supported as presented in the running example
          </p>
          <p>A user-friendly frontend language and a high-level API in the
Usability by normal software knowledge base helps (for hiding low-level engine concepts)
engineers on normal
machines</p>
          <p>A large amount of encoded lemmas helps aiming for high
automation (which implies less user expertise needed) - see
demo in: https://www.youtube.com/watch?v=krl4Q7MAAlo
be needed to make it a standard in the tool box of software
engineering development processes.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>APPENDIX</title>
      <p>FGS LLR
BUS LLR</p>
      <sec id="sec-4-1">
        <title>SysReq 2..5</title>
        <p>The System is in a stable situation whenever both FGSs
acknowledgements are true. If the system is in a stable
state, then at most one side is the active pilot flying side.
Pressing the transfer switch changes the inactive side,
if the system is in a stable state and the inactive side
receives the input.</p>
        <p>In the beginning one side has to be the active pilot flying
side. Without loss of generality we choose the left side
to be active. The right side has to be inactive.
Only switch to inactive side, if transfer switch is pressed.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>A.</given-names>
            <surname>Brahmi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Delmas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. H.</given-names>
            <surname>Essoussi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Randimbivololona</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Atki</surname>
          </string-name>
          , and T. Marie, “
          <article-title>Formalise to automate: deployment of a safe and costefficient process for avionics software</article-title>
          ,
          <source>” in 9th European Congress on Embedded Real Time Software and Systems (ERTS</source>
          <year>2018</year>
          ), (Toulouse, France), Jan.
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>D.</given-names>
            <surname>Delmas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Goubault</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Putot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Souyris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Tekkal</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Ve</surname>
          </string-name>
          <article-title>´drine, “Towards an industrial use of fluctuat on safety-critical avionics software</article-title>
          ,” in
          <source>International Workshop on Formal Methods for Industrial Critical Systems</source>
          , pp.
          <fpage>53</fpage>
          -
          <lpage>69</lpage>
          , Springer,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>E.</given-names>
            <surname>Payet</surname>
          </string-name>
          and
          <string-name>
            <given-names>F.</given-names>
            <surname>Spoto</surname>
          </string-name>
          ,
          <article-title>Checking Array Bounds by Abstract Interpretation</article-title>
          and Symbolic Expressions, pp.
          <fpage>706</fpage>
          -
          <lpage>722</lpage>
          . 06
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>T.</given-names>
            <surname>Bochot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Virelizier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Waeselynck</surname>
          </string-name>
          , and
          <string-name>
            <given-names>V.</given-names>
            <surname>Wiels</surname>
          </string-name>
          , “
          <article-title>Model checking flight control systems: the airbus experience</article-title>
          ,” pp.
          <fpage>18</fpage>
          -
          <lpage>27</lpage>
          ,
          <year>06 2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S. P.</given-names>
            <surname>Miller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. C.</given-names>
            <surname>Tribble</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. W.</given-names>
            <surname>Whalen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M. P. E.</given-names>
            <surname>Heimdahl</surname>
          </string-name>
          , “
          <article-title>Proving the shalls: Early validation of requirements through formal methods,”</article-title>
          <string-name>
            <given-names>Int. J. Softw. Tools</given-names>
            <surname>Technol</surname>
          </string-name>
          . Transf., vol.
          <volume>8</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>303</fpage>
          -
          <lpage>319</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>X.</given-names>
            <surname>Diao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Liu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Wang</surname>
          </string-name>
          , “
          <article-title>A survey of avionics analysis and simulation based on aadl model,”</article-title>
          <string-name>
            <given-names>Int. J.</given-names>
            <surname>Inf</surname>
          </string-name>
          . Commun. Techol., vol.
          <volume>9</volume>
          , pp.
          <fpage>282</fpage>
          -
          <lpage>299</lpage>
          , Jan.
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>F.</given-names>
            <surname>Mhenni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.-Y.</given-names>
            <surname>Choley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Nguyen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Frazza</surname>
          </string-name>
          , “
          <article-title>Flight control system modeling with sysml to support validation, qualification and certification,” IFAC-PapersOnLine</article-title>
          , vol.
          <volume>49</volume>
          , pp.
          <fpage>453</fpage>
          -
          <lpage>458</lpage>
          ,
          <year>12 2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>B.</given-names>
            <surname>Albert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Usach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Vila</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Crespo</surname>
          </string-name>
          , “
          <article-title>Development of integrated modular avionics applications based on simulink and xtratum</article-title>
          ,”
          <volume>05</volume>
          2013.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>G.</given-names>
            <surname>Berry</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bouali</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Fornari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Ledinot</surname>
          </string-name>
          , E. Nassor, and R. de Simone, “
          <article-title>Esterel: a formal method applied to avionic software development,” Science of Computer Programming</article-title>
          , vol.
          <volume>36</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>5</fpage>
          -
          <lpage>25</lpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>P.</given-names>
            <surname>Caspi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Pilaud</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Halbwachs</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Plaice</surname>
          </string-name>
          , “
          <article-title>Lustre: A declarative language for programming synchronous systems</article-title>
          ,” in POPL,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>S.</given-names>
            <surname>Kriebel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Raco</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          , “
          <article-title>Model-Based Engineering for Avionics: Will Specification and Formal Verification e.g. Based on Broy's Streams Become Feasible?,” in [Software Engineering (SE) und Software Management (SWM)</article-title>
          ,
          <source>SE SWM</source>
          ,
          <year>2019</year>
          -
          <volume>02</volume>
          -
          <fpage>18</fpage>
          - 2019-02- 22, Stuttgart, Germany], pp.
          <fpage>87</fpage>
          -
          <lpage>94</lpage>
          , BMW Group, Chair of Software Engineering at RWTH Aachen,
          <year>Feb 2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>J.</given-names>
            <surname>Souyris</surname>
          </string-name>
          , “
          <article-title>Formal methods at airbus: Experience feedback</article-title>
          ,”
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>K.</given-names>
            <surname>Ho</surname>
          </string-name>
          <article-title>¨ lldobler and B</article-title>
          .
          <string-name>
            <surname>Rumpe</surname>
            , MontiCore 5
            <given-names>Language</given-names>
          </string-name>
          <string-name>
            <surname>Workbench Edition 2017. Aachener</surname>
          </string-name>
          Informatik-Berichte,
          <source>Software Engineering, Band</source>
          <volume>32</volume>
          ,
          <string-name>
            <surname>Shaker</surname>
            <given-names>Verlag</given-names>
          </string-name>
          ,
          <year>December 2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          ,
          <article-title>Formale Methodik des Entwurfs verteilter objektorientierter Systeme</article-title>
          .
          <source>PhD thesis</source>
          , Zugl.: M u¨nchen, Techn. Univ, Zugl. Mu¨ nchen,
          <year>1996</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Haber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. O.</given-names>
            <surname>Ringert</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          , MontiArc
          <article-title>- Architectural modeling of interactive distributed and cyber-physical systems</article-title>
          , vol.
          <year>2012</year>
          ,3 of Technical report / Department of Computer Science, RWTH Aachen.
          <article-title>Aachen and Hannover and Go¨ ttingen: RWTH and Technische Informationsbibliothek u</article-title>
          .
          <source>Universita¨tsbibliothek and Niedersa¨chische Staatsund Universita¨tsbibliothek</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>T.</given-names>
            <surname>Nipkow</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. C.</given-names>
            <surname>Paulson</surname>
          </string-name>
          , and M. Wenzel, Isabelle/HOL:
          <article-title>A proof assistant for Higher-Order Logic</article-title>
          , vol.
          <source>2283 of Lecture notes in artificial intelligence</source>
          . Berlin [etc.]: Springer,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>D.</given-names>
            <surname>Harel</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          , “
          <article-title>Meaningful modeling: What's the semantics of ”semantics”</article-title>
          ?,” Computer, vol.
          <volume>37</volume>
          , pp.
          <fpage>64</fpage>
          -
          <lpage>72</lpage>
          ,
          <year>11 2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Broy</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Stølen</surname>
          </string-name>
          ,
          <article-title>Specification and development of interactive systems: Focus on streams, interfaces</article-title>
          , and
          <string-name>
            <surname>Refinement</surname>
          </string-name>
          . New York: Springer,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>R.</given-names>
            <surname>Milner</surname>
          </string-name>
          ,
          <article-title>Communicating and mobile systems: the pi calculus</article-title>
          . Cambridge university press,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>C. A. R.</given-names>
            <surname>Hoare</surname>
          </string-name>
          , “
          <article-title>Communicating sequential processes,” in The origin of concurrent programming</article-title>
          , pp.
          <fpage>413</fpage>
          -
          <lpage>443</lpage>
          , Springer,
          <year>1978</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>W.</given-names>
            <surname>Reisig</surname>
          </string-name>
          ,
          <article-title>Petri nets: an introduction</article-title>
          ,
          <source>vol. 4</source>
          . Springer Science &amp; Business
          <string-name>
            <surname>Media</surname>
          </string-name>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>J. O.</given-names>
            <surname>Ringert</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          , “
          <string-name>
            <given-names>A Little</given-names>
            <surname>Synopsis on Streams</surname>
          </string-name>
          ,
          <source>Stream Processing Functions, and State-Based Stream Processing,” Int. J. Software and Informatics</source>
          , vol.
          <volume>5</volume>
          , no.
          <issue>1-2</issue>
          , pp.
          <fpage>29</fpage>
          -
          <lpage>53</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>D.</given-names>
            <surname>Cofer</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Miller</surname>
          </string-name>
          , “
          <article-title>Do-333 certification case studies,” in NASA Formal Methods (J. M. Badger and</article-title>
          K. Y. Rozier, eds.),
          <source>(Cham)</source>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          , Springer International Publishing,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>J.</given-names>
            <surname>Warmer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Kleppe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Clark</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ivner</surname>
          </string-name>
          , J. Ho¨ gstro¨ m, M. Gogolla,
          <string-name>
            <given-names>M.</given-names>
            <surname>Richters</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Hussmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Zschaler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Johnston</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Frankel</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Bock</surname>
          </string-name>
          ,
          <source>Object Constraint Language 2.0. 01</source>
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>J. C</surname>
            . B u¨rger,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Kausch</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Raco</surname>
            ,
            <given-names>J. O.</given-names>
          </string-name>
          <string-name>
            <surname>Ringert</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Rumpe</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <article-title>St u¨ber, and</article-title>
          <string-name>
            <given-names>M.</given-names>
            <surname>Wiartalla</surname>
          </string-name>
          , “
          <article-title>Towards an Isabelle Theory for distributed, interactive systems - the untimed case</article-title>
          ,
          <source>” Tech. Rep. AIB-2020-02</source>
          ,
          <string-name>
            <given-names>RWTH</given-names>
            <surname>Aachen</surname>
          </string-name>
          ,
          <year>Jan</year>
          .
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>M.</given-names>
            <surname>Broy</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Rumpe</surname>
          </string-name>
          , “
          <article-title>Modulare hierarchische Modellierung als Grundlage der Software-</article-title>
          und
          <string-name>
            <surname>Systementwicklung</surname>
          </string-name>
          ,” Informatik-Spektrum, vol.
          <volume>30</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>18</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>C.</given-names>
            <surname>Ballarin</surname>
          </string-name>
          , “Locales and locale expressions in isabelle/isar,” in
          <source>International Workshop on Types for Proofs and Programs</source>
          , pp.
          <fpage>34</fpage>
          -
          <lpage>50</lpage>
          , Springer,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>L.</given-names>
            <surname>Bulwahn</surname>
          </string-name>
          , “
          <article-title>The new quickcheck for isabelle: random, exhaustive and symbolic testing under one roof</article-title>
          ,” pp.
          <fpage>92</fpage>
          -
          <lpage>108</lpage>
          ,
          <year>12 2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>J. C.</given-names>
            <surname>Blanchette</surname>
          </string-name>
          and
          <string-name>
            <given-names>T.</given-names>
            <surname>Nipkow</surname>
          </string-name>
          , “
          <article-title>Nitpick: A Counterexample Generator for Higher-Order Logic Based on a Relational Model Finder,” in Interactive Theorem Proving (M. Kaufmann</article-title>
          and L. C. Paulson, eds.), (Berlin, Heidelberg), pp.
          <fpage>131</fpage>
          -
          <lpage>146</lpage>
          , Springer Berlin Heidelberg,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>B. C.</given-names>
            <surname>Huffman</surname>
          </string-name>
          , HOLCF '11:
          <article-title>A definitional domain theory for verifying functional programs</article-title>
          .
          <source>[Portland</source>
          , Or.]: Portland State University,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>