<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Reliability Requirements Engineering in Socio-Cyber-Physical System Context</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ksenija Lace</string-name>
          <email>ksenija.lace@rtu.lv</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marite Kirikova</string-name>
          <email>marite.kirikova@rtu.lv</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Riga Technical University</institution>
          ,
          <country country="LV">Latvia</country>
        </aff>
      </contrib-group>
      <fpage>251</fpage>
      <lpage>266</lpage>
      <abstract>
        <p>One of the system quality characteristics is system reliability, which defines the degree to which a system, a product or a component performs specified functions under specified conditions for a specified period of time. System reliability significantly impacts also other system quality characteristics, such as performance and usability, and often is the key factor impacting overall quality of the system. With emerging new generation of cyber systems, where three dimensions - socio, cyber and physical are tightly linked together in order to achieve common goals or solve common problems, system reliability became even more important. This paper proposes the approach for reliability requirements engineering in the context of SCPS. The approach integrates Failure Mode Effects Analysis and Morphological Analysis in reliability requirements engineering, for the in-depth multi-dimensional analysis of potential failure scenarios.</p>
      </abstract>
      <kwd-group>
        <kwd>Socio-Cyber-Physical Systems</kwd>
        <kwd>Reliability</kwd>
        <kwd>Maintenance</kwd>
        <kwd>Downtime</kwd>
        <kwd>Deployment</kwd>
        <kwd>Software Upgrading</kwd>
        <kwd>Maintenance automation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Socio-cyber-physical systems (SCPS) are complex real-time systems of systems,
which have very high expectations for reliability. At the same time, achieving
reliability in SCPS is a very difficult task due to several reasons – high uncertainty, different
nature of system elements, emergent system behavior, and many interdependencies
between system components. This requires much more comprehensive analysis for
reliability requirements then is performed in traditional requirements engineering [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        Reliability research has dramatic importance, due to the following factors [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1, 2, 3</xref>
        ]:
 Reliability expectations dramatically increased during the latest decade.
 The complexity of developed systems is leading to the high level of uncertainty,
meaning the potential risk of lower system reliability.
 Development projects usually have limited resources, including time and budget,
that might again lead to decreased reliability.
      </p>
      <p>
        Despite the fact that reliability engineering has evolved during last decades and has
comprehensive research, and wide range of available techniques for reliability
prediction and failure analysis, there is still no common methodology for holistic reliability
engineering process [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        In both, reliability engineering and requirements engineering areas, not much
research is focused on reliability requirements perspective specifically, and there is no
research in the field of integration of reliability requirements engineering and
reliability engineering disciplines [
        <xref ref-type="bibr" rid="ref1 ref3">1, 3</xref>
        ].
      </p>
      <p>Much deeper integration of requirements engineering and reliability engineering
activities can be a possible solution.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Research Method</title>
      <p>In the scope of this research, the following activities were executed:
1. Investigation of existing research in the field of reliability requirements. The main
focus was on research covering reliability for real-time systems, reliability of
physical and social systems, and managing reliability in high uncertainty.
2. Identification of challenges for reliability requirements engineering in SCPS
context, comparative analysis of existing reliability requirements engineering
approaches from the perspective of their applicability in the context of SCPS
challenges.
3. Investigation of existing research in the field of morphological analysis. The main
focus was on approach applicability for process modeling, software requirements
analysis, and complex systems analysis.
4. Proposing the approach for reliability requirements engineering, combining
reliability engineering techniques with morphological analysis for multi-dimensional
failure analysis.
5. Practical application of the proposed approach for the SCPS example.
6. Building conclusions about applicability of the proposed approach and defining
possible further steps of research.
3</p>
    </sec>
    <sec id="sec-3">
      <title>SCPS Reliability Requirements Engineering</title>
      <p>
        The first official definition of reliability was stated in 1957, by Advisory Group on the
reliability of Electronic Engineering (AGREE). According to this definition,
reliability is the probability of a product performing a specified function without failure
under given conditions for a specified period of time [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>Two reliability standards, commonly used today – ISO and IEEE, have similar
definitions of reliability:
 According to the IEEE standard, focusing on the software reliability, the reliability
is the probability that software will not cause the failure of a system for a specified
time under specified conditions.
 According to the ISO standard, focusing on more generic system level, the
reliability is the degree to which a system, product or component performs specified
functions under specified conditions for a specified period of time.</p>
      <p>
        These definitions can be used also for reliability of SCPS. Usually, required
reliability is described in the format of reliability requirements [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. From the stated
reliability definitions, reliability requirements should provide the following:
 The definitions of system functions, for which reliability is defined.
 Conditions for function execution.
 Definitions of function successful execution or function failure, depending on the
reliability metrics.
      </p>
      <p>
        Reliability requirements should be based on the facts (real data) and should be
focused on the most critical reliability aspects (real need) [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        Reliability requirements should be a prerequisite to any reliability engineering
activities. Quality of reliability requirements has direct impact on the reliability of the
system. But unfortunately, there are several common reliability requirements
problems, which may make it harder to use reliability requirements for reliability
engineering activities later in the project [
        <xref ref-type="bibr" rid="ref5 ref6 ref7 ref8">5 – 8</xref>
        ]. Many of them can be addressed using
traditional requirements engineering techniques. However, the problem that
Reliability requirements are too generic and cannot be traced to the requirements
implementation strategies, cannot be solved in the scope of traditional requirements engineering
as it focuses on specifying reliability requirements, not the in-depth analysis of the
potential issues for achieving desired reliability. This problem can be solved through
integrating reliability engineering techniques into requirements engineering, which
would link together requirements and implementation activities [
        <xref ref-type="bibr" rid="ref1 ref9">1, 9</xref>
        ].
      </p>
      <p>
        However, using existing reliability engineering techniques for SCPS might have
several challenges due to SCPS complexity, diversity and emergency [
        <xref ref-type="bibr" rid="ref1 ref10 ref4">1, 4, 10</xref>
        ]:
 Techniques do not take into account the nature of system component – SCPS
socio, cyber and physical parts might need different approaches.
 Failure impact assessment and failure prioritization is not fully supported in the
existing techniques; however, due to high complexity of the SCPS, addressing all
possible failures is just unrealistic. This is why impact assessment and focus on the
failures with highest negative impact should be the key point in SCPS reliability.
 Historical failure data is a prerequisite for reliability prediction – SCPS adapt and
emerge, as well as human and software reliability cannot be so well predicted
based only on the statistics, and historical data not always will be relevant for the
future prediction.
 Approaches are more tended to focus on separate system components than on a
system as a whole – a SCPS cannot be seen as just a list of components, as
relationships and dependencies between components play significant role.
 Existing approaches often are based on the difficult calculations and are hard to
use – SCPS complexity might lead to the need of assumptions and simplifications,
which, in turn, will lead to lower accuracy.
      </p>
    </sec>
    <sec id="sec-4">
      <title>Reliability Engineering Techniques</title>
      <p>
        Based on the available research of specific reliability engineering techniques, the
most popular techniques were reviewed for potential applicability for reliability
requirements engineering [
        <xref ref-type="bibr" rid="ref11 ref2 ref8">2, 8, 11</xref>
        ].
      </p>
      <p>Techniques for reliability analysis can be applied for reliability requirements
indepth failure analysis and connecting with the potential reliability engineering
strategies.</p>
      <p>In the scope of this research failure mode and effect analysis (FMEA) technique
was selected for the integration into requirements engineering activities, specifically,
for analyzing possible failure reasons and potential negative impact for specified
reliability requirements. FMEA was preferred over fault tree analysis (FTA) technique as
it is, in particular, efficient when a large number of different failure scenarios exist
within a range of negative impact, not just a specific failure scenario that should be
investigated.</p>
      <p>
        FMEA usually consists of seven sequential phases [
        <xref ref-type="bibr" rid="ref12 ref13 ref14 ref15">12 – 15</xref>
        ]:
 Detection of possible failure modes for selected system components.
 Evaluate severity for each failure mode.
 Evaluate probability for each failure mode.
 Evaluate existing detection controls for each failure mode.
 Evaluate overall risk for each failure mode and select the ones with the highest
risk.
 Determine actions to reduce risk for selected failure modes.
 Take appropriate actions and recalculate risk.
      </p>
      <p>
        However, as stated before, there are several enhancements in existing reliability
techniques, which are required for their successful application in SCPS
domain [
        <xref ref-type="bibr" rid="ref11 ref16 ref17 ref18 ref19">11, 16 – 19</xref>
        ]:
 Technique should cover all three dimensions – socio, cyber and physical ones.
 Technique should be applicable in situations, when there is no historical data
available.
 Technique should be able to review and evaluate many possible failure reasons
from different possible failure aspects.
 Technique should support the selection of the most efficient strategy, based on the
multi-dimensional evaluation.
 Technique should have a clear algorithm, which could be automated.
5
      </p>
    </sec>
    <sec id="sec-5">
      <title>Morphological Analysis</title>
      <p>
        One of the proven approaches for the analysis and modelling complex
multidimensional systems is morphological analysis (MA). It was initially used for
modelling relationships between structural components in different scientific fields like
botany, linguistics, geology and mathematics [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. More abstract version of MA was
proposed by Swiss-American physicist and astronomer Fritz Zwicky. He also started
to use it for social problem solving and technological engineering. Nowadays MA is
applied in many different cases, and has proved to be efficient for the following
purposes [
        <xref ref-type="bibr" rid="ref21 ref22">21, 22</xref>
        ]:
 Developing alternative strategies.
 Assessing organizational readiness for different goals.
 Developing possible scenarios.
 Assessing possible risks.
 Analyzing cause-sequence relationships.
 Presenting complex situations in a more easy-to-understand format.
      </p>
      <p>
        MA is very effective method for analyzing complex problems from different
possible aspects. It also supports generation of various possible solutions for these
problems [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ].
      </p>
      <p>
        MA can be used in system engineering during two main activities – (1) Building
general model of a problem space and (2) Generating possible system solutions for
addressing the problem. MA also supports innovation engineering, as, through this
method, new previously unknown aspects can become visible, which stimulates new
ideas for the solution space [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ].
      </p>
      <p>
        In general, morphological analysis is the process of sequential analysis and
synthesis activities with a purpose of exploring different aspects of the specific complex,
non-quantified problem and identifying all possible solutions [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. During analysis
step, the problem is structured using problem describing parameters and possible
parameter values. During synthesis step, the values of different parameters are
grouped into possible configurations, and resulting configurations are assessed from
the aspect of probability. Configurations, that are not realistic, are excluded [
        <xref ref-type="bibr" rid="ref21 ref25">21, 25</xref>
        ].
      </p>
      <p>
        In the scope of engineering, morphological analysis should include also some
additional steps – (i) As a first activity – the problem should be formulated as precisely as
possible and (ii) As a final activity – remaining realistic configurations can be then
assessed from the selected perspective and the best configuration(-s) selected (for
instance, all configurations supporting specific value for specific parameter, are
identified) [
        <xref ref-type="bibr" rid="ref21 ref26">21, 26</xref>
        ].
      </p>
      <p>
        But like any other method, MA has its own disadvantages. The main constraint for
the application of this method is the workshop format, where highly motivated,
knowledgeable system-thinking participants are working together. Depending on the
complexity of the problem studied, MA also can be quite time consuming and
requiring decent automation level [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].
      </p>
      <p>In scope of this research, MA is integrated into FMEA approach for in-depth
multidimensional analysis of possible failure reasons for system components which can
affect defined reliability requirements. Additionally, MA is used on the possible
failure negative impact minimization analysis level – for the definition of possible impact
minimization strategies. This is not the traditional application of MA, when potential
solutions are generated based on the possible configurations, but rather might be
interpreted as the structuring method for failure negative impact problem space and
assessing efficiency of potential strategies for improving the negative impact.</p>
    </sec>
    <sec id="sec-6">
      <title>SCPS Reliability Requirements Elicitation Process</title>
      <p>
        Reliability requirements elicitation process is organized as four related phases [
        <xref ref-type="bibr" rid="ref1 ref14 ref5">1, 5,
14</xref>
        ]:
1. Reliability requirements initial definition – during this phase functional
requirements and appropriate reliability requirements are elicited and documented using
standard requirements engineering activities. After that, for selected functional
requirements, supporting system components are identified. At the end, the most
critical components are selected for further analysis.
2. Failure mode analysis – in this phase, for each selected component, possible failure
reasons are analyzed. Failure mode with the highest potential negative impact is
selected for further analysis. Selected reason is divided into possible specific
subreasons and for each sub-reason; the most possible negative failure impact is
evaluated. The sub-reason with the highest possible negative impact is selected for the
next phase.
3. Failure minimization strategy – during this phase possible solution strategies are
generated for previously defined and assessed failure sub-reason with highest
negative impact. The strategies improving failure mode parameters are selected for the
next phase.
4. Reliability requirements detailed definition – based on the selected solution, initial
reliability requirements are detailed with possible failure scenarios and selected
strategies. If initially defined reliability cannot be achieved, reliability
requirements are redefined.
      </p>
      <p>In the following sub-sections each phase is described in more detail. In Fig.1 can
be seen how exactly proposed method is organized by adjusting traditional FMEA
and MA, and then combining together adjusted methods. MA is integrated into
FMEA on the failure mode analysis phase, which is not very strictly defined for
FMEA. MA however helps to evaluate failure modes through different perspectives
and generate all possible failure mode configurations we should consider.</p>
      <p>
        Proposed methodology is illustrated by the example study – Live Intelligent
Tutoring System. The main goal of intelligent tutoring system is to create intelligent agents
in cyber space that can teach students like live tutors – be autonomous and adapt to
each student needs. [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ]. But Intelligent Tutoring System for corporate training
should be considered as the socio-cyber-physical System, having several additional
characteristics:
 Socio space – training organization and organization where employees will use
gained knowledge and skills should be considered as parts of a system.
 Physical space – physical equipment is used for training or training is aimed to
teach how to use specific physical equipment.
 Cyber space – should have representations of both socio and physical spaces.
      </p>
      <p>vising cyber elements.
6.1</p>
      <sec id="sec-6-1">
        <title>Reliability Requirements Initial Definition</title>
        <p>In this phase reliability requirements should be defined for the system as reliability
requirements for critical system functions.</p>
        <p>After reliability is defined on function level, system should be structured into
subsystems, which support specific system functions. Each sub-system should be
presented as a group of different cyber, physical and socio elements:
 Cyber element – can be software system, system module or specific service.
 Physical element – can be computer hardware, different equipment or equipment
 Socio element – can be humans operating with physical elements, using or
supertions, socio, cyber and physical components are listed. After all components and
functions are listed, for each function supporting components are identified, and for each
supporting component importance (I) is defined using 1 (required very rare) ... 10
(component is critical) scale – 1 to 10.</p>
        <p>Overall component significance (S) is calculated using the following formula (n –
number of functions supported by component):
 =</p>
        <p>∑
=1 ( )</p>
        <p>(1)
As the next step, for each user requirement supporting system components are
identified.
In this phase, for each selected component, the possible failure reasons are defined.
For this MA approach is applied, using the following parameters for the definition of
different possible failures:
 Reason – different types of failure reasons for specified cyber/physical/socio
elements. For instance – software/hardware failure, maintenance or upgrade, human
related accident.
 Scale – amount of other components that failure will affect. Some generic values
can be one, some, many, and all. Can be also a specific number.
 Length – different possible failure affect length intervals. For instance: &lt;5, 5-10,
10-30, 30-60, &gt;60 minutes.
 Frequency – failure incidents frequency in specific time interval. For instance – x
times in year, month, week, day, and hour.
 Recovery effort – different levels of required recovery effort during and after the
failure - almost none, low, medium, high, extremely high.
 Impact level – different levels of failure impact on key system efficiency metrics –
almost none, low, medium, high, extremely high.
 Control level – different levels of how well this failure reason can be controlled by
system internal mechanisms; or this reason is external and cannot be impacted.
Some possible generic values can be % of control.
 Process maturity – different maturity levels of existing failure handling process –
automatic, semi-automatic, fully manual, not formalized, not existing.</p>
        <p>Defining specific values for each of these parameters the space of all possible
failure aspects is created – each column in the Table 2 contains all possible values for the
specific parameter. Note, that some of parameters can have more possible values and
some less; at this point each parameter values are completely independent from other
parameters.
As the next step a cross-consistency check for all defined failure aspects is performed.
In Table 3 all possible parameter values are listed both in columns and in rows, and
then each possible pair of parameter values is evaluated from the two perspectives:
 Definition of interrelated parameters - several parameters are always directly
interconnected, that means that changing values for one parameter will directly
influence value of another parameter – for instance, failure length affects impact level.</p>
        <p>Related parameter values should be highlighted using specific color in the table.
 Definition of not consistent values – for interconnected parameters several values
are not consistent, that means that these values cannot coexist and are exclusive –
for instance, long failure period and small failure impact. Inconsistent parameter
values should be marked in the table using “X” or similar symbol.</p>
        <p>Note, that parameter “Reason” is not included in rows and parameter “Process
maturity” is not listed in columns; – this is not needed, as we are not evaluating pairs of
values for the same parameter, only for different parameters. Due to space
limitations, instead of full names of parameters and values in two first columns just first
letters of parameters and numbers of values are listed.</p>
        <p>Based on the cross-consistency check results, for each reason, number of
immutable impact parameter values with highest negative impact is counted (this means that
a specific reason cannot lead to the highest negative impact). The reason with the
lowest number is the one with highest possible impact.</p>
        <p>For the selected reason different possible failure sub-reasons are identified and for
each of them the appropriate failure mode is generated as a configurations of different
parameter values, where this reason can be a failure cause.</p>
        <p>.tl1onhyM .lee2kyW .lia3yD .ltse1onnoAm .o2Lw .ie3dumM .igh4H .ltse1ononAm .o2Lw .ie3dumM .i4ghH .1010% .052% .03%</p>
        <p>Evaluation is made manually, based on the forming parameter values.
Configuration with the highest possible negative impact is selected for further analysis. In our
example, in the Table 3 all immutable parameters are highlighted in red, and the
smallest number of such “red” cells has the reason “Software bug”. It is important to
mention, that the process should be iterative, and after selected configuration is
analyzed and possible impact minimization strategies selected, the negative impact
should be recalculated and next configuration with the highest negative impact should
be selected. The process continues until the configuration with the highest impact will
be the one, which was already analyzed.</p>
        <p>Next step is to identify and evaluate all possible failure modes in case of the reason
“Software bug”, taking into account immutable values for other parameters (for
instance, in the example “Low Impact” is immutable with “Many affected components”
and “High recovery”). As there can be too many possible combinations, we need to
divide “Software bug” reason into more specifics reasons; and for each of them to
evaluate negative impact. The following “Software bug” more specific reasons are
selected:
1. Software is not compatible with user device.
2. Software is not tested.
3. Change in one software part broke another software part.
4. Not compatible authentication service.
6.3</p>
      </sec>
      <sec id="sec-6-2">
        <title>Failure Minimization Strategy</title>
        <p>In this phase, for the selected failure reason with maximal possible negative impact
minimization strategies are generated. Some possible approaches for choosing the
strategy can be:
1. Prediction approach – failure prediction and addressing in preventive manner.
2. Quality approach – quality of system operation and produced output. Minimization
of the risk that it will make system failures more frequent and more difficult to
recover from.
3. Proactive maintenance approach – system health monitoring and maintenance in
advance.
4. Recovery efficiency approach – efficiency of recovery process after failure
occurred, minimization of over processing and not required actions and motions.
5. Backup utilization approach – different backup resources for usage during system
recovery after failure.
6. External services and vendors availability approach – efficiency of external
services handling, external resources reliability.
7. Inventory approach - waiting/wasted resources during recovery, resources
utilization during recovery to minimize related costs and effort.</p>
        <p>After that each possible strategy should be assessed from the point of the efficiency
in the selected failure mode. Strategy is added as one additional parameter for failure
mode space definition. Using cross-consistency matrix (see Table 4), the impact of
each strategy on the each failure mode parameter is evaluated. The combination of
strategies affecting as many as possible parameters, is selected for the next phase.
X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X X
X X X X X X X X X X X X X X X</p>
        <p>X X
As in our example we want to select strategies for improvements in all failure mode
parameters, two strategies would be sufficient – quality and recovery efficiency. We
also can define some specific activities in the scope of selected strategies (for
instance, define how exactly quality will be achieved – additional manual testing,
automated tests, etc.)
6.4</p>
      </sec>
      <sec id="sec-6-3">
        <title>Reliability Requirements Final Definition</title>
        <p>In this phase initially stated reliability requirements are elaborated, and the following
information is added to each of them:
 List of components supporting appropriate function.
 For each component the failure reason, sub-reason and failure mode with the
highest overall risk is described.
 For each failure the strategy with the highest benefit is described.</p>
        <p>If selected strategy cannot guarantee initially stated reliability requirement,
requirement should be adjusted.</p>
        <p>For instance, first component – system student interface had the following
methodology application results:
 The failure reason with the most possible negative impact is the software bug,
specifically – not compatible authentication service.
 The combination of strategies that can be used for minimization of failure impact
are quality and recovery efficiency.
7</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>Conclusions</title>
      <p>System reliability is one of the key quality characteristics of any system, meaning that
without the ability to perform system’s functions in specific conditions for specific
period of time, system cannot be used efficiently. At the same time reliability is very
hard to achieve due to high uncertainty about potential system failures during system
development. Reliability requirements should form the basis for system reliability
engineering, stating exactly what level reliability is required. However often
reliability requirements are stated based only on the stakeholders’ opinion and are too
generic to be transformed into specific system functions or reliability engineering activities.</p>
      <p>In the scope of this research the new approach of reliability requirements
engineering was proposed, as an integration of existing reliability engineering technique for
analysis of potential failure reasons and related negative impact – failure mode and
effect analysis (FMEA). This approach allows not only stating reliability requirement
as a number of successfully executed system functions per period of time or number
of usage, but also defines critical system components for supporting related functions,
possible failure reasons and impact, as well as the most efficient strategies for
addressing this impact.</p>
      <p>Morphological analysis is incorporated for multi-dimensional analysis of failure
impacts, as well as for evaluating possible strategies for addressing the impact.</p>
      <p>The approach was applied for the example of live tutoring system; and based on
the application results the following opportunities of the further research were
identified:
 The approach should support evaluation of the initially stated reliability
requirement, based on the defined failure mode and risk strategy, so that in the final stage
“Reliability requirements final definition” it is possible to evaluate realistic
reliability that can be achieved and agree on the new requirement or reiterate failure
analysis and come up with new strategies for achieving desired reliability level.
 Different evaluations in the approach that now are based on the human assessments
(for instance, functions supporting components, failure modes impact) should be
reworked into automated evaluation that can be performed by the machine. This is
needed if we want to integrate this approach into SCPS.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Woo</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Introduction to Reliability Design of Mechanical/Civil System</article-title>
          .
          <source>In Reliability Design of Mechanical Systems: A Guide for Mechanical and Civil Engineers</source>
          , Cham: Springer International Publishing, pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Jiang</surname>
          </string-name>
          , R.:
          <article-title>Design Techniques for Reliability</article-title>
          . In Introduction to Quality and Reliability Engineering, Berlin, Heidelberg: Springer Berlin Heidelberg pp.
          <fpage>147</fpage>
          -
          <lpage>168</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Malkawi</surname>
            ,
            <given-names>M. I.:</given-names>
          </string-name>
          <article-title>The art of software systems development: Reliability, Availability, Maintainability, Performance (RAMP). Human-centric Comput</article-title>
          .
          <source>Inf. Sci.</source>
          ,
          <volume>3</volume>
          (
          <issue>1</issue>
          ), 22,
          <string-name>
            <surname>Dec.</surname>
          </string-name>
          (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Tan</surname>
            ,
            <given-names>C. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carlo</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Overview of Reliability Engineering</article-title>
          .
          <source>In Theory and Practice of Quality and Reliability Engineering in Asia Industry</source>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>23</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Alho</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mattila</surname>
          </string-name>
          , J.:
          <article-title>Breaking down the requirements: Reliability in remote handling software</article-title>
          .
          <source>Fusion Eng</source>
          . Des.,
          <volume>88</volume>
          (
          <issue>9</issue>
          ),
          <fpage>1912</fpage>
          -
          <lpage>1915</lpage>
          (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Immonen</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pakkala</surname>
            ,
            <given-names>D.:</given-names>
          </string-name>
          <article-title>A survey of methods and approaches for reliable dynamic service compositions</article-title>
          .
          <source>Serv. Oriented Comput. Appl.</source>
          ,
          <volume>8</volume>
          (
          <issue>2</issue>
          ),
          <fpage>129</fpage>
          -
          <lpage>158</lpage>
          , Jun. (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Kusters</surname>
            ,
            <given-names>R. I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Van</surname>
            <given-names>Solingen</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            ,
            <surname>Trienekens</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. J. M.</given-names>
            ,
            <surname>Wijnands</surname>
          </string-name>
          , H.:
          <article-title>User-perceptions Of Embedded Software Reliability</article-title>
          . In: Gritzalis D. (eds) Reliability,
          <article-title>Quality and Safety of Software-Intensive Systems</article-title>
          . IFIP - The
          <source>International Federation for Information Processing</source>
          . Springer, Boston, MA, pp
          <fpage>67</fpage>
          -
          <lpage>82</lpage>
          (
          <year>1997</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Ahuja</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Reliability Requirements, Risk Management, and Associated Building Systems Engineering</article-title>
          .
          <article-title>In Integration of Nature and Technology for Smart Cities</article-title>
          , Cham: Springer International Publishing, pp.
          <fpage>203</fpage>
          -
          <lpage>222</lpage>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Lawrence</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dunn</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pe</surname>
            ,
            <given-names>R. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dunn</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>South</surname>
            ,
            <given-names>W. L.</given-names>
          </string-name>
          :
          <article-title>Optimal system reliability: The way forward</article-title>
          .
          <source>In 2008 55th IEEE Petroleum and Chemical Industry Technical Conference</source>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>7</lpage>
          (
          <year>2008</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Xie</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Models</surname>
            ,
            <given-names>S. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Applications</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Software Reliability Models for Practical Applications</article-title>
          . In Software Quality and
          <article-title>Productivity: Theory, practice, education and training</article-title>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.-Z.</given-names>
            <surname>Barta</surname>
          </string-name>
          , and P. Juliff, Eds. Boston, MA: Springer US, pp.
          <fpage>211</fpage>
          -
          <lpage>214</lpage>
          (
          <year>1995</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Kravets</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Calvert</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Krishnan</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schwan</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Adaptive Variation of Reliability. In High Performance Networking VII: IFIP TC6 Seventh International Conference on High Performance Networks (HPN `97), 28th April -</article-title>
          -
          <source>2nd May</source>
          <year>1997</year>
          ,
          <string-name>
            <given-names>White</given-names>
            <surname>Plains</surname>
          </string-name>
          , New York, USA, A. Tantawy, Ed. Boston, MA: Springer US, pp.
          <fpage>202</fpage>
          -
          <lpage>216</lpage>
          (
          <year>1997</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Peeters</surname>
            ,
            <given-names>J. F. W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Basten</surname>
            ,
            <given-names>R. J. I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tinga</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Improving failure analysis efficiency by combining FTA and FMEA in a recursive manner</article-title>
          .
          <source>Reliab. Eng. Syst. Saf.</source>
          , vol.
          <volume>172</volume>
          , no.
          <source>December</source>
          <year>2017</year>
          , pp.
          <fpage>36</fpage>
          -
          <lpage>44</lpage>
          (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Khaiyum</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kumaraswamy</surname>
            ,
            <given-names>Y. S.:</given-names>
          </string-name>
          <article-title>An Effective Method for the Identification of Potential Failure Modes of a System by Integrating FTA and FMEA</article-title>
          .
          <source>In ICT and Critical Infrastructure: Proceedings of the 48th Annual Convention of Computer Society of India- Vol I, vol. I</source>
          , pp.
          <fpage>679</fpage>
          -
          <lpage>686</lpage>
          (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Gigante</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gargiulo</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ficco</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pascarella</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>For Consistency Verification Between Requirements and FMEA</article-title>
          . pp.
          <fpage>403</fpage>
          -
          <lpage>413</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Spreafico</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Russo</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rizzi</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>A state-of-the-art review of FMEA/FMECA including patents</article-title>
          .
          <source>Comput. Sci. Rev</source>
          ., vol.
          <volume>25</volume>
          , pp.
          <fpage>19</fpage>
          -
          <lpage>28</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Fleischmann</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kohl</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Franke</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>A modular web framework for socio-CPS-based condition monitoring</article-title>
          .
          <source>In 2016 IEEE World Conference on Factory Communication Systems (WFCS)</source>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Lenzini</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mauw</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ouchani</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Security analysis of socio-technical physical systems</article-title>
          .
          <source>Comput. Electr. Eng.</source>
          , vol.
          <volume>47</volume>
          , pp.
          <fpage>258</fpage>
          -
          <lpage>274</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Ritchey</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Modelling Complex Socio-Technical Systems Using Morphological Analysis Adapted from an address to the Swedish Parliamentary IT Morphological Analysis: What is MA used for ? Mess</article-title>
          . Russell J. Bertrand Russell Arch. (
          <year>2002</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Harvey</surname>
            ,
            <given-names>P. L.</given-names>
          </string-name>
          :
          <article-title>Toward a Discovery and Strategic Alignment Matrices for Socio-technical Systems ' Design. Community Informatics Design Applied to Digital Social Systems</article-title>
          , vol.
          <volume>12</volume>
          , pp
          <fpage>279</fpage>
          -
          <lpage>312</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Ritchey</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Society</surname>
            ,
            <given-names>S. M.</given-names>
          </string-name>
          :
          <article-title>Wicked Problems: Modelling Social Messes with Morphological Analysis</article-title>
          .
          <source>Acta Morphol. Gen.</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ), pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Duczynski</surname>
          </string-name>
          , G.:
          <article-title>Morphological analysis as an aid to organisational design and transformation</article-title>
          .
          <source>Futures</source>
          , vol.
          <volume>86</volume>
          , pp.
          <fpage>36</fpage>
          -
          <lpage>43</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Johansen</surname>
            ,
            <given-names>I.: Technological</given-names>
          </string-name>
          <string-name>
            <surname>Forecasting</surname>
          </string-name>
          &amp;
          <article-title>Social Change Scenario modelling with morphological analysis</article-title>
          .
          <source>Technol. Forecast. Soc. Chang.</source>
          , vol.
          <volume>126</volume>
          , no.
          <source>February</source>
          <year>2017</year>
          , pp.
          <fpage>116</fpage>
          -
          <lpage>125</lpage>
          (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Levina</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kranich</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Mobility and the Internet of People: A Morphological Analysis</article-title>
          .
          <source>In 2016 Intl IEEE Conferences on Ubiquitous Intelligence &amp; Computing</source>
          , (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), pp.
          <fpage>915</fpage>
          -
          <lpage>920</lpage>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>For</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Creative</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Morphological approach to non-quantified modeling Morpological approach to problem solution (</article-title>
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Ritchey</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Principles of Cross-Consistency Assessment in General Morphological Modelling</article-title>
          .
          <volume>4</volume>
          (
          <issue>2</issue>
          ), pp.
          <fpage>1</fpage>
          -
          <lpage>20</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Araújo</surname>
            ,
            <given-names>R. D. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Soares</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oliveira</surname>
            ,
            <given-names>A. L. I.</given-names>
          </string-name>
          :
          <article-title>Expert Systems with Applications Hybrid morphological methodology for software development cost estimation</article-title>
          . Vol.
          <volume>39</volume>
          , pp.
          <fpage>6129</fpage>
          -
          <lpage>6139</lpage>
          (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Nwana</surname>
            ,
            <given-names>H. S.:</given-names>
          </string-name>
          <article-title>Intelligent tutoring systems: an overview</article-title>
          .
          <source>Artif. Intell. Rev.</source>
          ,
          <volume>4</volume>
          (
          <issue>4</issue>
          ), pp.
          <fpage>251</fpage>
          -
          <lpage>277</lpage>
          , Dec. (
          <year>1990</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>