[Motivation] In the engineering of cyber-physical systems, attention is given to early validation of requirements artifacts. The use of prototypes is one known technique to identify incorrectness and aberrations from the stakeholder intentions early. [Problem] Software prototypes of cyber-physical systems' software, however, often need to be adapted to the prototype's hardware, which may differ from the system's hardware. This leads to differences between the system's and the implemented prototype's specification, impeding the applicability of validation results to the system's requirements. [Solution Idea] To support the validation of requirements artifacts for cyber-physical systems, we propose the generation of an explicit prototype specification. Traceability relations aid the requirements engineer in deciding whether a finding actually corresponds to a defect in the requirements or whether it results from changes due to the prototype's different hardware. [Contribution] In this paper, we propose a model-based prototyping approach for validating requirements artifacts of cyber-physical systems. The approach relies on the generation of an explicit prototype specification and on a categorization scheme for validation results. The latter allows determining the impact of the validation results on the system's requirements, and facilitates the correction of errors.
A major challenge in system development is specifying requirements that correctly
reflect underlying stakeholder needs. Relevant stakeholder intentions are often unclear,
ambiguous, or simply unknown, which leads to the development of systems that fail to
fulfill their purposes. The later these problems are discovered, the higher the cost for
fixing them [
To validate existing requirements and elicit further stakeholder intentions by prototyping, it is important to evaluate a system’s behavior and its respective functional interdependencies in a sufficiently realistic environment. However, this is often not feasible in the engineering of cyber-physical systems (CPS) software. Since CPS consist of software and hardware parts, either the hardware needs to be simulated or the software needs to be deployed on some hardware that is different from the hardware that the final system will use (e.g., an electronic control unit of a previous version). Hence, there may be significant differences between the system’s and the prototype’s hardware, which impedes the use of the prototype for requirements validation. The prototype hardware, for example, may not offer a specific sensor that the system needs to gain information about its environment. In this case, the required information must be obtained from other sources, e.g., by using a combination of different sensors or by simulation. This causes a difference between the prototype’s requirements and the requirements for the actual system. Hence, some validation results (e.g., those concerning the missing sensor) are only applicable to the prototype but not to the system itself.
To help determine if a validation result is applicable to the system requirements and, consequently, to discover incorrect and missing requirements, we propose a modelbased approach for deriving a prototype specification from the system specification, as well as for analyzing and transferring the validation results. To this end, we propose a technique and a supporting classification scheme for categorizing validation results, which allow for determining, which validation results are applicable to the system specification. Furthermore, this model-based approach supports the correction of the system specification based on a corrected prototype specification.
This paper is structured as follows: Section 2 introduces our solution concept. Subsequently in Section 3, we demonstrate the solution concept using an avionics collision avoidance system as case example. Section 4 discusses related work and Section 5 concludes the paper. 2
Developing prototypes has proven to be an effective way for validating requirements
(see [
System specification 1. Derive prototype Prototype specification specification
Traceability links
Rationales for changes
Revised system specification 5. Revise system specification 2. Implement prototype
according to the prototype specification Prototype
Validation results
4. Check applicability of validation results 3. Obtain validation results from stakeholders In the following subsections, we elaborate on the main parts of our contribution. First, we will describe how to derive the prototype specification from the system specification and how to document the changes made. Then we elaborate on the validation step of our approach, including the attribute scheme, which provides attributes to categorize validation results. Last, we explain how this categorization helps determining whether a validation result is applicable to the system specification or not. 2.1
Due to limited capabilities of the prototype’s hardware, it is often not feasible to implement the system requirements specification as is and deploy it onto the prototype’s hardware. Instead, a specific prototype specification has to be created to ensure structured prototype development. Our approach relies on the idea to derive this prototype specification from the original system specification. In doing so, changes from the system specification are incorporated in the prototype specification. These changes take the specific capabilities of the prototype’s hardware into account, which most likely differs from the originally intended CPS hardware. For example, in the absence of a specific sensor, the prototype might estimate context measurements.
A crucial task that goes along with these changes is the documentation thereof. As
model-based engineering of cyber-physical systems can be seen as a de-facto standard
[
Our classification of validation results is based on the Orthogonal Defect
Classification scheme (ODC) [
Fig. 2 illustrates the steps involved in checking if a validation result is applicable to the system specification: (For a coherent example please refer to Section 3) First, for each validation result, the prototype specification is checked if the respective functionality is documented correctly therein (Step 4.1 in Fig. 2). Whether a certain functionality is documented correctly can be assessed from its respective attributes (cf. Section 2.2). Two combinations of attributes indicate a correctly documented functionality in the prototype specification: 1. The functionality is necessary or arbitrary, it is correctly implemented w.r.t. the stakeholder intentions, and it is present in the prototype specification 2. The functionality is not necessary and not present in the prototype specification.
This case, however, is unlikely to be relevant as stakeholders are unlikely to identify any non-implemented behavior as unnecessary. If a functionality has been identified as correctly documented in the prototype specification, the documented traceability information (see Section 2.1) is used to identify corresponding parts in the system specification. Analyzing respective trace links facilitates checking whether functionality is correctly specified in the system specification (Step 4.2 in Fig. 2). This check leads to one of the following cases: 4.1 Check if respective functionality is documented correctly (i.e. according to the stakeholder intentions) in the prototype specification
Yes
No Traceability links
Prototype specification
Rationales 4.4 Check if change was necessary
Functionality documented correctly
in system specification? Yes
No Validation
result Functionality documented correctly in the prototype specification? Fuctionality correctly documented in the prototype specification 4.2 Check automatically if the functionality is documented correctly in the system specification (i.e., if the system specification contains corresponding
functionality) Valid functionality identified Case 1. The corresponding parts in the system specification can now also be considered correct with respect to the stakeholder intentions. In this case, a valid functionality has been identified.
Case 2. The system specification contains a defect if the documented traceability information shows no corresponding part in the system specification. For unwanted functionality, which has been identified, corresponding parts in the system specification can be removed automatically and necessary functionality that is missing in the system specification can be added automatically if it is correctly documented in the prototype specification. If a functionality has not been documented correctly in the prototype specification, there could be two reasons for this. Either the prototype and its specification contain a defect, or the prototype and its specification had to be altered for a specific reason. Hence, it is necessary to check if there are reasons for changes (Step 4.4 in Fig. 2). Since all alterations, which were made when deriving the prototype specification, and the reasons for them were documented, it is easy to determine which of the following is the case: Case 3. If a functionality is not documented correctly in the prototype specification and subsequently not implemented as desired in the prototype because the prototype’s hardware is incapable of supporting this functionality, the reason has been documented. Here, automated validation of the system specification is not possible, and the validation has to be done manually, i.e., the correctness of the functionality represented in the system specification needs to be checked manually (Step 4.3 in Fig. 2). Nevertheless, the insights gained from the stakeholders during the demonstration of the prototype may offer valuable support in these cases as well.
Case 4. If there is no reason for a functionality not being documented correctly, a defect in the prototype, the prototype specification, and thus in the corresponding part of the system specification has been discovered. Instead of remedying this defect manually in the system specification and thereby risking introducing new defects, a manual correction of the prototype specification and the prototype (Step 4.5) enables the later validation of the correction in another iteration.
In addition to Fig. 2, Table 3 briefly summarizes these four distinct cases that can be identified in applicability checking. As can be seen, the validation results obtained can also aid in the correction of the system specification if defects have been discovered. In some cases, this correction can even be done automatically.
traceable? Rationale documented?
n/a n/a
(Automated correction applicable) Manual review necessary Revision of prototype and prototype specification necessary 3
In this section, we will discuss the application of our approach to an avionic collision
avoidance system. Therefore, we will provide details on each step of the approach as
described in Section 2 and outlined in Fig. 1. We evaluated the applicability of the
approach using an avionic proactive collision avoidance system (PCAS) as a case
example. The PCAS shall exchange comprehensive flight data (i.e., complete flight
plans), and propagate these flight information among the different aircraft forming a
CPS network. This shall allow airplanes to prevent collisions proactively, i.e., long
before a potential collision may occur. Some more details on the PCAS can be found in
[
Based on the system specification of the collision avoidance system PCAS, we derived an adapted prototype specification for the chosen technology. Subsequently, we implemented and deployed the software prototype to several quadcopters. Hence, multiple prototypes consisting of hardware and software parts were used to demonstrate the behavior of a collision avoidance system in a cyber-physical system network. The demonstration of the prototype’s behavior allowed for validating parts of the system specification and revealed exemplary defects in other parts of the system specification, some of which have been corrected automatically. 3.1
While the collision avoidance system was meant to be deployed in aircraft, the prototype was implemented using a quadcopter. These hardware differences necessitated several adaptions to the system specification. Fig. 3 shows an excerpt of the system specification for the PCAS and the corresponding part from the prototype specification. The differences between them are due to the fact that aircraft send out their ID via their secondary radar to recognize each other, while quadcopters can only recognize other quadcopters via tags using their cameras. For each change, we explicitly documented the traceability information using trace links (cf. Section 2.1).
PCAS
Secondary radar
Foreign aircraft <<replace>> PCAS prototype Primary radar
Own quadcopter We developed the prototype for the collision avoidance system according to the prototype specification, and tested it extensively to ensure correctness of the prototype w.r.t. the prototype specification. Beside the development of the software prototype for the collision avoidance system, it was also necessary to develop further software prototypes for several other avionic systems, which interact with the PCAS. The collision avoidance system relies on information provided by, e.g., the flight management system and the radar. Additionally, the system relies on other avionic systems, such as the flight control system to execute necessary flight maneuvers, for example. Hence, these necessary contextual systems had to be prototypically implemented as well, to ensure proper functioning of the collision avoidance system prototype.
The prototype as well as these other systems were implemented using the OSGI
framework [
Embedded Controller
Proactive
Collision Avoidance System ISecondaryRadar Secondary Radar REST
IAPFD
Autopilot /
Flight Director IFlightControlSystem
Flight Control System
Socket
Flight Management
System
IPrimaryRadar
Primary Radar Since the software could not be deployed on the quadcopters directly, for each quadcopter an instance of the embedded controller ran on an ordinary computer, which sends commands to their respective quadcopter via Wi-Fi. The control center monitors and controls the simulation scenarios. The resulting hardware architecture is depicted in Fig. 5.
AR.Drone 2.0 – 1 AR.Drone 2.0 – 2 AR.Drone 2.0 – 3
„Brutus“ „Cicero“ „Lucius“ caesAR Control Center caesAR Embedded Controller - 1 caesAR Embedded Controller - 2 caesAR Embedded Controller - 3
Wi-Fi „Colosseum“
Message exchange Network connection As can be seen, each quadcopter communicates with its own control logic on its embedded controller via Wi-Fi. These embedded controllers are interconnected and simulate the communication between the different quadcopters. In case of a detection of another quadcopter, the detecting quadcopter informs its embedded controller. Subsequently, the embedded controller negotiates adapted routes with the embedded controller of the detected quadcopter. Finally, both embedded controllers inform their quadcopters about necessary route changes. 3.3
The next step is to obtain, document, and categorize results from the validation activities conducted using the prototype (cf. Section 2.2). An excerpt from the validation results obtained from the prototype’s demonstration is shown in Table 4. For example, the functionality evaluated in row 1 (Aircraft Identification) shows that the prototype is capable of identifying other aircraft (in this case other quadcopters). Hence, this functionality is present in the prototype. Since it is desired for the PCAS to identify aircraft in its vicinity, this functionality was furthermore categorized as necessary. However, as the prototype does not exchange aircraft IDs via the secondary radar, the stakeholders deemed the implementation of this functionality incorrect.
For another example, the validation result concerning the collision avoidance trigger functionality (row 2 in Table 4) was categorized as present, correctly implemented in the prototype, and necessary.
As for the result referring to the functionality of exchanging a Priority Parameter (row 3 in Table 4), it became obvious during the development of the prototype that so far no decision had been made regarding the course adaptation. In particular, it had not been determined which aircraft should have to adapt its course and which could stay its course in case of a collision threat. Hence, the exchange of a parameter, which determines priority and thus right of way, was added to the prototype and its specification. Consequently, this added functionality was categorized as present, correct, and necessary, although it is not present in the original system specification. To see if a validation result is applicable to the system specification, first it needs to be checked if the respective functionality is documented correctly in the prototype specification (see Section 2.3). If the functionality is documented incorrectly therein, it further needs to be checked whether there is a reason that prevents the correct implementation of the respective functionality in the prototype.
As can be seen in Table 4, the functionality for validation result 1 (Aircraft Identification) is not documented correctly in the prototype specification. However, as this functionality is not the same as in the system specification (cf. Fig. 3), but has been replaced due to hardware constraints, this does not indicate a defect in the system specification. Instead, this validation result is not applicable to the system specification and this functionality has to be validated manually (cf. Case 3 in Table 3).
The functionality for validation result 2 (Collision Avoidance Trigger) is documented correctly in the prototype specification. As there is no documented trace information that would indicate a difference between this functionality in the prototype specification and in the system specification, the documentation of this functionality in the system specification can be validated automatically (cf. Case 1 in Table 3).
The functionality for validation result 3 (Priority Parameter) is documented
correctly as well. However, as previously described, this functionality has been added to
the prototype specification and does not exist in the system specification. Since it
corresponds to a functionality that was found to be a valuable extension to the original
requirements, a defect in the system specification has been discovered here (cf. Case 2
in Table 3).
The automated correction of the system specification is illustrated in Fig. 6 using the
missing exchange of a priority parameter defect. To correct this missing functionality,
the model elements referring to this functionality (highlighted in Fig. 6), which were
documented in the prototype specification, are added to the system specification. Based
on a correct prototype specification, this correction can be made automatically using
model transformation techniques (cf. e.g. [
Since early validation of requirements is commonly considered of vital importance to
avoid defective software and higher costs later on [
Priority parameter ...
...
...
...
Foreign air route Autopilot
PCAS
and the intended laws and regulations are considered. As automated verification is
usually not applicable in early phases, attention is paid to techniques such as inspections
[
Rapid prototyping is often suggested for early validation of first drafts and ideas as
well as for eliciting new requirements (cf. [
In general, prototyping aids in the validation of requirements as it enables
stakeholders to experience the system’s look and feel and its behavior. Thus, it not only allows
stakeholders to express their opinions about existing requirements but also to identify
missing requirements. To support the systematic evaluation of the prototype,
pre-determined usage scenarios can be executed for prototype demonstration (cf. [
Although prototyping is a well-known approach for validating requirements [
In this paper, we presented a model-based approach to aid validation of requirements artifacts for cyber-physical systems. As shown, validation results for the prototype specification can aid in the identification of defects in the system specification. The extent to which validation results are applicable to the system specification depends on how many changes need to be made to the system specification when deriving the prototype specification due to differences in the technology used.
We evaluated our approach using a collision avoidance system from the avionics domain. For such safety-critical systems a prototypical validation approach is not sufficient on its own. In particular, it is still of utmost importance to validate the system itself and not only a prototype of the system. Nevertheless, the prototypical evaluation can aid the validation and the exploration of solution options in early phases of development. The approach is of particular interest for those domains (like the embedded domain) which have a strong interest in early validation to decrease the number development cycles for highly complex software.
Beyond the validation of functional requirements, the prototypical implementation provides additional benefits for the development. For example, the prototypical implementation provided further insights about aspects that are usually not specified in requirements specifications but are important for the system’s development. For instance, in the presented case example, the decision needed to be made as to what constitutes a collision threat. For quadcopters as well as aircraft, collisions can occur even if their flight paths never cross, but they only get too close to each other. Therefore, it was not sufficient to check for crossing flight paths and determine when each aircraft will reach this intersection. A reasonable safety margin around each aircraft also needed to be taken into account.
Thanks to Stefan Beck, Arnaud Boyer, and Jürgen Meilinger from Airbus for their support in application of the approach to an industry example. The research serving as basis for this paper was funded by the German Federal Ministry of Education and Research (support code: 01IS12005C).