[Context and motivation] For the past several years, Cyber Physical Systems (CPS) have emerged as a new system type like embedded systems or information systems. CPS are highly context-dependent, observe the world through sensors, act upon it through actuators, and communicate with one another through powerful networks. It has been widely argued that these properties pose new challenges for the development process. [Question/problem] Yet, how these CPS properties impact the development process has thus far been subject to conjecture. An investigation of a development process from a cyber physical perspective has thus far not been undertaken. [Principal ideas/results] In this paper, we conduct initial steps into such an investigation. We present a case study involving the example of a software simulator of an airborne traffic collision avoidance system. [Contribution] The goal of the case study is to investigate which of the challenges from the literature impact the development process of CPS the most.
Cyber Physical Systems (CPS) blur the line between traditional system types. On the
one hand, CPS observe the environment through sensors and act upon it like
Embedded Systems [
Consider the home health network example in [
In this example, the individual CPS that make up the CPS network each have a
specific role as part of the home health network. However, as outlined in [
These dynamic and heterogeneous properties of CPS networks must be accounted
for during the development of any individual system that becomes part of a CPS
network. This means that the development process of traditional systems, such as
adaptive cruise control systems, or blood glucose meters, must take on a cyber physical
perspective. In the past several years, existing approaches have been applied or
extended and novel approaches have been proposed to aid a cyber physical development
process. Such approaches encompass many areas of software engineering (e.g.,
security engineering, [
In this paper, we take initial steps to investigate the challenges faced during software development of Cyber Physical Systems. We present a case study of the development of a simulator of an airborne Traffic Collision Avoidance System (TCAS). This paper is structured as follows: The following Section 2 reviews the related work on challenges in the development of CPS and outlines CPS properties that are subject of investigation in the case study. Section 3 gives a brief overview over the TCAS systems and motivates why it is a suitable example of a CPS. Section 4 discusses the case study design and the research questions that are based on the challenges from Section 2. Section 5 discusses the results of the case study with regard to the research questions from Section 4 before Section 6 concludes the paper. 2
Several authors have defined overlapping properties of CPS. Some authors place
emphasis on the connected nature of CPS or focus on the cloud-like aspect of their
networks (like Systems of Systems, see e.g., [
Homogeneous Heterogeneous
In Table 1, the term “homogeneous” implies that CPS have the same externally
observable features, but does not imply that all CPS are the same type. Instead,
individual CPS can be developed by a different vendor, etc. All four categories have in
common that the CPS within the network collaboratively achieve a common goal. In
contrast to Multi-Agent Systems [
In the past several years, a plethora of papers have been published discussing these
challenges (e.g., [
Deployment and Main- Deploying large CPS networks is burdened by the [
Validation/Verification Runtime adaptation of CPS network topology and of CPS of Behavior network behavior must be validated and verified with regard to the operational purpose of each CPS, the common goal of the CPS network, and human intentions.
Failure Robustness, Sec- Not only must each CPS be robust to failures and not [
The purpose of this paper is to investigate the development process of an individual CPS, which becomes part of a CPS network during operation. In this section, we describe the case example, which was the artifact to be developed in the case study, and we discuss the rationale for selecting the TCAS as the case example. 3.1
The case example is a Traffic Collision Avoidance System (TCAS). TCAS is an
avionics system designed to prevent mid-air collisions between aircraft in flight. To do
so, TCAS detects other aircraft (i.e., “traffic”) within a certain distance in front of the
own aircraft, alerts the pilots through “traffic advisories”, and produces “resolution
advisories.” Resolution advisories are recommended changes in the vertical speed
(i.e. climb or descend rate) in order to increase the separation altitude between the
own aircraft and the threat aircraft such that a collision can be avoided. Fig. 1 shows
TCAS’ simplified functionality based on [
As can be seen, a TCAS monitors the airspace around the aircraft for traffic equipped with a corresponding active transponder (“Mode-S Interrogation” in Fig. 1). This allows establishing two-way communication between the own aircraft and traffic nearby to coordinate collision avoidance. If some other aircraft in the vicinity is not equipped with an active transponder (e.g., many recreational aircraft), or no transponder at all (e.g., ultralight or hostile aircraft), the TCAS uses the own aircraft’s transponder and onboard radar to detect uncooperative traffic (“Mode-C-Only All Call”). The function “target surveillance” determines the flight trajectories for all traffic detected through radar contacts and transponder replies and discriminates traffic into “intruders,” i.e. traffic that might pose a collision threat, and other traffic.
For each intruder, and under consideration of the own aircraft’s flight path, the function “Comparison with Own A/C trajectory” computes the relative approach vector to the intruder, the distance to the intruder (“Range” in Fig. 1), the intruder’s relative altitude (“Alt”) and the time until the intruder is intercepted (“Tau”).
Based on this information, a traffic advisory (“TA”) is issued. Concurrently, the TCAS computes the closest point of approach (“CPA”), i.e. the point in threedimensional space, where the own aircraft is closest to the intruder. If it is determined that the CPA is smaller than a minimal safe separation altitude, the intruder is considered a collision “threat” and a resolution advisory (“RA”) is produced through auditory alerts and cockpit instruments. 3.2
In Section 2, we have listed challenges previous authors have assumed to burden the development of static and dynamic, homogeneous and heterogeneous CPS networks and in Section 3.1, we have introduced the case example. We have selected the implementation of a TCAS simulator because, depending on the specific interactions between the TCAS of two or more aircraft, the TCAS can participate in all four CPS network types from Section 2: • Homogeneous vs. Heterogeneous: A CPS network involving multiple TCAS instances is homogeneous, if the TCAS instances are cooperative in the sense outlined in Section 3.1, i.e. using Mode-S transponder functionality actively work together to avoid collisions. A CPS network involving multiple TCAS instances is heterogeneous, if at least one intruder aircraft is uncooperative, e.g., when the intruder aircraft is not equipped with TCAS functionality or a hostile aircraft. • Static vs. Dynamic: A CPS network involving multiple TCAS instances is static, if there is a known set of aircraft, either cooperative or uncooperative, in the vicinity of the own aircraft and their altitudes must be coordinated such that no aircrafts collide. In reality, this is an atypical case. A CPS network involving multiple TCAS instances is dynamic, if aircraft spontaneously enter each other’s air space during flight, resulting in the need to increase separation altitude. This is the typical use case of TCAS.
It must be noted, that even when one intruder aircraft is uncooperative in the sense outlined in Section 3.1, other TCAS in the CPS network can still react appropriately such that the common goal of avoiding a collision is achieved collaboratively (unless an intruder is actively attempting a collision). It follows that the TCAS example can partake in the different CPS networks outlined in Section 2. TCAS is therefore a suitable surrogate for investigations into the development process of CPS participating in such networks. 4
The purpose of the case study is to investigate the development process of a TCAS,
which will form CPS networks with other TCAS during operation. In this section, we
describe the design of the case study following the checklist and guidelines from [
The case study took place as part of a software engineering seminar course at the authors’ affiliation, instructed in fall 2016 by the third author. The course is geared towards advanced students with considerable knowledge in development. The course objective is for students to apply their theoretical knowledge about software development in a significant development project over the course of one semester.
Participants. There was a total of eight students enrolled in the course. Three of the students participated in the TCAS case study, while the remaining students worked on other course projects. All three participants were enrolled in an undergraduate computer science or software engineering program. As for the course prerequisite, participants had demonstrable prior experience in software development, yet no experience with CPS development. Age and gender were assumed to not impact development ability and hence were not recorded.
Requirements Phase. The course was structured over 15 weeks with three class
meetings every week. Approximately half of the semester (seven out of 15 weeks)
was dedicated to requirements engineering and system design. In this phase, the first
class meeting each week was dedicated to lecturing on model-based requirements
artifacts. Specifically, lectures included context diagrams, KAOS goal models, and
UML sequence diagrams for scenario modeling (see [
Implementation Phase. After the requirements phase completed, the case example
implementation began. Implementation followed the agile SCRUM methodology, as
given the limited timeframe of a semester, agile implementation approaches allow for
quick progress in producing executable artifacts. At the onset of this phase, students
were introduced to agile development and asked to derive a backlog from their
requirements artifacts, which were developed a more rigid approach (i.e. [
The implementation phase concluded at the end of the semester with a graded final presentation. Participants’ course grades were established by means of a departmental criteria list, which is solely based on the students’ individual performance as a team member and their application of knowledge from previous courses. Participants’ course grades were hence independent of project success or case study outcome. Approval from the authors’ university’s ethical review board was sought before students agreed to participate in the case study by providing informed consent and usage rights for their artifacts and code. 4.2
The key goal is to gain initial insights into the specific challenges that arise when developing a system from a cyber physical perspective, as outlined in Section 1. To do so, the following research questions were defined: • RQ1: How did students execute the development process in the context of the course instruction? Approaching system development from a cyber physical perspective was not a main focus of the curriculum in the students’ degree program and may have influenced the results. • RQ2: Which challenges from the literature are the most relevant? As was illustrated in Section 2, in the literature, a large amount of hypothetical challenges for the development process of CPS have been suggested. These challenges pertain to the dynamic allocation of CPS to homogeneous or heterogeneous networks. Yet, which of these hypothetical challenges impact development is to be explored. • RQ3: Which development discipline is impacted the most by these challenges? Each of the challenges discussed in Section 2 may affect some development discipline more than others. For example, challenges in requirements modeling (Challenge 5 in Table 2) may impact the requirements engineering phase of development, but also implementation. For approaches to be able to address these challenges, the impact of these challenges must be known. 4.3
The TCAS case example from Section 3.1 is intended to be implemented as a plugin
for the X-Plane flight simulator [
There were three types of artifacts that were collected and analyzed: requirements artifacts milestones from the requirements phase, implemented executable aritfacts (e.g., code, test cases, configurations), and reports from the daily SCRUM meetings.
Requirements Artifact Milestones. The four milestones during the requirements phase collected through an online campus system every second week. Since a milestone consisted of revising previous versions of artifacts based on feedback and knowledge discovery, milestones were incremental. Moreover, participants were asked at the end of the implementation phase to produce a fifth milestone of their requirements artifacts to account for the changes to the artifacts necessitated during the implementation phase. During data analysis, milestones were compared to one another with regard to revisions made due to knowledge discovery, i.e. revisions to one artifact that became necessary due to progress made while working on another artifact. This means that changes due to instructor feedback were not recorded. In particular, changes due to the cyber physical properties of TCAS were recorded and categorized according to each research question (see Section 4.1).
Implementation Code. All source-code-based artifacts, which included code, third-party libraries, image artwork for TCAS instruments, configuration files, etc., were collected incrementally throughout the implementation phase using a remote repository. Participants were asked to commit their changes to these artifacts at least once at the end of each sprint. During data analysis, changes to code and code quality were not taken into account. However, progress during implementation was tracked with regard to the specific aspect that was worked on during a sprint. In particular, we took note on whether or not a particular implementation aspect pertained to traditional system properties or cyber physical properties of the TCAS. For example, when issues arose regarding networking, interface rendering, or platform-plugin interaction, this was considered an implementation issue pertaining to either traditional system properties or pertaining to the simulation platform. On the other hand, when an issue arose pertaining to establishing a connection between the own aircraft and other aircraft and subsequently discriminating other aircraft into threats, intruders, or other nearby traffic, this was considered an issue pertaining to CPS.
Participant Reports. During the daily SCRUM meetings, participants reported their progress in the previous sprint, and reported on current issues they were working on. Albeit these reports are subjective and qualitative, they became the richest source of insight into the development process, as they gave an “uncensored” account on the specific challenges students encountered during development. We use the term “uncensored” to address the notion that participants were eager to remind us of the level of difficulty and challenges they experienced with the TCAS case example system. Similarly, when a challenge was resolved, participants keenly reported the specific strategy which lead to success. For the purpose of the case study, we took particular note of challenges pertaining to the interaction of two or more TCAS systems during operation, as these reflect the cyber physical nature of TCAS. 4.5
Albeit great care was taken to design and conduct the case study, threats to validity remain. In the following, we discuss these threats, their mitigation, and open threats.
research questions. We have applied the best practices outlined in [
alizability of results. In Section 3, we have explained the case example system in sufficient detail to allow replication in larger contexts. Moreover, we have outlined in Section 3.2, why the case example is a suitable surrogate for a CPS. Nevertheless, the case study did not develop a “real” TCAS, but was concerned with developing a simulator, leaving open questions as to the level of realism of the simulated plugin. Albeit the development process aimed at developing realistic TCAS functionality based on official regulatory documents, this threat partially remains.
Construct Validity: Measurements must be appropriate to measure results. As with many case studies, the study at hand basis its findings on qualitative observations, participant reports, and artifacts created by participants. These information sources are inherently subjective. To mitigate this threat, we resort to a mere objective report on the development process in Sections 5.1 and 5.3. Moreover, to increase confidence in our work, we make available all case study materials to researchers and enthusiasts (see footnote in Section 4.5).
bias is the fact that one of the authors was the instructor of the course in which the case study took place. This may have influenced case study success and participant performance. Of course, it is the third author’s key interest to see his students succeed. Yet, by university policy, course grades do not depend on project success. Moreover, findings are objective reports regarding the development process, not student performance. Nevertheless, individual student performance may have impacted case study results. Lastly, the process of summarizing CPS challenges in Section 2 may have influenced conclusions regarding the impact of the cyber physical nature onto development. Albeit we defined a clear scope in Section 2 to investigate the challenges, more investigations into the development process of CPS are necessary to confirm our results. 5
In the following subsections, we report on the findings of the case study with regard to the research questions from Section 4.1. 5.1
The case study began in the first week of the semester with participants researching
the basic functionality of TCAS by studying FAA regulatory documents (e.g., [
As can be seen, the X-Plane platform simulates all aircraft. These can be either the user’s own aircraft, or potential intruders. Aircraft Data for non-user aircraft are collected through a X-Plane, through the Multiplayer Network connection. The Multiplayer Network connection represents aircraft in the simulated world. This component 1 An example of such a resource can be found on the StackExchange post by user “DeltaLima” (available at https://goo.gl/fnXOhI, accessed January 3, 2017). Albeit StackExchange is neither a reliable academic nor an official technical reference, it provided sufficient technical details for the participants to move forward with the case study. was already existing and not part of the case study. Aircraft Data is exchanged using X-Plane’s internal Data Bus, which is a collection of variables that store values pertaining to the current simulation. The TCAS is designed as a plugin for X-Plane, which makes use of the Data Bus and queries Aircraft Data such as, vertical speed, airspeed, or latitudinal and longitudinal positions. From the perspective of the user’s own aircraft, it cannot be known whether or not the intruder is cooperative or not. Therefore, only the user aircraft is composed of a TCAS in Fig. 2. A Local Area Network (LAN) connection is used by the TCAS to simulate transponder and radar sweep (i.e. Mode-S and Mode-C calls in Fig. 1), in order to find the subset of simulated aircraft close to the user’s own aircraft within the simulated world. This subset constitutes nearby traffic (see Section 3.1).
Based on the context model and TCAS’ principle functionality outlined in regulatory documents, a goal model was developed. An excerpt is shown in Fig. 3.
Participants began with the anti-goal “Collide With Intruding Aircraft,” which is to be
avoided. Following the KAOS tutorial from [
Once goal models were complete, scenario models were constructed. Fig. 4 shows
two example scenarios.
As shown in Fig. 4, scenarios were created depicting the conceptual interaction
between TCAS components. Following the approach outlined in [
In total, participants produced more than 20 goal, scenario, and context diagrams, which were iterated and refined weekly and more than 5,000 lines of code. Due to space limitations, these artifacts cannot be fully discussed here. Therefore, we are making these artifacts available2 to interested researchers and enthusiasts.
The case study was predominantly impacted by challenges pertaining to the interaction between the user’s own aircraft and intruder aircraft. In particular, a considerable challenge was to differentiate simulated aircraft close to the user’s aircraft from those that are far away, e.g., on another continent, but within the simulated world. This challenge arose due to the specific architecture of the simulated aircraft: The Multiplayer Network connection does not discriminate between close and far away aircraft, making it necessary for the TCAS simulator to make this differentiation before regular TCAS functionality can be carried out. Real-life TCAS identify nearby aircraft through the transponder and radar, making this challenge unique to the case study. Moreover, once nearby traffic was identified, only cooperative intruders could be reliably identified (i.e. when they actively send a Mode-S reply, see Fig. 1). For uncooperative intruders, it can only be heuristically established through Mode-C replies and the absence of a Mode-S reply. 2 https://goo.gl/kjvyw
This introduced uncertainty (Challenge 1 in Table 2) considering the specific
makeup of the CPS network: in previous literature, it was assumed that CPS are
designed to either be part of a homogeneous or heterogeneous network. Yet, as we
learned through the case study, the network makeup in this sense cannot be reliably
established and may change at runtime. Especially the fact that the CPS network
makeup changes during runtime requires the TCAS to react appropriately,
necessitating predictable behavioral adaptation (Challenge 2). For example, when the own
aircraft is to avoid a non-cooperative intruder, a decision must be made to climb or
descend to increase separation altitude. In the case study, a Decider component was
created, which issues “climb” or “descend” RAs depending (based on heuristics from
[
The development process largely resembled that of an embedded system. However, uncertainty and behavioral adaptation (Challenges 1 and 2) mainly impacted requirements engineering phases of development (Challenges 4 and 5) as well as the quality assurance (Challenges 7 and 8), while the implementation phase was nearly equivalent to previous projects from the embedded systems domain. Specifically, uncertainty about the specific type of the CPS network constituents (i.e. homogeneous or heterogeneous CPS) required frequent revisions of the model-based requirements artifacts (Challenge 5). For example, frequent changes to scenario models necessitated by different interactions motivated changes in the goals. It is interesting to note that changes in requirements models were preceded by frequent consultations of the context model (i.e. Fig. 2). While the context model was only rarely adapted, the information documented therein lead to fruitful discussions about the interaction between the user’s own aircraft and a possible intruder, which guided the implementation.
The interaction of several cyber physical TCAS impacted deployment (Challenge
6), verification (Challenge 7), and functional robustness (Challenge 8) also. After unit
tests were complete, verification included testing the TCAS functionality using two
computers, each running an X-Plane instance, to which the compiled TCAS plugin
was deployed. Initially, participants did not account for testing TCAS functionality
using different versions (e.g., the latest build together with a TCAS that is a few
revisions old). Instead, when a new version was compiled, the new revision was deployed
to both test computers. During later development stages, when frequent bug fixes and
redeployment took place, this process was streamlined to deploying merely onto one
computer. This introduced a kind of CPS heterogeneousness into the CPS network
that was discussed in [
In this paper, we presented a case study into the development process of homogeneous and heterogeneous, dynamic Cyber Physical System networks. The case example system of an airborne Traffic Collision Avoidance System was chosen to investigate which challenges previously suggested in the literature impact the software development process of a CPS. Case study results show that the most significant challenges pertain to runtime uncertainty of CPS network configurations, predictable behavioral adaptation, and human-in-the-loop control. The most significantly impacted development disciplines are requirements engineering, quality assurance, and deployment.
The case study at hand provides merely an initial, exemplary insight into the development process of a system from a cyber physical perspective. It is not as a definitive investigation into the challenges underlying CPS development in general. Additional empirical investigations are needed to investigate the challenges of CPS development in more detail and with more robust evidence. This is subject of future work.
We encourage repetition and extension of our research. Therefore, we make available the current implementation of the TCAS simulator along with its development artifacts (see Section 5). The current status of the TCAS simulator handles arbitrary TAs and RAs for the closest threat.
Acknowledgments. The authors thank Laminar Research for making available their X-Plane simulator platform, their industry partners for their consultation regarding TCAS, and our participants. The project is sponsored in part by the Research Fund for Chemistry and Math/Science.