Self-adaptive systems need to be designed with respect to threats within their operating conditions. Identifying such threats during the design phase can bene t from the involvement of stakeholders. Using a system model, the stakeholders, who may neither be IT experts nor security experts, can identify threats as a rst step towards formulating security requirements. To support it, the modeling language might possess adequate features to support this task. This paper investigates how iconic signs as a feature of an informal modeling language can contribute to eliciting security requirements by non-experts. Taking urban grid as a case, we relate bene ts and speci cs of using iconic signs to the two modeling challenges: i) reducing the cognitive complexity required to understand and model a system by non-experts, and ii) facilitating the threat identi cation activity using a system model. Outputs of three experiments suggest that iconic signs do assists in addressing the challenges.
Modern cyber-physical systems, such as smart grids, should account for the
context in which they operate to ensure the continuous service delivery. In principle,
designing complex systems demands that multiple stakeholders are directly
involved [
Identifying possible misuse cases of a system { as a step toward threat
modeling and then formulating security requirements { leads to earlier focus on
Copyright 2017 for this paper by its authors. Copying permitted for private and
academic purposes.
security. In case of self-adapting systems, a list of identi ed threats can also be
later used to consider later how the system should react to speci c threats. For
instance, adaptation patterns (see, e.g., [
This paper follows the paradigm that requirements engineering starts with
problem identi cation and needs input from stakeholders. The involvement of
stakeholders' encourages their creativity and invites them into discussion even
if they lack signi cant technical expertise [
System models can aid in the communication between stakeholders and
system architectures. It can hardly be expected that stakeholders, who are
concerned with proper functioning of complex adaptive systems, possess signi cant
expertise in modeling (e.g., DFD) or using security-related approaches (e.g.,
UMLsec). Importantly, modeling notations "should be palatable to the users"
[
Even though a number of researchers concentrated on visual notations (see,
e.g., a seminal work [
We study how iconicity [
For the purpose of this research, we take an urban electricity grid as an example of an adaptive cyber-physical system. The grid model represents city-level grid components (e.g., a power substation, hospital) and connections between them. Such a model is similar to a UML deployment diagram, and consists of: i) nodes as modelling elements that represent the system components and ii) links among the nodes.
After reviewing relevant background in the next section, the paper introduces the methodology used, presents results of three experiments, discusses them, and concludes with future work. 2
Often, di erent stakeholders should collaborate to ensure that a system will deliver desired services. This is particularly relevant for Critical Infrastructures (CIs). Typically, CIs operate in complex social, economic, and technical contexts that imply collaborations between a number of stakeholders. The focus of risk management in CI concerns threats to safety and security. For comparison, risk management in the insurance, engineering, and nance domains aims at protecting against nancial losses. All these aspects justify considering an urban electricity grid, which is a speci c CI, as a particular relevant case for studying the security requirements elicitation process.
Being complex systems, urban electricity grids need to re-act to changes in their environment. In normal operation mode a number of elements are highly interconnected. At the same time, grid should be ready to adapt to for rapid changes in real-time. For instance, a part of the grid (including both power and ICT systems) might be able to became an autonomously operating island (or a microgrid) to prevent cascading failures. Proper identifying and modeling of threats is critical to devise adequate security mechanisms.
Numerous grid stakeholders possess speci c (tacit) knowledge relevant to
ensuring proper functioning of the grid. City managers might have signi cant
expertise in daily administrative operations. They might ensure how renewables
are related to renewable energy-related landscape features and to the reduction
of greenhouse gas emission [
Several solutions exist to assist stakeholders in identifying risks and threats to the grid, as shown in the next subsection. However, it is unclear how representations of a modeling language can assist in modeling a system (MC1 ) and identifying threats to it (MC2 ). 2.1
The need for adequate risk management when considering CI is highlighted
by government agencies [
Often, CI-focused risk assessment methodologies are built around three core
tasks that start with threat identi cation: i) identi cation and classi cation of
threats, ii) identi cation of vulnerabilities and iii) evaluation of impact. For
instance, Risk Assessment (RA) for CIs, according to U.S. Department for
Homeland Security, should always start with obtaining a clear and agreed view of the
infrastructure from all public and private partners. This shared view is used
to assess the relevant risks, in terms of threats, vulnerabilities, and impacts[
Several tools were developed to support critical infrastructure analysis.
Primarily, they address the policy maker point of view. An example is the CARVER
tool (Criticality Accessibility Recoverability Vulnerability Espyability
Redundancy) [
It is desirable that (informal) modeling languages can contribute to reducing the cognitive complexity of a modeling task (MC1 ) and identifying threats to a system (MC2 ). Together, the two challenges concern investigating utility and usability of di erent representation of a grid modeling language for the threat identi cation task.
Cognitive theories support the argument that a less suitable format of the
model representation (the signs used) can hamper understanding of non-experts.
These theories particularity stress the importance of having an intuitive and
understandable representation of concepts. Cognitive t theory, for instance,
suggests that the cognitive load is reduced when a representation matches the
problem [
This paper describes a study on the basic di erentiation between using iconic and symbolic signs. Iconic signs visually resemble the concepts that they represent, whereas symbolic signs are arbitrary and the relationship with the concept they represent is purely conventional. We investigate the e ect of utilizing either symbolic or iconic signs for the modelling and threat identi cation tasks given the same list of modelling constructs. Noticeably, we go beyond a simple consideration that iconic signs are bene cial for understanding concepts, as we study the role of signs within a modelling language to identify threats to a system. The experiment design, devised for this paper, accounts for these challenges, as shown next. 3
To study how iconicity in uences modeling and threat identi cation, we designed and conducted three experiments. These experiments are interrelated as shown in Fig. 1. The combination of three experiments covered di erent combinations of possible usage of the iconic modeling language. The focus of the second and the third experiments concerned modeling a system and identifying threats to a system accordingly. Together, the system of experiments dealt with both modeling and threat identi cation steps. With respect to evaluation criteria, we studied perception of users and compared amounts of threats identi ed by di erent groups.
The participants were provided with either iconic or textual grid elements for experiments 1 and 2. Constructed grid models (either iconic or textual ones) together with a generic threat list formed the input to experiment 3. The conguration of experiments formed a structure that assessed the utility of iconic models, in uence of iconicity to model the grid, and its role in identifying threats. As real experts were unavailable, we used a sample of BSc, MSc and PhD students. Experiments 1 and 2 were conducted at University of Twente (UT) during the CuriousU summer school. Later, Experiment 3 took place at University of Florence (UNIFI). This section describes the set-up in detail.
The experiments tackled challenges MC1 and MC2 as shown in Table 1. Also, the table describes sample populations, modeling tartgets, and treatments of the three experiments.
The intended target of generalization of the three experiments are city-level stakeholders with no speci c experience on modeling and threat identi cation. While students are not representative city planners (and we acknowledge that it somewhat weakens evaluation e orts), the outcomes of the experiments are produced by general cognitive mechanisms (such as cognitive t) which are shared by both groups. Furthermore, both students and city planners are unlikely to possess knowledge or experience with regard to critical infrastructure modelling tools or threat identi cation techniques. We did not control for pre-existing knowledge of the students, but to enhance external validity, participants were rstly introduced to typical infrastructure components of grids simulating the basic knowledge that city planners might have.
Participants, supplied with speci c materials, joined one of the three experiments. Each experiment consisted of a task performed by two groups. Members of each group worked collectively on their task. After the completion of task, the participants lled-out their questionnaires individually. While one can debate whether the individual questionnaire responses are fully independent, each task implied the idea of collaborative work. In this connection, we were interested if a speci c representation (possibly, by supporting interactions) in uenced perceptions of individuals after they completed a collaborative task.
Due to self-forming, the group sizes were not equal. Still, the amount of people in each of the groups (within a single experiment) did not deviate largely (see Table 1). The di erence can potentially be seen as a threat. However, as the experiments involved participants of a summer school and visiting students, they (to our best knowledge) did not have signi cant previous experience of working with each other. Thus, we did not limit the way how the groups were formed. While we didn't collect group pro les, participants within each experiment shared a similar level of education. In each of the three experiments, the treatments were randomly allocated to the groups involved.
Experiment 1 focused on whether modeling a grid (MC1 ) and identifying threats (MC2 ) can be performed within a comparable time interval by using iconic or symbolic modeling constructs. For this, the groups were tasked to construct a model how they imagine the campus grid of the UT in 5{10 years. Potential threats to the validity of this experiment that we could not control are pre-existing security or safety knowledge and experimenter expectancy (as the exercise was supervised). However, we attempted to ensure treatment and measurement validity by running the two sessions in parallel provided each group with the same tools (MS Visio) and instructions (handouts). Two supervisors involved in the experiment were allowed only to answer questions strictly related to the threat lists.
Experiment 2 concerned only with the modeling task and did not cover the
threat identi cation step. It investigated whether iconicity of the modeling
language would in uence modeling changes in the system and understandability
(MC1 ). After performing the task, the participants lled in a questionnaire
formed by 4 questions to document their perception of di culty and success
of the modeling task. Questionnaire design asked for a score from 1 to 5 to each
question following a psychometric semantic di erential scale to reduce
acquiescence bias [
Rate from 1 (Very easy) to 5 (Very Di cult); { E2Q2. "How satis ed are you with the tools provided to complete the task?
Rate from 1 (Not Satis ed) to 5 (Very Satis ed); { E2Q3. "How would you rate the amount of time it took to complete the task? Rate from 1 (Very little time) to 5 (Too long); { E2Q4. "How much do you agree with the nal version of the model? Rate from 1 (Don't agree) to 5 (Fully agree).
This second experiment was conducted under stricter conditions: supervisors were not allowed to assist the modelers. Participants answered printed questionnaires immediately after the task. However, group dynamics could have in uenced the measurement validity. For instance, one can assume that some participants might have reported lower agreement or perceived the task as more di cult due to intra-group personality or skill mismatches. The experiments did not investigated either of these aspects. Nevertheless, as the groups were formed from a pool of participants with similar education experiences, we expect that in uences of these aspects were limited. Another threat to validity to the second (as well as to the rst) experiment is that both groups worked in a single, although very large room. To counter it two supervisors tried to limit cross-group interaction.
Experiment 3 explicitly dealt with identifying threats to a grid. It
concentrated on how participants relate an iconic or symbolic grid model to a generic
threat list. It was designed to understand how the iconicity feature of a model
in uences the ability of non-experts to perform an e ective { complete, precise,
and accurate { threat identi cation task (MC2 ). After de ning two groups of 3
students at UNIFI, we asked the participants to identify all the possible threat
occurrences of a given modeled scenario considering a reference threat list [
The obtained threat lists were compared with a list provided by an expert from UNIFI to assess the completeness of students' lists. Also, the participants lled in the following questionnaire: { E3Q1. "How would you describe the di culty of building the list of threats?
Rate from 1 (Very easy) to 5 (Very Di cult); { E3Q2. "Was the graphical/symbolic description enough to complete the task? Rate from: 1 (Unnecessary) to 5 (Very Useful); { E3Q3. "Did you feel that additional software supports were needed? Rate from: 1 (No) to 5 (Yes, I was lost); { E3Q4. "How would you rate the amount of time it took to complete the task? Rate from: 1 (Very little time) to 5 (Too long); { E3Q5. "Do you feel that the list you provided is complete? Rate from 1 (Very poor list) to 5 (Very complete list).
Afterwards, we asked the students to anticipate how the threat identi cation exercise would be if they would have the model described in another way (iconic for Group 2 and symbolic for Group 1 ). By doing so, we aimed to collect perceived bene ts of using alternative description of a scenario. The threat identi cation exercise was not repeated. To di erentiate between the two questionnaires lled out, the rest of the paper refers to the "perceived bene ts" questionnaire as Experiment 3b, while the initial survey is referred to as Experiment 3a. 4
Input to the rst and the second experiments included lists of i) generic threats to grid components and ii) either an iconic or a symbolic list of grid components to build an urban grid. The latter input was organized as a template in a MS Visio le. In the third experiment, the students were supplied with a list of generic threats and with either an iconic or symbolic model. The provided model was similar in complexity to those obtained during the rst two exercises.
Iconic modeling constructs are described in [
This experiment aimed to consider the utility of the provided language to model
the grid and identify threats to it. The main task was to create grid models
(see, e.g., Fig. 2). Also, we asked participants to identify threats relevant to
particular steps of the grid development (using a generic list of possible threats,
as described in [
Main Findings. An interesting nding of this experiment was that the iconic group decided to proceed with modeling the grid in MS Visio directly, while another group started to draft their plans on a whiteboard and paper sheets. We did not anticipate that groups would utilize alternative media when confronted with non-iconic notations. An explanation could be that in this case a lack of iconicity eliminated perceived bene ts of using a software-modelling tool, while the exibility a orded by free-hand drawing led to the use of whiteboard. This potentially points out that the notation of a modelling language can directly impact the modelling process. Both groups were capable to construct grid models and identify a comparable number of relevant threats, despite their previous lack of experience with this task. It suggests that the both representations, as well as the language, can be used for relating components to threat sources. 4.2
This experiment concentrated on obtaining initial quantitative data whether modeling using software tools with iconic signs is perceived by non-experts as more understandable compared to modeling with non-iconic signs. Similar to Experiment 1, two groups of ten students each were asked to construct models of a smart future university campus. Afterwards, we collected four questionnaires from the group that used iconic signs (Group 1 ) and seven questionnaires from the other group (Group 2 ).
Main Findings. Table 2 describes the collected data. The members of Group 2 found the task more di cult (by 64%) and were less satis ed with the tool to model the infrastructure (24%). The E2Q1 answers from the two groups di er signi cantly and their con dence intervals do not overlap. It highlights di culties that the students from Group 2 encountered during modeling the future grid. The replies to E2Q3 and E2Q4 are less illustrative: while being comparable, they deviate largely.
The outcome of this experiment suggests that software-based modelling with iconic signs is perceived as less di cult than when using symbolic signs. 4.3
The last experiment focused on investigating how an iconic/non-iconic model
in uences the outcomes of the threat identi cation task. Two groups each of 3
students participated in the experiment: Group 1 worked with an iconic
description of the grid of the scienti c complex of UNIFI, while Group 2 worked with
a non-iconic (symbolic) version. Provided with a list of generic threats (a subset
of threats 7, 10, 17, 18, 19, 21, 24, 29, 31, 37 of the threat list in Appendix B
of [
Main Findings. Table 3 shows that the amount of valid identi ed threats is signi cantly higher for participants who were supplied with the iconic model. Group 1 members identi ed 17, 10, and 19 threats. Members from Group 2 identi ed 8, 8, and 9 valid threats.
The expert evaluated most of the threats identi ed by the students as being valid. Some threats, e.g., "conduct physical attacks on organizational facilities", were commonly identi ed. Some others threats were identi ed less often ((for instance, only two out of six students identi ed "conduct attacks using unauthorised ports, protocols and services"). An explanation can be that some threats are di cult to understand (and identify), because they require speci c technical knowledge.
The Iconic group reported less di culty (E3AQ1) and more satisfaction of the results (E3AQ5). Also, they were indicated (E3AQ3) that additional software support is needed less, if compared to the symbolic group. Interestingly, the participants didn't anticipate that employing another representation format can result in a more complete list of threats. E3AQ5 and E3BQ5 answers of Group 1 both score 3.0. More speci cally, there is only a relatively small increase (0.3) in the di erence between E3BQ5 and E3AQ5 for Group 2.
In summary, all subjects in possession of the iconic model constructed more complete lists of plausible threats compared to their counterparts. It suggests that the threat identi cation task can bene t from employing an iconic model of a system. 5
5.1
both notations can be potentially used to identify threats to a system, E2Q1 from Experiment 2 and to a smaller extent E2Q3 showed that the perceived di culty of the modeling task slightly decrease when iconic signs are used. Notably, the Iconic group was less satis ed with the tools provided (E2Q2). Nevertheless, based on the outcome of the experiments we can argue that the use of iconic signs instead of symbolic ones lowered the cognitive complexity of the task.
identify threats to a system regardless of the model's representation (Experiment 1 ). However, if supplied with a readily made iconic models | in contrast to a symbolic one { they performed better (Experiment 3 ) and considered that such the iconic description was completely enough to perform the task. 5.2
As noted in [
Speci cally, this research provides initial empirical support for claims related to devising and employing means for eliciting security requirements. Our ndings hint at high-level suggestions how to approach eliciting security requirements from stakeholders who are less experienced in modeling. In particular: i) using icons for modeling compared to pure text representation of modeling constructs facilitates comprehension of non-experts; ii) iconic models can assist in identifying potential threats by non-experts. We envision that an informal iconic model of a system, such as the one shown in Fig. 2, can facilitate collaboration between stakeholders. 5.3
Notes on experiments. Some aspects related to the con guration of
experiments should be noted. First, it can be possible that outcomes of the
experiments were obtained by pure chance. However, it is the consistency of outcomes
of several experiments that points out that using icon-based informal
modeling language can be useful to identify threats to a complex system. Second,
the experiments were conducted with a low number of participants.
Nevertheless, the con guration of experiments was not intended to make statistic-based
claims. The potential for generalizations is related to ideas within cognitive
theories. More experiments with larger amounts of groups will clearly be bene cial.
Third, the experiments were focused on assets-threats connections. We did not
account for compliance obligations, raw requirements, security requirements, as
well as security measures at large. All these aspects (see, e.g., [
Model Quality. In this study the semantics (i.e., correctness and completeness) of the models was not investigated in detail. Also, although an RA expert examined the threats identi ed by students within Experiment 3, we cannot make any claims with regard to the e ects of iconicity on the absolute quality of the results. Besides, the "quality" of the identi ed threats was not part of the evaluation. It is the next steps of the security development process that should account for such a merit. Besides, we did not study how iconicity can explicate tacit knowledge (as experts are needed for this task) and creativity (students were provided with a list of possible threats). Still, we can anticipate that iconic models, due the reduction of cognitive load, can also contribute to these aspects.
Adherence to syntax. We observed that groups with symbolic signs started
to freely draw schemes on the whiteboard, thereby reducing possibilities to
enforce syntax of the modelling language. In connection to the cognitive t theory
[
Choice of signs. Symbolic signs were kept as simple as possible, by using
only boxes, arrows and colors. However, the complexity and suitability of iconic
signs were not evaluated. It is possible that these icons can be simpli ed, employ
more discriminable symbols, and possess more semantic transparency. Also, this
research didn't concern the modeling constructs themselves, as well as portability
of the modeling approach to a large-scale scenario. Huge CPS with lots of detail
might call for nding a particular level of abstraction. We acknowledge that the
set-up of this research accounts for only a fragment of the real world's complexity.
It does not investigate how having a very large number of iconic representations
can negatively impact human comprehension because of, for instance, similarity
across potentially similar elements. We can expect that in such cases modeling
languages might bene t from grouping elements. Also, di erent notations aspects
[
Eliciting security requirements, as a collaborative process that starts with identifying threats (in other words, misuse cases), should account for inputs from diverse stakeholders. In this paper, we empirically investigated iconicity | a feature of an informal modeling language. We concentrated on identifying threats to an urban grid as a case of cyber-physical system.
Our ndings indicate that individuals with little modelling experience do bene t from employing an iconic representations of an informal language. Participants of three experiments perceived iconic models as easier to construct and more understandable. Moreover, participants equipped with the iconic model were capable to point out more threats relevant to the system.
From a requirements standpoint, the ndings suggest that iconic representations of informal modeling languages constructs can bene t the threat identi cation task. This task is an important step of security requirements. Those concerned with developing and employing languages and tools for security requirements can consider employing such notations. Ultimately, the ndings can also assist specialists involved in communicating risk assessment and risk management processes to stakeholders. Future work should attempt to replicate the experiments at a large scale, ideally with practitioners.
Acknowledgments This work has been partially supported by the Joint Program Initiative (JPI) Urban Europe via the IRENE project and has received funding from the European Union Seventh Framework Programme (FP7/20072013) under grant agreement no 318003 (TREsPASS). We thank the students who participated in the experiments. This publication re ects only the author's views and the Union is not liable for any use that may be made of the information contained herein.