-This paper adapts System-Theoretic Early Concept Analysis (STECA), an instrumental safety risk management technique, for privacy to better identify and address privacy risks early in the engineering process. The technique, STECAPriv, aims to infer a nominal functional privacy control structure based on a conceptual system description and privacy-related system behavioral constraints. Model-based systems engineering (MBSE) is employed in conjunction with STECA-Priv to validate the projected control structure and to identify privacy risks in the form of constraint violations. To illustrate STECAPriv as supported by MBSE, it is applied to the simplified example of a smart television.
Keywords—privacy risk; System-Theoretic Early Concept Analysis; STECA; STECA-Priv; model-based systems engineering; MBSE
Engineering invariably involves both analytical and instrumental methods. The former tend to be more straightforward than the latter as they target an extant situation or specification, analyzing something that in some shape or form already exists. Instrumental methods, in contrast, support the creation of something new. It is not surprising, then, that methods for managing privacy risk as part of socio-technical system design are thinner on the ground than methods for assessing risk in already designed (and possibly already implemented) systems. This is not to imply the adequacy of current methods for privacy risk analysis, but rather to note that however problematic the state of analytical privacy risk management techniques, the state of instrumental privacy risk management techniques is even more so.
Previously [
In addition to adapting STECA for privacy (yielding
STECA-Priv) in much the same way we adapted STPA for
privacy (yielding STPA-Priv), we leverage model-based
systems engineering (MBSE) as a mechanism for checking the
integrity of the postulated control structure. (STECA itself was
partially inspired by MBSE [
The remainder of the paper is organized as follows. Section II provides background on STAMP, STECA, and MBSE. Section III discusses STECA and the modifications for privacy that result in STECA-Priv. Section IV uses the example of a smart TV to illustrate the combined use of STECA-Priv and MBSE. Section V situates STECA-Priv with respect to some related work while Section VI presents some concluding thoughts.
STAMP frames safety in terms of constraints rather than
events [
Controls are structured hierarchically with controls at each
level enforcing constraints on processes in the level below it.
This control structure exhibits multiple aspects. As described
by Leveson [
For a control to properly constrain a process, it must maintain a model of that process. Control errors arise when the process model being used by a controller doesn’t properly correspond to the process being controlled. (This can happen either because there is an error or gap in the model or because the model’s state does not match the actual process state.)
STPA is a safety risk analysis methodology based on the
concepts of STAMP. It was adapted for security (STPA-Sec)
[
Because STECA aims to project a nominal functional
control structure consistent with the system description and
relevant constraints, it lends itself to MBSE as a means of
describing and assessing that control structure. MBSE has been
driven in part by a 2007 joint initiative of INCOSE and OMG
[
A variety of powerful tools are available that support MBSE using various modeling languages, including UML and SysML. For our purposes, SysML is preferable due to how it handles constraints. Constraints in UML (specified using Object Constraint Language) are annotations, i.e., they convey information to the engineer but are not operationally integrated within the model. SysML constraints, in contrast, are integrated into model execution so that as it executes, constraint violations can be observed and captured.
This capability is essential to extracting full benefit from using MBSE in conjunction with STECA. MBSE using SysML will reveal aspects of the projected control structure that could potentially result in constraint violations and thus present risks. Those aspects of the projected control structure can then be reconsidered and the risks represented by the constraint violations appropriately managed through mitigation, avoidance, transfer, or explicit acceptance. Any resulting changes to the control structure are captured by the model and their effects verified through its execution.
As previously noted, STECA aims to project a system
control structure from a description of system behavior and a
safety risk model expressed in the form of system behavioral
constraints. Developed by Fleming [
2.
Derive system safety constraints (safety-driven design)
4. Identify hazardous scenarios and causal factors (ConOps analysis) 5. 6.
Derive refined safety constraints (safety-driven design)
Refine, modify control structure (safety-driven design)
In the same way that STECA leverages the activities
developed for STPA, in adapting STECA for privacy we can
avail ourselves of the adaptations of STPA developed for
STPA-Priv. Indeed, the first several steps are largely identical
to STPA-Priv (and we refer the reader to [
As with STPA-Priv, we structure STECA-Priv to accommodate any of a variety of privacy frameworks (i.e., risk models), recognizing the pluralism of understandings of privacy in this regard.
In STPA-Priv, Step 3 combines the specification of privacy constraints with the specification of the system functional control structure. However, for STECA-Priv we must infer that structure, a less straightforward process. It is at this point, therefore, that STECA-Priv substantively departs from STPAPriv: 4. Identify system privacy control concepts and infer privacy control models 5.
OR 5.
Use MBSE to represent privacy control structure and constraints as an executable model to identify risks in the form of constraint violations and their causal factors, including malicious actions
Note the absence of explicit reference to a ConOps. While one could be used for STECA-Priv if it exists, it is not strictly necessary. (Nor, arguably, is it strictly necessary for STECA). While substituting other types of high-level system description (nominal use cases or data flows, for example) may introduce additional difficulty, it does not fundamentally change the approach. By the same token, opting to represent the control structure and privacy constraints as an executable model doesn’t fundamentally change the approach either. Rather, it enhances it by enabling the identification and management of the privacy risks represented by constraint violations, a more focused lens for viewing the conceptual system than the less precise characteristics in the alternative Step 5. Table I shows how the steps of STECA-Priv compare with the steps of STECA, assuming use of MBSE.
As before [
The smart TV will collect and store a record of the programs watched on the device. This viewing data will consist of entries that include the date and time (timestamp), the channel number selected, the program name, and the service provider (to enable matching the channel number to a specific network). This viewing data will be regularly transmitted to the TV manufacturer, who will combine it with demographic data (based on IP address) and maintain the combined data, including IP address, in a repository. Users must opt-in (provide explicit consent) to the collection and use of viewing data by the manufacturer, as governed by the privacy policy associated with the TV.
For the purposes of this example, the preceding paragraph will function as the system description that serves as input to STECA-Priv.
As we did when illustrating the use of STPA-Priv
previously, we will use Calo’s subjective/objective privacy
harms [
The use of explicit consent goes a long way toward avoiding the potential for subjective privacy harms. However, an objective privacy harm may result if that consent isn’t accompanied by an accurate governing privacy policy that conveys the collection and use of viewing data and any relevant terms (e.g., regarding user access to the collected data). Similarly, an objective privacy harm may result if consent is not obtained or renewed following a change to that policy. This may be accompanied by a subjective privacy harm upon realization by a user of a material change to the policy absent consent. The relevant vulnerabilities can be expressed as:
The privacy policy associated with the smart TV is inaccurate as it pertains to viewing data collection and use.
Privacy policy and viewing data consent become unsynchronized.
These vulnerabilities can be reframed as corresponding system privacy constraints:
At any point in time, the privacy policy associated with the TV must be accurate.
This can be decomposed into two distinct constraints:
TV correctly describes the applicable privacy practices.
Consent must correspond to the current privacy policy.
This also can be decomposed into two distinct constraints:
has been registered.
To this point, the process has been the same as if we were applying STPA-Priv to the described system. But, of course, this is not a fully described or specified system, but a (circumscribed) high-level concept which must be fleshed out. It is at this point that STECA-Priv diverges from STPA-Priv.
In the absence of a more or less complete system specification, we must infer a functional privacy control structure rather than extract the control structure. The initial step in performing this inference is identifying the system privacy control concepts from the limited description that we have. To accomplish this, we assign control loop roles to applicable elements in the system description. Figure 1 shows the general form of a STAMP control loop. A STAMP control loop includes four principal roles: controller, actuator, controlled process, and sensor. The controller issues control actions, which are executed by some kind of (not necessarily physical) actuator, which acts on the controlled process, the relevant behavior of which is conveyed back to the controller by the sensor. the control loop can achieve the four necessary conditions2 of process control and adequately interact with its environment, other processes, and other controllers. In other words, these guide words are necessary to ensure that a control loop is controllable and coordinable with other controlled processes.
11. External
input 2.
Actuator 9. Control input (setpoint) or other commands 8. Feedback to higher level controller 7. Control Action 1. Controller 6. Control Algorithm
OnTchee insfeortmsatoiofn inthFeigsuere 1e1leanmdethnetsabohvealvisets (Cboenetnrolleird, eAncttuiaftioerd,C, ontthroelled privParcoycessc,Soenntsroor)lcanmthoedneble ucseodrrtoessypsotenmdaitincaglly tpoarseeaancdhqusereytthecananturbalelandevegluoagpeedde.sc(rIipttiiosnnoortgreaxphpiecacltdeedpitchtioant ianlla pcoontceepnttioaflopmeroadtioenls.elTehmereensutslting willmboedelsapnedcsiufibesedq;uennottdaatalblaseelearme eeansytstoairnetermrogaantedaantdorvyisualizae. cTohensteroqulaliin looptiesahnedlp thliemaniatelydst toinchfoecrkmfoartiinotenrnamlinacyonspisrteencclieusdaend/iodr emnistsiifnygiinngformoartion projtehcattimnagy result in unsaTtiasfibeldecontrol conditions, qanudeasltsiootonscheck for inconsitshteinscies others.) II provides to guide procaecrsoss.s tThehsiyssteymiehlidersarcthhye. control models in Tables III and IV,
Table 6 provides a series of prompts that an analyst can use when rTeaadbinlegsa tVext or addressing constraints 1.a and 1.b respectively, and and gVraIp,haicdindareCsosniOnpgs.constraints 2.a and 2.b respectively. To save space,Inthoredseer totaobbtlaeisn ao“ncolymplceoten”tmaiondeltohfotshee CroonOwpss, twhiisthmodpeol pduevlealotepmdent approach should be applied recursively over the entire ConOps document. The keydescriptions.
words, with associated questions and comments (Tables 6 and 7), can be applied to H2Saeevpiangeg56.inferred these control models, we can now use MBSE to represent the functional privacy control structure of these aspects of the system togethe61r with the applicable privacy constraints. To keep the example manageable, we will focus on the control loops implied by Tables V and VI in which the smart TV is the controller. These control models address constraint 2.
We begin this process by creating the overall structure of the model. Figure 2 shows this structure using SysML block definition diagrams. The principal blocks represent the smart TV and the smart TV manufacturer. (Although denoted separately, the TV viewing dataset is part of the smart TV manufacturer.) We have also included the smart TV user and the demographics provider for context. (As a practical matter, the modeler will play the role of the user.) The specified multiplicities are nominal and do not affect the example. Communications between the user (actually the modeler) and the TV and the TV and manufacturer are implemented by SysML signals and events via ports that are denoted on internal block diagrams representing the TV and manufacturer blocks.
The remaining block is a constraint block, Consent-Policy Sync. Constraint blocks define conditions that should hold at all times. As long as the constraint expression evaluates to true, the constraint is satisfied. If the expression evaluates to false, 7. 15. 1. 2. 3. 6. 7. 15. the constraint has been violated. While primarily intended to apply to models involving substantial mathematical calculation, we are using this facility to define a logical constraint using Boolean variables, which are described below.
The control models described by Tables V and VI are represented by the state machine associated with the smart TV block and shown in Figure 3. As a practical matter, control models and their representations may require interative refinement. Within the control representation, control actions take the form of SysML activities. Thus, the activities called in the state machine diagram correspoind to the similarly named control actions described in Tables V and VI.
The process models corresponding to constraint 2 are represented using Boolean variables whose values indicate whether a privacy policy update has been received from the smart TV manufacturer (2.a) and whether consent for the collection and use of viewing data has been granted (2.b). To avoid viewing interruption, policy updates are presented to the user when the TV is turned on, requiring two distinct control actions for receiving and processing policy updates. Note that because a policy update is registered as the default value (policyUpdated == true), this will happen “out of the box.”
If policyUpdated == true, the TV channel is switched to channel 0 (the TV’s user messaging channel) to display the updated policy and the policy update indicator is cleared (policyUpdated = false). The user must then grant or deny consent, which sets the consent indicator accordingly. The TV then enters its Operating state. (If policyUpdated == false, the TV moves immediately from the On to the Operating state.) If consent has been granted (consent == true), the TV periodically transmits viewing data to the manufacturer, where it is processed and stored. Thus, policyUpdated acts as a transition guard in the On state while consent acts as a transition guard in the Operating state. While in the Operating state, the TV monitors for privacy policy updates and, if one is received, replaces the previous policy with the new one and sets the policy update indicator (policyUpdated = true).
The constraint reflects the fact that at no time should there be an updated policy indicator (policyUpdated == true) together with affirmative consent (consent == true). If this occurs, it implies that an updated privacy policy has not yet been presented to the user for them to consent to, but that consent has nevertheless been granted. This would constitute a violation of constraint 2.a. Note that constraint 2.b is enforced by the guard condition on the Operating state transition that results in the transmission of viewing data.
As the left window of Figure 4 shows, this constraint was violated (“failed”) when the model was executed. Execution involved turning the TV on, granting consent, then updating the privacy policy. The right window shows the implicated variables shaded red. (While the constraint held, they were shaded green.) Upon examination, the problem becomes apparent. Because an updated privacy policy is not immediately presented to the user and the consent indicator is not cleared when the policy update indicator is set, viewing data continues to be transmitted. In other words, if the user had previously consented to the collection and use of viewing data by the manufacturer, that consent would continue under a new policy until such time as the TV was turned off and then on again.
From a STAMP perspective, this represents a control action provided at the wrong time. The most straightforward way of addressing this (thereby mitigating the privacy risk related to the constraint violation), is to clear the consent indicator (consent = false) at the same time as the policy update indicator is set (policyUpdated = true). This prevents viewing data from being transmitted under the new policy until the user has had the opportunity to grant or deny consent. This is verified by making the necessary changes executing the modified model, and observing that the constraint failure no longer occurs (and the relevant variables in the right window of Figure 4 are shaded green rather than red).
STECA-Priv, like STPA-Priv, bears some relationship to
goal-oriented modeling [
Also like STPA-Priv, but unlike goal-oriented modeling, STECA-Priv takes an approach explicitly grounded in systems theory. This manifests itself not only in a typology of problematic control actions, but also in the use of rolebased control models. Together with the use of a chosen privacy framework, these contribute to a more focused and practically-structured methodology. As such, it is a methodology that strikes a balance between rigidity and open-endedness.
This balance is also reflected in the use of MBSE in
support of STECA-Priv. By definition, MBSE is a general,
open-ended process, as is reflected by some of its specific
methods [
STECA-Priv is an instrumental method for performing privacy risk management on complex socio-technical systems at early life cycle stages. It does not require a relatively complete system description, instead inferring provisional privacy control structure in a systematic fashion for the purpose of system specification. Note, though, that the use of STECA-Priv in early life cycle stages does not obviate the need for downstream risk analysis, such as STPA-Priv, once the system is more fully fleshed out; earlier assumptions and inferences may no longer be valid and/or other privacy risks may have been introduced. We are currently applying STECA-Priv to a real-world identity management project, which should provide insights into its utility and practicality. As it is a new method, modifications based on that experience may be needed, though we are confident that the fundamental soundness of the approach will be validated.
Clearly, we also believe in the potential value of combining STECA-Priv with MBSE. The ability to model a system’s inferred privacy control structure and to execute that model with integrated constraints adds further rigor to the application of STECA-Priv. It supports identification and correction of privacy control problems early in the SELC. While STECA is premised on the availability of a system ConOps, this is arguably an arbitrary target. Whether sufficient descriptive information exists to justify the effort required to apply STECA-Priv will be a case-by-case judgment call.
So too will be the use of MBSE in conjunction with STECA-Priv. Our experience with a representative tool has been that the learning curve is steep. Even once that learning curve has been climbed, developing an executable SysML model of a complex system will require a substantial resource commitment, more so if the investment is to be maintained over time by capturing all future system changes in the model. However, we believe that the potential value of such models may be significant, enabling the detection of privacy risks, including emergent system properties, initially and as the system undergoes changes. While our illustrative example is relatively straightforward, real-world complex systems are decidedly less so. We anticipate MBSE typically would reveal unexpected privacy control problems. This would be especially valuable (and worth the resource expenditure) for systems that are intrinsically high risk due to their data, technology, and/or usage contexts.
Further work is needed to develop criteria for when an explicit constraint in the form of a constraint block is or isn’t needed. In the case of the former, it would also be desirable to develop guidance on how to formulate such constraints based on the control models as represented in SysML. That representation involves a translation from the relative abstractions of the STECA-Priv control models to the more specific constructs of SysML. Such guidance actually might be considered a subset of broader guidance regarding how to perform that translation. If such guidance can be developed, the application of STECA-Priv in conjunction with MBSE would become less of an art and more of a discipline.