'Privacy by design' is not only important from an economic perspective but also from a legal one. The upcoming European General Data Protection Regulation makes privacy by design and default mandatory. One concrete step an organisation can take towards privacy by design is to perform a privacy impact assessment. To verify the assumption that the outcome of the assessment leads to sufficient and adequate input for designing privacy-friendly products and systems that comply with privacy regulations and social norms regarding privacy we performed a descriptive field study in the Netherlands. In this paper, we present the results of this study. Our main results are the following. When performing a privacy impact assessment, organisations use the organisation itself as a focal point, instead of the data subjects whose data is being processed. The proposed countermeasures tend to address the effect rather than the cause of a privacy risk. A consequence of this focus is that the outcome of the privacy impact assessment will lead, at best, to a product or system that is compliant with data protection regulation. It will not lead to a product or system that is privacy-friendly, or one that takes into account social norms regarding the processing of personal information. Another significant result is that the data protection officers who were interviewed perceive the process of determining privacy risks, based on the information gathered about a specific product or system, as vague. Further research is needed to develop a more rigorous and transparent process for determining privacy risks that can be used by organisations.
To build privacy-friendly products and systems that comply
with legislation and social norms, privacy 1 needs to be
addressed from the very beginning during product or system
development. Ex-post implementation of privacy preserving
mechanisms into an existing system is in practice very difficult.
It mostly involves in-depth system adjustments and is therefore
relatively costly. The principle, to take privacy into account
throughout the entire development process — from the earliest
design stages, through the implementation phase, right until
deployment — is called ‘privacy by design’ [
Jaap-Henk Hoepman
One concrete step an organisation can take towards privacy
by design (and actually one that is required for certain types of
processing in the upcoming Regulation) is to perform a privacy
impact assessment. According to Wright [
In this paper, we present the results of this field study regarding the use of privacy impact assessments in practice, and compare this to the theory and the requirements stipulated in the upcoming Regulation (Section V). For our study, we selected fourteen organisations across eight sectors with different data subject categories and different sizes. We interviewed the data protection officers of these organisations using a predefined survey. Our methodology is explained in Section IV.
The main answer (see section VI for details and substantiation) to our research question is that the outcome of the privacy impact assessment for most of the interviewed organisation will lead, at best, to a product or system that is compliant with data protection regulation. It will not lead to a product or system that is privacy-friendly, or one that takes into account social norms regarding the processing of personal information. We conclude this paper with suggestions for further research on this topic (see section VII).
In this paper, we do not only take the legal requirements on data protection into account, but also the social norms (values/expectations) regarding the processing of personal data. This broadening of the scope is prompted by Wright’s definition of privacy impact assessments and the concerning article in the Regulation which mentions that (representatives) of the data subject (the person about whom personal data is processed) can be consulted during such a privacy impact assessment. Also, this approach is inspired by the fact that non-compliance with societal values may lead to significant negative publicity. For example, in the Netherlands social indignation arose in 2014 when Equens (a payment service provider) launched the idea to sell the payment transaction information of customers. The same occurred in 2014 when ING Bank wanted to do a pilot in which it would offer personalised third-party ads to their customers (with their consent) based on their individual spending patterns. Both ideas were formally compliant with the Dutch Data Protection Act.
Because we not only take into account the legal requirements regarding data protection but also social norms and expectations we use the terms privacy impact assessment and privacy by design instead of the terms used in the Regulation such as ‘data protection impact assessment’ and ‘data protection by design’. III. PRIVACY BY DESIGN AND PRIVACY IMPACT ASSESSMENTS
Privacy by design is intended to improve overall privacy
friendliness when designing an information system. The
fundamental principle of privacy by design is that privacy
requirements must be taken into account throughout the entire
system development process. Privacy is a core property of a
system that is heavily influenced by the underlying system
design. As a consequence, privacy by design cannot be
implemented as an add-on [
As mentioned earlier a privacy impact assessment is a process for assessing the impacts on privacy of a product or service, and for taking remedial actions as necessary in order to avoid or minimize negative impacts. These remedial actions can be 2 taken into account when implementing technical and organisational measures to ensure a level of protection appropriate to the risks of infringement on the rights and freedoms of natural persons. Roughly one can distinguish the following three phases in a privacy impact assessment: 1) collect the necessary information, 2) determine privacy risks and 3) propose mitigating measures to avoid or reduce the determined privacy risks. The outcome is normally documented in a report. That report can be used both as input for the concept development and analysis phase of the system development lifecycle, as well as for the testing and evaluation phase of that cycle. The use of the report in the latter phases helps to determine if the countermeasures ultimately chosen during the implementation phase have indeed eliminated or mitigated the initial identified privacy risks. We did not assess the quality of the outcome of the privacy impact assessment.
We performed a descriptive field study in the Netherlands among fourteen organisations between late 2015 and mid 2016. The selected organisations are distributed across eight sectors (see Table I) with different data subject categories (e.g. consumer, passenger, patient, civilian) and different sizes of organisations. In this way, we gave preference to a wide variety of sectors above the ability to compare results per sector. 2 One of the amendments of the European Parliament on the proposal for the Regulation was that the output of the privacy impact assessment needs to be taken into account. This amendment has not been adopted in the final version of the Regulation. The final text of the Regulation merely mentions that a privacy impact assessment needs to be conducted where the type of processing is likely to result in high privacy risk.
We interviewed the data protection officers (or someone with an equivalent role) of each of the fourteen organisations using a predefined survey. We did not question or discuss the answer (to prevent bias), apart from asking for clarification when the answer was not clear.
At the time of the interviews the Data Protection Directive
[
E.
F.
Results from the PIA (PIA and PbD) 1. How do you determine that the output of the PIA is used for concept development and analysis (information system development)? i) If the output is used, how is guaranteed that the results of the
PIA are known and used by the IT-department? ii) If not why? What do you need? 2. How and when is monitored if the mitigating measures of PIA are implemented during the development phases? 3. Did the outcome of the PIA resulted in changes in the (specs of the) information system.
Consultation with stakeholders 1. Who are the stakeholders? 2. Are the results of the PIA consulted with stakeholders? Which stakeholders? If not, why not? Governance PIA 1. Is the quality of the PIA assessed? By whom? 2. Is somebody assigned to manage the PIAs (e.g. the privacy
officer) 3. Are PIAs periodically revised (is this an obligation)?
Table 2 describes the questions used during the interviews to verify our assumption that the outcome of the privacy impact assessment should lead to sufficient and adequate input for designing privacy-friendly products and systems that comply with privacy regulations and social norms. This is why, it is in our opinion necessary to get insight into why organisations conduct privacy impact assessments, what their definition of privacy risk is, what their strategies of reducing privacy risk are, when and how the assessments are conducted, whether the organisation scales the assessment (small/full) depending on the phase of development and/or the type of data processing, who the stakeholders are, and how the quality is assured. We also wanted to gain insight into how organisations use the output of the privacy impact assessment for privacy by design.
In this section, we present and discuss the outcome of our
survey. We do this treating for each of the six topics separately.
For each topic, we first present a summary of the responses for
each of the questions that belong to that topic. We then follow
through with our analysis of that topic: we compare the outcome
of our interviews with the theory (especially the work of Wright
and De Hert [
A. Why and when to conduct a privacy impact assessment 1) Questions and answers • How does your organisation define a privacy impact assessment? Has the definition been published? Most organisations defined the privacy impact assessment as a tool/process to determine whether there are privacy risks, how big they are and to provide recommendations for mitigating measures. According to these organisations, the definition used was described briefly in the privacy impact assessmentdocumentation. In a few cases the privacy impact assessments were an integral part of the system development process and were not treated and thus not documented separately. • Why does your organisation conduct a privacy impact assessment? Most organisations conducted a privacy impact assessment because they thought it was mandatory for them. In a few cases it was mentioned that the assessment was conducted to prevent the loss of customer trust or to prevent an inappropriate infringement on the personal life of the customer. • Since when has your organisation conducted privacy impact assessments? Most of the organisations started conducting privacy impact assessments in 2012-2013, some in 2006-2010 and one organisation as early as 2002. • How many privacy impact assessments are conducted in your organisation? Most organisations had no (central) database with all conducted privacy impact assessments and had to make an estimation. The amount varied from 15 to 550. Most organisations only conducted privacy impact assessments on new or revised systems. Others also conducted the assessments on existing systems because they did not do it in the past and now wanted to have insight into the privacy risks the organisation could face.
Under the current data protection legislation most of the
selected organisations, except for governmental authorities
under certain circumstances, are not obliged to conduct a
privacy impact assessment. Nevertheless, most data privacy
officers mentioned that it is mandatory. This obligation can be
stipulated in the Binding Corporate Rules4 or other Group policy
rule that some of the organisations have implemented. Others
wrongly perceived it as an obligation. Although a privacy
impact assessment should be more than simply a compliance
check, it does nevertheless enable an organisation to
demonstrate its compliance with privacy legislation in the
context of a subsequent complaint, privacy audit or compliance
investigation. A privacy impact assessment enhances informed
decision-making and exposes internal communication gaps or
hidden assumptions about the project [
Because there was no real obligations to conduct privacy
impact assessments for most of the selected organisations we
expected that data protection officers would mention reasons for
conducting the assessment spotting potential privacy problems
and taking effective countermeasures (early warning),
avoidance of inadequate solutions, avoidance of negative public
reaction or loss of trust and reputation, avoidance of unnecessary
costs or education, raising awareness about privacy among
employees or gaining competitive advantage [
Under the upcoming Regulation conducting a privacy impact assessment will be mandatory, dependent on the nature of the processing. For processing likely to result in a high risk to the rights and freedom of natural persons organisations have to carry out the assessment. The Regulation stipulates that the assessment shall in particular be required in the case of a) automated processing (including profiling) on which decisions are based that produce legal effects concerning natural persons; b) processing on a large scale of special categories of data or of personal data relating to criminal convictions and offences; and c) a systematic monitoring of publicly accessible area on a large scale (art. 35 par. 3 GDPR).
• Can you describe how a typical privacy impact
assessment is initiated and executed within your
organisation? Almost all organisations executed the
privacy impact assessment more or less the same way.
They started by gathering the necessary information
for the assessment (mostly through a questionnaire).
Based on that information the privacy risks were
determined and mitigating measures were proposed to
and agreed to be implemented. Within some
organisations the residual privacy risks that remain
because not all measures were implemented must be
approved by senior management.
• In which cases does your organisation conduct / not
conduct a privacy impact assessment (is there a
threshold)? Most organisations conducted the privacy
impact assessment for each system in which personal
data was processed: there was no real threshold. Some
organisations used the amount of financial investment
for the new/changed information system as threshold
to determine whether a privacy impact assessment was
needed, for example investments worth over 1 million
euros. Some other organisations performed a pre-scan,
which provided a preliminary determination whether a
privacy impact assessment was required.
• Is there a guideline for how to conduct a privacy
impact assessment? On which methodology or
standard is it based? Most organisations had some
kind of guideline or framework for conducting privacy
impact assessments. There was no uniformity at this
point. For governmental authorities the “Framework
privacy impact assessment Dutch National
Government” [
assessment was part of a larger assessment. In order of occurrence (from many to few) the privacy impact assessment was part of: compliance, project delivery, information security and business impact assessment. The credo of one of the data protection officers is “to burden the organisation as little as possible by ‘freeriding‘ on existing procedures”. • Who conducts the privacy impact assessment (an individual or a team; which functions are represented)? More than half of the organisations conducted the privacy impact assessment through several bilateral consultations between the data protection officer/privacy advisor and other officers of that organisations (business owner, senior staff, analyst (business/infra), information security officer, lawyer, etc. The remaining organisations conducted the assessment with a team of which the data protection officers/privacy advisor is a (supporting) team member. The size of the team depended on the project, and typically consisted of the aforementioned other officers of the organisation. In some organisations there was a strict separation between the monitor compliance-task and the advisory-task of the data protection officer. The data protection officer monitored compliance and the privacy advisor advised. When a privacy advisor was appointed, he or she participated in the privacy impact assessment and the data protection officer revised it. • In which phase or phases in the product and/or information system development is the privacy impact assessment conducted? Almost all data protection officers mentioned that they intend to conduct the privacy impact assessment in the early phases of system development. The problem was that it was not always common practice for project managers to consult the data protection officer about a new project. Within some organisations, it was a requirement that the privacy impact assessment had been conducted before the development could continue (this was part of a gateway review). Although it could take several meetings to complete a privacy impact assessment, it was not a dynamic process for these organisations. It was conducted in a specific moment (phase), not over a period of time. A few organisations followed a process oriented approach, where they started during product development and supplemented the assessment during the system development. • Is there one questionnaire for all data processing or is it tailor-made (e.g. depending on the development phase or depending on standard or tailored software)? Almost all organisations used one questionnaire for all phases and for all types of personal data or data subjects. Some organisations used different types of frameworks depending the kind of data processed and thus different questionnaires. One organisation used a master privacy impact assessment for the repetitive part of projects and used an addition privacy impact assessment for the unique parts of the projects. None of the organisations had different questionnaires depending on whether the product/service would be supported by standard software or tailored software.
Most of the data protection officers of the selected organisations
conduct privacy impact assessments in more or less the same
way and for all processing with one questionnaire. The
assessment is, with a few exceptions, conducted early in the
development process. The threshold to conduct an assessment or
nor is the question whether personal data is processed or not.
This is not appropriate. First, the degree of risk created by
projects varies enormously. Second, projects vary widely – from
updating a small database to implementing new legislation, or
developing a new product or service. Some authors recommend
that organisations conduct a limited preliminary evaluation, to
establish whether the organisation needs to invest in a
smallscale or a full-scale privacy impact assessment [
All selected organisations check at the end of the development
process (test and evaluation) whether the agreed upon measures
are indeed implemented. In that phase, the data protection
officers do not re-assess the privacy impact assessment. Privacy
risks could have changed or new risks may appear as a result of
design and/or implementation decisions. A re-assessment
should therefore be carried out. (See Fig. 1 for a graphical
representation for the relationship between these three types of
privacy impact assessments and the other product and system
development phases). However, as mentioned earlier, Wright
states that the privacy impact assessment should be regarded and
carried out as a process and not just as a single task that results
in the completion of a report [
An organisation should determine the roles and
responsibilities of its officers with regard to privacy impact
assessment, for example who initiates one, who carries it out and
who approves them. A team of experts, including external ones,
might be necessary. The privacy expertise is crucial here but it
does not exclude other fields. Outsourcing the privacy impact
assessment in full is not desirable. The line manager should be
responsible for conducting the assessment because, first and
foremost, she is accountable for the risks posed by her
products/services. Secondly, she knows the product/service well
and hence should be able to tell where the main risks are. Finally,
doing a privacy impact assessment internally would help to
create privacy awareness throughout the organisation [
C. How to determine privacy risks and measures 1) Questions and answers • How do you define privacy risk? In most cases privacy risk was defined from the perspective of the controller, i.e. unlawful processing of personal data resulting in high fines of the Supervisor Authority and loss of reputation. In a few cases the risk was perceived primarily from the perspective of the data subject, e.g. infringement on the personal life of the data subject, resulting in loss of trust of the customer which could cause loss of market share. In these cases possible fines were only secondary. • How are privacy risks determined/identified in a privacy impact assessment (automatically/ manually)? Within almost all organisations the privacy risks were determined manually (mostly supported by the data protection officer/privacy advisor). A few organisations used a mechanism which determined possible risks and mitigating measures automatically. The organisations that used privacy advisors mentioned that the quality of the determined the privacy risks was very dependent on the skills and experience of the person determining that risk. The data protection officers who were interviewed perceive the process of deriving privacy risks based on the filled-out questionnaire as vague. One of the data protection officers compared it to a black-box. • How does your organisation cope with reducing privacy risk (strategy)? Most data protection officers mentioned that their organisation did not had a general strategy for reducing privacy risks. When asked to give examples of solutions to reduce the privacy risk, the organisations that defined the privacy risk from the perspective of the controller tended to favour measures that mitigate the risk (e.g. encryption or access management) instead of avoiding risks (e.g. pseudonymisation or data minimisation).
In the Regulation “data protection risk (privacy risk)” is not defined. The corresponding article about privacy impact assessment only mentions “…the rights and freedoms of natural persons…”. This indicates that, from the point of view of the Regulation, the data subject perspective is more relevant than the controller perspective. The process of determining risks and measures is not well defined, and no guidance is provided. As a result, the quality of it very much depends on the person performing the privacy impact assessment. It is a black box. In addition, solutions to reduce the privacy risk are sought in measures mitigating the risk instead of avoiding the risk; especially in organisations that define privacy risk from the perspective of the controller. This is understandable (but not defendable). When the data protection officer defines privacy risk as the risk of getting fined by the Supervisory Authority he will look at the effect of a privacy risk instead of the cause. When you subsequently determine measures to reduce the privacy risk –bearing in mind the effect of the privacy risk– you are more likely to start thinking in terms of measures to reduce the risk of non-compliance. When you determine measures – bearing in mind the cause of the privacy risk– you probably start thinking in measures that reduce the inherent risk, i.e. the cause. This does not mean that in all cases the ultimately chosen solution will be sought in avoiding privacy risks. See Fig. 2 for a graphical representation.
Focussing on the risk to the controller will lead at best to products or systems that are compliant with data protection regulation, but the resulting system may not always be privacyfriendly.
• How do you establish that the output of the privacy impact assessment is used for concept development and analysis (information system development)? If the output is used, how is guaranteed that the results of the privacy impact assessment are known and used by the
organisations (in the person of the project owner, data protection officer, information security officer, executive management, etc.) agreed to implement the measures proposed in the privacy impact assessment. In the organisations where information security officer was involved the data protection officers believed that the measures were more likely to be developed. The project owner was ultimately responsible for implementing the agreed measures. • How and when do you monitor whether the mitigating measures of privacy impact assessment are implemented during the development phases? As part of the information system design cycle the developed system was tested to determine whether it is built in conformance with the specifications (including the ones from the privacy impact assessment). The test team gave a "go/no go". Sometimes the project owner must sign off explicitly that the measures of the privacy impact assessment had been implemented; otherwise the project would be placed on hold. • Did the outcome of the privacy impact assessment result in changes in the (specifications of the) information system. As a result of the privacy impact assessments personal data was better secured, in some cases less personal data was collected and in other less personal data was presented (e.g. on screens and letters). Besides the specific improvements in information systems, conducting privacy impact assessments resulted in enhancing awareness of data protection throughout the organisation.
As part of the information system design cycle the developed system is tested to verify that it was built in conformance with its specifications. As mentioned earlier, the data protection officers should re-assess the privacy impact assessment during the 'testing and validation'-phase because privacy risks could have changed or new risks may appear as a result of design and/or implementation decisions.
• Who are the stakeholders? The data protection officers mentioned departments/ officers within the organisation as stakeholders. The ultimate stakeholder, the data subject was hardly mentioned. Only when the data processing involved personnel, the working counsel was mentioned as stakeholder. • Are the results of the privacy impact assessment consulted with stakeholders? Which stakeholders? If not, why not? The results of the privacy impact assessment were only shared with the involved officers within the organisation; not everyone within the organisation had access to (a subset of) the report. None of the selected organisation published (a subset of) the privacy impact assessment report externally. Only one case involved data subjects. This organisation involved customers for improving the quality/friendliness of the consent notice in an UX-lab to achieve a higher consent rate of their customers as legal grounds for processing personal data.
The data subject is one of the stakeholders of the privacy
impact assessment-process whose remarks must be taken into
account [
F. Governance privacy impact assessment 1) Questions and answers • Is the quality of the privacy impact assessment assessed? By whom? The quality of the privacy impact assessment was secured through the participation of experts in the team. If privacy advisors were used the data protection officer typically reviewed it. In some organisations, the report was signed off by key parties (like applicable line manager, data protection officer, information security officer and depending on the residual risks also executive management). This not only improved the involvement of the key parties but also the quality of the report. Little or no auditing of the privacy impact assessment was performed. • Is somebody assigned to manage the privacy impact assessments? Among the selected organisations there was no common understanding. The following people were mentioned as being responsible: the product owner, the data protection officer, the chief information officer, risk management department. • Are privacy impact assessments periodically revised (and is this an obligation)? About half of the organisations did not specify conditions for revising a privacy impact assessment. The other organisations had explicit conditions for reassessment of the impact of privacy risks (every two to three years, or earlier in case of large changes). In one case the revision of the privacy impact assessment was part of a certification program for that information system (5 years).
As seen earlier, in most organisations the roles and responsibilities involved in conducting privacy impact assessments are described. But managing the life cycle of the privacy impact assessment is not. At best a revision term is specified. This needs to be improved.
We conducted a field study regarding the use of privacy impact assessments in practice in the Netherlands. The main results of our study are the following: • •
Most of the data protection officers who were interviewed perceive wrongly that they are obliged to conduct a privacy impact assessment. The European Data Protection Directive (which was in force at the time we performed our study) does not mention such an obligation at all. The upcoming European General Data Protection Regulation stipulates that only in circumstances where the processing is likely to result in high risks to the rights and freedoms of natural persons does an assessment need to be carried out.
Most organisations use an uniform approach (incl. one questionnaire) for assessing all data processing, regardless of the type of processing and the type of project. Based on existing research a preliminary evaluation was expected to determine whether to conduct a small-scale or full-scale privacy impact assessment. • • • •
Most organisations conduct the privacy impact assessment at one phase during system development (in the early phases) but they do not supplement the assessment during the development process. Existing research states that the assessment should be regarded as a process, and not just as a single task.
Most data protection officers define privacy risks from the perspective of the controller (the risk of getting fined by the Supervisory Authority) instead of the perspective of the data subjects. This is not in accordance with the spirit and the legal requirements specified in the Regulation.
When reducing the assessed privacy risks most organisations favour measures that mitigate risks, instead of measures that avoid them.
Most organisations do not consult (representatives of) the
data subjects as part of the privacy impact assessment
process. Consultation is advised by a number of authors [
Most of the participating organisations were highly controller-oriented instead of data subject-oriented when considering privacy risks. This was apparent from the reasons for conducting privacy impact assessments and the definitions of privacy risk given by the data protection officers, the proposed measures for reducing the privacy risk, and the practice of not consulting (representatives of) the data subject as stakeholders. These organisations tend to look at the effect rather than the cause of a privacy risk. When the outcome of a privacy impact assessment by these highly controller-oriented organisations is used to implement the principles of ‘privacy by design’, this will lead at best to a product or system that is compliant with data protection regulation. It will not lead to a privacy-friendly product or system and/or one that takes into account social norms regarding privacy.
A more rigorous and transparent process for determining privacy risks that can be used by organisations in practice needs to be developed. Data subject risks, instead of controller risks, should be central. And these risks should be avoided instead of merely being mitigated: the output of a privacy impact assessment should steer the initial system design. In fact we believe the privacy impact assessment process and the resulting privacy by design process should be integrated into a single methodology (what we call a Privacy Impact Reduction Methodology) that fosters the development of truly privacyfriendly products and systems that, by default, comply with both data protection regulations and social norms.