Privacy Impact Assessment in Practice The Results of a Descriptive Field Study in the Netherlands Jeroen van Puijenbroek Jaap-Henk Hoepman Radboud University Nijmegen Radboud University Nijmegen P.O. Box 9010, 6500 GL Nijmegen, the Netherlands P.O. Box 9010, 6500 GL Nijmegen, the Netherlands J.vanPuijenbroek@cs.ru.nl jhh@cs.ru.nl Abstract: ‘Privacy by design’ is not only important from an Protection Regulation [2] (hereafter: the Regulation) which economic perspective but also from a legal one. The upcoming comes into force on 25 May 2018 makes privacy by design and European General Data Protection Regulation makes privacy by by default mandatory. Organisations need to implement data design and default mandatory. One concrete step an organisation protection when designing products and services that process can take towards privacy by design is to perform a privacy impact personal data. Because of the extra territorial scope of the assessment. To verify the assumption that the outcome of the Regulation this requirement is also important for organisations assessment leads to sufficient and adequate input for designing established outside the European Union when they process privacy-friendly products and systems that comply with privacy personal data of people residing in Europe. regulations and social norms regarding privacy we performed a descriptive field study in the Netherlands. In this paper, we Unfortunately, there are currently no concrete mechanisms present the results of this study. Our main results are the that can be used to integrate privacy throughout the entire following. When performing a privacy impact assessment, development process. But such mechanisms are being organisations use the organisation itself as a focal point, instead of developed. For example, privacy design strategies have been the data subjects whose data is being processed. The proposed proposed as a means to translate legal norms into engineering countermeasures tend to address the effect rather than the cause goals that assists to shape a privacy-friendly design during the of a privacy risk. A consequence of this focus is that the outcome early stages of system development [3] [4]. Also, the PRIPARE of the privacy impact assessment will lead, at best, to a product or project has proposed a methodology based on best practices, system that is compliant with data protection regulation. It will not integrating goal-oriented and risk-based approaches [5]. lead to a product or system that is privacy-friendly, or one that takes into account social norms regarding the processing of One concrete step an organisation can take towards privacy personal information. Another significant result is that the data by design (and actually one that is required for certain types of protection officers who were interviewed perceive the process of processing in the upcoming Regulation) is to perform a privacy determining privacy risks, based on the information gathered impact assessment. According to Wright [6] “A privacy impact about a specific product or system, as vague. Further research is assessment is a process for assessing the impacts on privacy of needed to develop a more rigorous and transparent process for a project, policy, programme, service, product or other initiative determining privacy risks that can be used by organisations. and, in consultation with stakeholders, for taking remedial Keywords: privacy; privacy impact assessment; privacy by design, actions as necessary in order to avoid or minimize the negative General Data Protection Regulation, data protection, data impacts”. We wish to establish whether the outcome of the protection impact assessment; data protection by design privacy impact assessment leads to sufficient and adequate input for designing privacy-friendly products and systems that comply I. INTRODUCTION with privacy regulations and social norms regarding privacy. To To build privacy-friendly products and systems that comply verify whether this is indeed the case we performed a descriptive with legislation and social norms, privacy 1 needs to be field study between late 2015 and mid 2016 in the Netherlands. addressed from the very beginning during product or system In this paper, we present the results of this field study development. Ex-post implementation of privacy preserving regarding the use of privacy impact assessments in practice, and mechanisms into an existing system is in practice very difficult. compare this to the theory and the requirements stipulated in the It mostly involves in-depth system adjustments and is therefore upcoming Regulation (Section V). For our study, we selected relatively costly. The principle, to take privacy into account fourteen organisations across eight sectors with different data throughout the entire development process — from the earliest subject categories and different sizes. We interviewed the data design stages, through the implementation phase, right until protection officers of these organisations using a predefined deployment — is called ‘privacy by design’ [1]. Privacy by survey. Our methodology is explained in Section IV. design is not only important from an economic perspective but also from a legal one. The upcoming European General Data The main answer (see section VI for details and 1 In this paper, we focus on safeguarding personal data processing. We have chosen the term “privacy” rather than “data protection” because of the broader scope. See section II substantiation) to our research question is that the outcome of the privacy impact assessment see Fig. 1. In this paper, we the privacy impact assessment for most of the interviewed concentrate on the influence of privacy impact assessment on organisation will lead, at best, to a product or system that is information system development. compliant with data protection regulation. It will not lead to a product or system that is privacy-friendly, or one that takes into Fig. 1. Privacy impact assessment (PIA) in relation to product and system development account social norms regarding the processing of personal information. We conclude this paper with suggestions for further research on this topic (see section VII). II. DATA PROTECTION OR PRIVACY In this paper, we do not only take the legal requirements on data protection into account, but also the social norms (values/expectations) regarding the processing of personal data. This broadening of the scope is prompted by Wright’s definition of privacy impact assessments and the concerning article in the Regulation which mentions that (representatives) of the data subject (the person about whom personal data is processed) can be consulted during such a privacy impact assessment. Also, this approach is inspired by the fact that non-compliance with societal values may lead to significant negative publicity. For example, in the Netherlands social indignation arose in 2014 when Equens (a payment service provider) launched the idea to sell the payment transaction information of customers. The same occurred in 2014 when ING Bank wanted to do a pilot in which it would offer personalised third-party ads to their customers (with their consent) based on their individual spending patterns. As mentioned earlier a privacy impact assessment is a Both ideas were formally compliant with the Dutch Data process for assessing the impacts on privacy of a product or Protection Act. service, and for taking remedial actions as necessary in order to avoid or minimize negative impacts. These remedial actions can Because we not only take into account the legal requirements be 2 taken into account when implementing technical and regarding data protection but also social norms and expectations organisational measures to ensure a level of protection we use the terms privacy impact assessment and privacy by appropriate to the risks of infringement on the rights and design instead of the terms used in the Regulation such as ‘data freedoms of natural persons. Roughly one can distinguish the protection impact assessment’ and ‘data protection by design’. following three phases in a privacy impact assessment: 1) collect III. PRIVACY BY DESIGN AND PRIVACY IMPACT ASSESSMENTS the necessary information, 2) determine privacy risks and 3) propose mitigating measures to avoid or reduce the determined Privacy by design is intended to improve overall privacy privacy risks. The outcome is normally documented in a report. friendliness when designing an information system. The That report can be used both as input for the concept fundamental principle of privacy by design is that privacy development and analysis phase of the system development requirements must be taken into account throughout the entire lifecycle, as well as for the testing and evaluation phase of that system development process. Privacy is a core property of a cycle. The use of the report in the latter phases helps to system that is heavily influenced by the underlying system determine if the countermeasures ultimately chosen during the design. As a consequence, privacy by design cannot be implementation phase have indeed eliminated or mitigated the implemented as an add-on [3]. Traditionally, privacy by design initial identified privacy risks. We did not assess the quality of is linked to the system development process. We believe, the outcome of the privacy impact assessment. however, that the ‘cradle to grave’ philosophy of privacy by design means we should not start thinking about privacy in the IV. RESEARCH METHODOLOGY first phases of the system development process, but in fact We performed a descriptive field study in the Netherlands already in the initial phase of the product development process. among fourteen organisations between late 2015 and mid 2016. After all, the development of an information system is not a goal The selected organisations are distributed across eight sectors in itself but supports a product or a service. When, for instance, (see Table I) with different data subject categories (e.g. the outcome of the initial privacy impact assessment, as part of consumer, passenger, patient, civilian) and different sizes of the scoping phase of product development, is taken into account organisations. In this way, we gave preference to a wide variety when building the business case an informed decision can be of sectors above the ability to compare results per sector. made. Therefore, the privacy impact assessment can and should provide input for both development processes, which blend into each other. For a graphical representation of our positioning of 2 One of the amendments of the European Parliament on the proposal for the of the Regulation. The final text of the Regulation merely mentions that a Regulation was that the output of the privacy impact assessment needs to be privacy impact assessment needs to be conducted where the type of taken into account. This amendment has not been adopted in the final version processing is likely to result in high privacy risk. TABLE I. DISTRIBUTION SELECTED ORGANISATIONS OVER SECTORS D. Results from the PIA (PIA and PbD) 1. How do you determine that the output of the PIA is used for Sectors3 Number of concept development and analysis (information system selected development)? Section Description organisations i) If the output is used, how is guaranteed that the results of the PIA are known and used by the IT-department? C Manufacturing 2 ii) If not why? What do you need? J Information and communication 2 2. How and when is monitored if the mitigating measures of PIA are H Transport and Storage 2 implemented during the development phases? K Financial and insurance activities 1 3. Did the outcome of the PIA resulted in changes in the (specs of M Professional, scientific and technical activities 1 the) information system. N Administrative and support service activities 1 E. Consultation with stakeholders O Public administration and defence 3 1. Who are the stakeholders? Q Human Health and social work activities 2 2. Are the results of the PIA consulted with stakeholders? Which stakeholders? If not, why not? F. Governance PIA We interviewed the data protection officers (or someone 1. Is the quality of the PIA assessed? By whom? with an equivalent role) of each of the fourteen organisations 2. Is somebody assigned to manage the PIAs (e.g. the privacy using a predefined survey. We did not question or discuss the officer) answer (to prevent bias), apart from asking for clarification 3. Are PIAs periodically revised (is this an obligation)? when the answer was not clear. At the time of the interviews the Data Protection Directive Table 2 describes the questions used during the interviews to [7] was still in force and implemented in the Netherlands verify our assumption that the outcome of the privacy impact through the Dutch Data Protection Act [8]. Under that assessment should lead to sufficient and adequate input for legislation, the conduction of a privacy impact assessment is designing privacy-friendly products and systems that comply only obliged for some types of processing of personal data by with privacy regulations and social norms. This is why, it is in public authorities. The European General Data Protection our opinion necessary to get insight into why organisations Regulation was not finalised yet. Only the proposal [9], the conduct privacy impact assessments, what their definition of position paper and amendments of the European Parliament [10] privacy risk is, what their strategies of reducing privacy risk are, and the position paper of the European Council [11] were when and how the assessments are conducted, whether the published. organisation scales the assessment (small/full) depending on the phase of development and/or the type of data processing, who TABLE II. SURVEY QUESTIONS the stakeholders are, and how the quality is assured. We also A. Why and when to conduct a PIA wanted to gain insight into how organisations use the output of 1. How do you define PIA? Has the definition been published? the privacy impact assessment for privacy by design. 2. Why do you conduct a PIA? 3. Since when has your organisation conducted PIAs? V. RESEARCH RESULTS 4. How many PIA’s are conducted in your organisation? In this section, we present and discuss the outcome of our B. How to conduct a PIA survey. We do this treating for each of the six topics separately. 1. Can you describe how a typical privacy impact assessment is initiated and executed within your organisation? For each topic, we first present a summary of the responses for 2. In which cases does your organisation conduct / not conduct a PIA each of the questions that belong to that topic. We then follow (is there a threshold)? through with our analysis of that topic: we compare the outcome 3. Is there a guideline how to conduct a PIA? On which methodology of our interviews with the theory (especially the work of Wright or standard is it based? and De Hert [6] [12] [13] [14]), our own expectations and the 4. Has the PIA been built into the project management of another relevant articles and recitals of the Regulation. The latter to business process? 5. Who conducts the PIA (an individual or a team, which functions determine what the selected organisations need to take to are represented)? “migrate” from the current practice to the practice they have to 6. In which phase or phases in the product and/or information system comply with in the near future. development is the PIA conducted? 7. Is there one questionnaire for all data processes or is it tailor made A. Why and when to conduct a privacy impact assessment (e.g. depending on the development phase or depending on 1) Questions and answers standard or tailored software)? C. How to determine privacy risk and measures • How does your organisation define a privacy impact 1. How do you define privacy risk? assessment? Has the definition been published? Most 2. How are privacy risks determined/identified in a PIA organisations defined the privacy impact assessment (automatically/manually)? as a tool/process to determine whether there are 3. How does your organisation cope with reducing privacy risk privacy risks, how big they are and to provide (strategy)? recommendations for mitigating measures. According to these organisations, the definition used was described briefly in the privacy impact assessment- documentation. In a few cases the privacy impact 3 The section and description of each sector is taken from the International Standard Industrial Classification (ISIC) of the United Nations [18] assessments were an integral part of the system of the processing. For processing likely to result in a high risk to development process and were not treated and thus not the rights and freedom of natural persons organisations have to documented separately. carry out the assessment. The Regulation stipulates that the • Why does your organisation conduct a privacy impact assessment shall in particular be required in the case of a) assessment? Most organisations conducted a privacy automated processing (including profiling) on which decisions impact assessment because they thought it was are based that produce legal effects concerning natural persons; mandatory for them. In a few cases it was mentioned b) processing on a large scale of special categories of data or of that the assessment was conducted to prevent the loss personal data relating to criminal convictions and offences; and of customer trust or to prevent an inappropriate c) a systematic monitoring of publicly accessible area on a large infringement on the personal life of the customer. scale (art. 35 par. 3 GDPR). • Since when has your organisation conducted privacy B. How to conduct a privacy impact assessment impact assessments? Most of the organisations started 1) Questions and answers conducting privacy impact assessments in 2012-2013, • Can you describe how a typical privacy impact some in 2006-2010 and one organisation as early as assessment is initiated and executed within your 2002. organisation? Almost all organisations executed the • How many privacy impact assessments are conducted privacy impact assessment more or less the same way. in your organisation? Most organisations had no They started by gathering the necessary information (central) database with all conducted privacy impact for the assessment (mostly through a questionnaire). assessments and had to make an estimation. The Based on that information the privacy risks were amount varied from 15 to 550. Most organisations determined and mitigating measures were proposed to only conducted privacy impact assessments on new or and agreed to be implemented. Within some revised systems. Others also conducted the organisations the residual privacy risks that remain assessments on existing systems because they did not because not all measures were implemented must be do it in the past and now wanted to have insight into approved by senior management. the privacy risks the organisation could face. • In which cases does your organisation conduct / not 2) Main findings - Why and when to conduct a privacy conduct a privacy impact assessment (is there a impact assessment threshold)? Most organisations conducted the privacy Under the current data protection legislation most of the impact assessment for each system in which personal selected organisations, except for governmental authorities data was processed: there was no real threshold. Some under certain circumstances, are not obliged to conduct a organisations used the amount of financial investment privacy impact assessment. Nevertheless, most data privacy for the new/changed information system as threshold officers mentioned that it is mandatory. This obligation can be to determine whether a privacy impact assessment was stipulated in the Binding Corporate Rules4 or other Group policy needed, for example investments worth over 1 million rule that some of the organisations have implemented. Others euros. Some other organisations performed a pre-scan, wrongly perceived it as an obligation. Although a privacy which provided a preliminary determination whether a impact assessment should be more than simply a compliance privacy impact assessment was required. check, it does nevertheless enable an organisation to • Is there a guideline for how to conduct a privacy demonstrate its compliance with privacy legislation in the impact assessment? On which methodology or context of a subsequent complaint, privacy audit or compliance standard is it based? Most organisations had some investigation. A privacy impact assessment enhances informed kind of guideline or framework for conducting privacy decision-making and exposes internal communication gaps or impact assessments. There was no uniformity at this hidden assumptions about the project [6]. point. For governmental authorities the “Framework Because there was no real obligations to conduct privacy privacy impact assessment Dutch National impact assessments for most of the selected organisations we Government” [15] was required in case of new or expected that data protection officers would mention reasons for revised legislation that results in the collection or conducting the assessment spotting potential privacy problems processing of personal data, and for large IT projects. and taking effective countermeasures (early warning), Some organisations used the privacy impact avoidance of inadequate solutions, avoidance of negative public assessment framework of the NOREA [16] (the reaction or loss of trust and reputation, avoidance of unnecessary professional association for IT costs or education, raising awareness about privacy among auditors in the Netherlands). Some used the employees or gaining competitive advantage [14]. This was not frameworks (incl. questionnaires) of the law firms that the case, however. helped them with implementing Binding Corporate Rules and others developed their own framework. Under the upcoming Regulation conducting a privacy • Has the privacy impact assessment been build into the impact assessment will be mandatory, dependent on the nature project management of another business process? 4 ‘Binding corporate rules’ means personal data protection policies which are or processor in one or more third countries within a group of undertakings, adhered to by a controller or processor established on the territory of a or group of enterprises engaged in a joint economic activity (art. 4 par. 20 Member State for transfers or a set of transfers of personal data to a controller GDPR). Almost all organisations said that the privacy impact of the organisations had different questionnaires assessment was part of a larger assessment. In order of depending on whether the product/service would be occurrence (from many to few) the privacy impact supported by standard software or tailored software. assessment was part of: compliance, project delivery, 2) Main findings – How to conduct a privacy impact information security and business impact assessment. assessment The credo of one of the data protection officers is “to Most of the data protection officers of the selected organisations burden the organisation as little as possible by ‘free- conduct privacy impact assessments in more or less the same riding‘ on existing procedures”. way and for all processing with one questionnaire. The • Who conducts the privacy impact assessment (an assessment is, with a few exceptions, conducted early in the individual or a team; which functions are development process. The threshold to conduct an assessment or represented)? More than half of the organisations nor is the question whether personal data is processed or not. conducted the privacy impact assessment through This is not appropriate. First, the degree of risk created by several bilateral consultations between the data projects varies enormously. Second, projects vary widely – from protection officer/privacy advisor and other officers of updating a small database to implementing new legislation, or that organisations (business owner, senior staff, developing a new product or service. Some authors recommend analyst (business/infra), information security officer, that organisations conduct a limited preliminary evaluation, to lawyer, etc. The remaining organisations conducted establish whether the organisation needs to invest in a small- the assessment with a team of which the data scale or a full-scale privacy impact assessment [17]. The protection officers/privacy advisor is a (supporting) scalability of the assessment and thus questionnaire should in team member. The size of the team depended on the our opinion also depend on the phase of the development project, and typically consisted of the aforementioned process. Up front, we expected that different questionnaires other officers of the organisation. In some would be used in different phases of development or that the organisations there was a strict separation between the questionnaire had separate sections for the different phases. This monitor compliance-task and the advisory-task of the is required to steer the process. An “initial” privacy impact data protection officer. The data protection officer assessment would be conducted during product development monitored compliance and the privacy advisor and the first phase of system development (concept advised. When a privacy advisor was appointed, he or development) to determine if the project is even viable taking she participated in the privacy impact assessment and privacy risks into account. During the development process the the data protection officer revised it. initial privacy impact assessment could then be supplemented • In which phase or phases in the product and/or with a ’follow-up‘ version. information system development is the privacy impact All selected organisations check at the end of the development assessment conducted? Almost all data protection process (test and evaluation) whether the agreed upon measures officers mentioned that they intend to conduct the are indeed implemented. In that phase, the data protection privacy impact assessment in the early phases of officers do not re-assess the privacy impact assessment. Privacy system development. The problem was that it was not risks could have changed or new risks may appear as a result of always common practice for project managers to design and/or implementation decisions. A re-assessment consult the data protection officer about a new project. should therefore be carried out. (See Fig. 1 for a graphical Within some organisations, it was a requirement that representation for the relationship between these three types of the privacy impact assessment had been conducted privacy impact assessments and the other product and system before the development could continue (this was part development phases). However, as mentioned earlier, Wright of a gateway review). Although it could take several states that the privacy impact assessment should be regarded and meetings to complete a privacy impact assessment, it carried out as a process and not just as a single task that results was not a dynamic process for these organisations. It in the completion of a report [14]. Based on our interviews we was conducted in a specific moment (phase), not over conclude that this process-oriented approach needs further a period of time. A few organisations followed a improvement in organisations. process oriented approach, where they started during product development and supplemented the An organisation should determine the roles and assessment during the system development. responsibilities of its officers with regard to privacy impact • Is there one questionnaire for all data processing or is assessment, for example who initiates one, who carries it out and it tailor-made (e.g. depending on the development who approves them. A team of experts, including external ones, phase or depending on standard or tailored software)? might be necessary. The privacy expertise is crucial here but it Almost all organisations used one questionnaire for all does not exclude other fields. Outsourcing the privacy impact phases and for all types of personal data or data assessment in full is not desirable. The line manager should be subjects. Some organisations used different types of responsible for conducting the assessment because, first and foremost, she is accountable for the risks posed by her frameworks depending the kind of data processed and thus different questionnaires. One organisation used a products/services. Secondly, she knows the product/service well master privacy impact assessment for the repetitive and hence should be able to tell where the main risks are. Finally, part of projects and used an addition privacy impact doing a privacy impact assessment internally would help to assessment for the unique parts of the projects. None create privacy awareness throughout the organisation [14]. In our opinion these reasons also favour the team based approach over of the bilateral approach. In the latter, there is a risk that the measures mitigating the risk instead of avoiding the risk; line manager no longer feels accountable anymore for the especially in organisations that define privacy risk from the privacy risks posed by her products/services. The data protection perspective of the controller. This is understandable (but not officer faces the risk that accountability is shifted towards him. defendable). When the data protection officer defines privacy This is clearly undesirable. (Line) management is responsible risk as the risk of getting fined by the Supervisory Authority he and the data protection officers provides advice where requested will look at the effect of a privacy risk instead of the cause. as regard to the privacy impact assessment and monitors its When you subsequently determine measures to reduce the performance pursuant the requirements mentioned in the Article privacy risk –bearing in mind the effect of the privacy risk– you 35 GDPR. are more likely to start thinking in terms of measures to reduce the risk of non-compliance. When you determine measures – C. How to determine privacy risks and measures bearing in mind the cause of the privacy risk– you probably start 1) Questions and answers thinking in measures that reduce the inherent risk, i.e. the cause. • How do you define privacy risk? In most cases This does not mean that in all cases the ultimately chosen privacy risk was defined from the perspective of the solution will be sought in avoiding privacy risks. See Fig. 2 for controller, i.e. unlawful processing of personal data a graphical representation. resulting in high fines of the Supervisor Authority and Focussing on the risk to the controller will lead at best to loss of reputation. In a few cases the risk was products or systems that are compliant with data protection perceived primarily from the perspective of the data regulation, but the resulting system may not always be privacy- subject, e.g. infringement on the personal life of the friendly. data subject, resulting in loss of trust of the customer which could cause loss of market share. In these cases Fig. 2. Layers of privacy risk possible fines were only secondary. • How are privacy risks determined/identified in a privacy impact assessment (automatically/ manually)? Within almost all organisations the privacy risks were determined manually (mostly supported by the data protection officer/privacy advisor). A few organisations used a mechanism which determined possible risks and mitigating measures automatically. The organisations that used privacy advisors mentioned that the quality of the determined the privacy risks was very dependent on the skills and experience of the person determining that risk. The data protection officers who were interviewed perceive the process of deriving privacy risks based on the filled-out questionnaire as vague. One of the data protection officers compared it to a black-box. D. Results privacy impact assessment • How does your organisation cope with reducing 1) Questions and answers privacy risk (strategy)? Most data protection officers • How do you establish that the output of the privacy mentioned that their organisation did not had a general impact assessment is used for concept development strategy for reducing privacy risks. When asked to and analysis (information system development)? If the give examples of solutions to reduce the privacy risk, output is used, how is guaranteed that the results of the the organisations that defined the privacy risk from the privacy impact assessment are known and used by the perspective of the controller tended to favour measures IT department? If not why? What do you need? Most that mitigate the risk (e.g. encryption or access organisations (in the person of the project owner, data management) instead of avoiding risks (e.g. protection officer, information security officer, pseudonymisation or data minimisation). executive management, etc.) agreed to implement the 2) Main findings - How to determine privacy risks and measures proposed in the privacy impact assessment. measures In the organisations where information security officer In the Regulation “data protection risk (privacy risk)” is not was involved the data protection officers believed that defined. The corresponding article about privacy impact the measures were more likely to be developed. The assessment only mentions “…the rights and freedoms of natural project owner was ultimately responsible for persons…”. This indicates that, from the point of view of the implementing the agreed measures. Regulation, the data subject perspective is more relevant than • How and when do you monitor whether the mitigating the controller perspective. The process of determining risks and measures of privacy impact assessment are measures is not well defined, and no guidance is provided. As a implemented during the development phases? As part result, the quality of it very much depends on the person of the information system design cycle the developed performing the privacy impact assessment. It is a black box. In system was tested to determine whether it is built in addition, solutions to reduce the privacy risk are sought in conformance with the specifications (including the ones from the privacy impact assessment). The test F. Governance privacy impact assessment team gave a "go/no go". Sometimes the project owner 1) Questions and answers must sign off explicitly that the measures of the • Is the quality of the privacy impact assessment privacy impact assessment had been implemented; assessed? By whom? The quality of the privacy impact otherwise the project would be placed on hold. assessment was secured through the participation of • Did the outcome of the privacy impact assessment experts in the team. If privacy advisors were used the result in changes in the (specifications of the) data protection officer typically reviewed it. In some information system. As a result of the privacy impact organisations, the report was signed off by key parties assessments personal data was better secured, in some (like applicable line manager, data protection officer, cases less personal data was collected and in other less information security officer and depending on the personal data was presented (e.g. on screens and residual risks also executive management). This not letters). Besides the specific improvements in only improved the involvement of the key parties but information systems, conducting privacy impact also the quality of the report. Little or no auditing of assessments resulted in enhancing awareness of data the privacy impact assessment was performed. protection throughout the organisation. • Is somebody assigned to manage the privacy impact 2) Main findings - Results from the privacy impact assessments? Among the selected organisations there assessment was no common understanding. The following people As part of the information system design cycle the developed were mentioned as being responsible: the product system is tested to verify that it was built in conformance with owner, the data protection officer, the chief its specifications. As mentioned earlier, the data protection information officer, risk management department. officers should re-assess the privacy impact assessment during • Are privacy impact assessments periodically revised the 'testing and validation'-phase because privacy risks could (and is this an obligation)? About half of the have changed or new risks may appear as a result of design organisations did not specify conditions for revising a and/or implementation decisions. privacy impact assessment. The other organisations E. Consultation with stakeholders had explicit conditions for reassessment of the impact of privacy risks (every two to three years, or earlier in 1) Questions and answers case of large changes). In one case the revision of the • Who are the stakeholders? The data protection officers privacy impact assessment was part of a certification mentioned departments/ officers within the program for that information system (5 years). organisation as stakeholders. The ultimate stakeholder, the data subject was hardly mentioned. Only when the data processing involved personnel, the 2) Main findings – Governance privacy impact assessment working counsel was mentioned as stakeholder. As seen earlier, in most organisations the roles and responsibilities involved in conducting privacy impact • Are the results of the privacy impact assessment assessments are described. But managing the life cycle of the consulted with stakeholders? Which stakeholders? If privacy impact assessment is not. At best a revision term is not, why not? The results of the privacy impact specified. This needs to be improved. assessment were only shared with the involved officers within the organisation; not everyone within VI. CONCLUSIONS the organisation had access to (a subset of) the report. We conducted a field study regarding the use of privacy None of the selected organisation published (a subset impact assessments in practice in the Netherlands. The main of) the privacy impact assessment report externally. results of our study are the following: Only one case involved data subjects. This organisation involved customers for improving the • Most of the data protection officers who were interviewed quality/friendliness of the consent notice in an UX-lab perceive wrongly that they are obliged to conduct a privacy to achieve a higher consent rate of their customers as impact assessment. The European Data Protection Directive legal grounds for processing personal data. (which was in force at the time we performed our study) does 2) Main findings - Consultation not mention such an obligation at all. The upcoming The data subject is one of the stakeholders of the privacy European General Data Protection Regulation stipulates that impact assessment-process whose remarks must be taken into only in circumstances where the processing is likely to result account [6]. Even the selected organisations that use customer in high risks to the rights and freedoms of natural persons panels for judging new products/services did not seek does an assessment need to be carried out. consultation with the customer or their representatives about • Most organisations use an uniform approach (incl. one their perceived privacy risk, and which mitigating measures are questionnaire) for assessing all data processing, regardless or are not acceptable. Based on the Regulation, the controller of the type of processing and the type of project. Based on shall, where appropriate, seek the views of the data subject or existing research a preliminary evaluation was expected to their representatives on the intended processing. determine whether to conduct a small-scale or full-scale privacy impact assessment. • Most organisations conduct the privacy impact assessment REFERENCES at one phase during system development (in the early ` phases) but they do not supplement the assessment during [1] A. Cavoukian, “Privacy by design,” Office of the Information and the development process. Existing research states that the Privacy Commissioner of Ontario (IPC), Ontario, 2009. assessment should be regarded as a process, and not just as [2] EC, “Regulation (EU) 2016/679 of the European Parlement and of the a single task. Counsil on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (L119/1),” vol. • Most data protection officers define privacy risks from the L119/1, 2016. perspective of the controller (the risk of getting fined by the [3] J.-H. Hoepman, “Privacy Design Strategies,” IFIP SEC, pp. 446-459, Supervisory Authority) instead of the perspective of the data 2014. subjects. This is not in accordance with the spirit and the [4] M. Colesky, J.-H. Hoepman and C. Hillen, “A Critical Analysis of legal requirements specified in the Regulation. Privacy Design Strategies,” 2016. • When reducing the assessed privacy risks most organisations [5] N. Notario, A. Crespo, Y.-S. Martín, J. M. d. Alamo, D. L. Métayer, T. favour measures that mitigate risks, instead of measures that Antignac, A. Kung, I. Kroener and D. Wright, “PRIPARE: Integrating avoid them. Privacy Best Practices into a Privacy Engineering Methodology,” in IEEE CS Security and Privacy Workshops, 2015. • Most organisations do not consult (representatives of) the [6] D. Wright, “The State of the art in privacy impact assessment,” data subjects as part of the privacy impact assessment Computer Law & Security review, vol. 28, pp. 54-61, 2012. process. Consultation is advised by a number of authors [6] [7] EC, “Directive 95/46 EC of the European Parliament and of the Council [14] [17], and the Regulations also stipulates that “where of 24 October 1995 on the protection of individuals with regard to the appropriate, the controller shall seek the views of the data processing of personal data and the free movement of such data,” vol. subjects or their representatives on the intended processing”. L281:31. • The process of determining privacy risks, based on the [8] DP, “Ducth Data Protection Act (Transl. Wet bescherming persoonsgegevens),” Dutch Official Gazette, vol. 302, 2000. information gathered about a specific product or system, is perceived as vague and its quality is very dependent on the [9] EC, “Proposal for a Regulation of the European Parliament and of the Council on the protection of individuals with regard to the processing of person who assesses the privacy impact assessment. personal data and on the free movement of such data,” vol. COM(2012)11, 2012. Most of the participating organisations were highly controller-oriented instead of data subject-oriented when [10] EC, “EP legislative resolution of 12 March 2014 on the proposal for a regulation of the EP and of the Council on the protection of individuals considering privacy risks. This was apparent from the reasons with regard to the processing of personal data and on the free movement for conducting privacy impact assessments and the definitions of such data (GDPR),” vol. P7_TA(2014)0212. of privacy risk given by the data protection officers, the [11] EC, “Position of the Council of 19 December 2014 on the proposal for a proposed measures for reducing the privacy risk, and the regulation of the EP and of the Council on the protection of individuals practice of not consulting (representatives of) the data subject as with regard to the processing of personal data and on the free movement stakeholders. These organisations tend to look at the effect rather of such data,” vol. Doc.15395/14. than the cause of a privacy risk. When the outcome of a privacy [12] D. Wright, K. Wadhwa, P. D. Hert and D. Kloza, “A Privacy Impact impact assessment by these highly controller-oriented Assessment Framework for data protection and privacy rights - organisations is used to implement the principles of ‘privacy by Deliverable D1,” Brussels, 2011. design’, this will lead at best to a product or system that is [13] G. Hosein and S. Davies, “A Privacy Impact Assessment Framework for compliant with data protection regulation. It will not lead to a data protection and privacy rights - Deliverable D2 (Empirical research of contextual factors),” Brussels, 2012. privacy-friendly product or system and/or one that takes into [14] P. d. Hert, K. Daiusz and D. Wright, “Recommendations for a privacy account social norms regarding privacy. impact assessment framework for the European Union - Deliverable D3,” Brussel, London, 2012. VII. NEXT STEPS, FURTHER RESEARCH [15] Rijksdienst, „Framework privacy impact assessment Dutch National A more rigorous and transparent process for determining Government (Transl.Toetsmodel Privacy Impact Assessment (PIA) privacy risks that can be used by organisations in practice needs Rijksdienst),” juni 2013. to be developed. Data subject risks, instead of controller risks, [16] NOREA, “Priacy Impact Assessment; Introduction, Guidance and should be central. And these risks should be avoided instead of Questionnaire (Transl. Privacy Impact Assessment; Introductie, merely being mitigated: the output of a privacy impact handreiking en vragenlijst),” 2015. assessment should steer the initial system design. In fact we [17] A. Warren, R. Bayley, C. Bennett, A. Charlesworth, R. Clarke and C. believe the privacy impact assessment process and the resulting Oppenheim, “Privacy Impact Assessments: International experience,” Computer Law& Security Report, vol. 24, pp. 233-242, 2008. privacy by design process should be integrated into a single methodology (what we call a Privacy Impact Reduction [18] UN, “International Standard Industrial Classification of All Economic Activities (ISIC), Rev. 4,” United Nations Publication, New York, 2008. Methodology) that fosters the development of truly privacy- friendly products and systems that, by default, comply with both data protection regulations and social norms.