It is possible to model the meaning of articles of the GDPR in logic formul and this enables a semi-automatic reasoning over the Regulation, e.g., to build an argument of compliance. However, any formal reasoning requires that the formul are validly expressing the legal meaning(s) of the articles, including potential disagreements between legal experts over their interpretation. The problem is that IT experts may anticipate some uno cial legal meaning, invalid under any interpretation, while verifying if this happens requires legal expertise. However, legal experts are unlikely familiar with the logic formalism and cannot give informed feedback without understanding the legal interpretation(s) that a formula embodies. On a previous work, we devised a methodology and a human-readable intermediate representation to help non-experts reading formul in Rei ed I/O logic (RIO), a formalism expressing GDPR provisions introduced to reason about data protection compliance. This paper validates the methodology and shows that is possible to retrieve feedback from legal experts about the validity the RIO representation of the Regulation. Precisely, we collect and evaluate the feedback on the RIO version of Art. 5.1a and Art. 7.1, and show how to elicit suggestions to improve the formalization thereof. What emerges is an agile process to support public trust in the formal framework and in its use.
The processing of personal data { and therefore data protection legislation { is an essential facet of a modern economy. Personal data is widely associated with a modern business model made of Internet-based services o ered free of charge and whose revenues come from the collection, the processing and the use of personal data for advertising purposes, but it comes into play also with more traditional business models, such as the product market, which are evolving to e-commerce, or tailoring their o ers to individual customers.
The legal landscape of the protection of personal data within and outside the
European Union (EU) has been redesigned by the amplitude of the material and
territorial scope of the General Data Protection Regulation (GDPR) [
This new landscape, coupled with the heavy nes that supervisory authorities are entitled to issue in case of violation, calls for a need to ensure that data processing activities (and tools thereof) comply with the GDPR. Data controllers and processors could therefore take advantage from tools helping design and verify compliance, thus diminishing their risks of violating provisions and incurring into nes. Such tools could also help companies build, in an automated or computer-assisted way, their case in response to one-time casuistic decisions emanated by supervisory authorities and courts against them.
A critical facet for such automation is to be able to build executable rules
for a computer-assisted assessing compliance. In previous research, the authors
have proposed a complete model of the GDPR for legal reasoning and legal
compliance [
However, as is widely acknowledged in literature [
The concept of validation refers to the determination of the correctness of the
system with respect to user needs and requirements [
As \algorithmic representations of law are typically very poor as regards
their transparency", \one cannot begin to devise an algorithm to apply legal
provisions without determining rst its intended purpose and by whom it will
be used" [
In the particular domain of modelling the GDPR, a considerable e ort has
been reserved to represent the Regulation's provision in a logic formalism,
precisely the Rei ed Input/Output (RIO) logic [
Pursuant to this, we discussed in [
The customizable human-readable representation has been assessed as understandable, increasing our con dence on it being an eligible candidate to validate the formalized GDPR articles. In this work we proceed further and show that the methodology is e ective in gathering feedback of legal experts on the legal validity of the representation of the GDPR articles, so as to provide quality assurance of our methodology as a whole.
Some discussion within the AI and Law community [
A set of six categories was compiled to de ne the broad types of evaluation found therefrom. They include the following assessments: i. Gold Data: evaluation performed with respect to domain expert judgments (e.g., classi cation measurements or measures on accuracy, precision, recall, F-score, etc.); ii. Statistical : evaluation performed with respect to comparison functions (e.g., unsupervised learning: cluster internal-similarity, cosine similarity, etc.); iii. Manual Assessment : performance is measured by humans via inspection, assessment, review of output; iv. Algorithmic: assessment made in terms of performance of a system, such as a multi-agent system; v. Operational-Usability : assessment of a system's operational characteristics or usability aspects; vi. Other : those systems with distinct forms of evaluation not covered in the categories above (task-based, conversion-based, etc.). In our case, we combined the following types of evaluation: gold data (i.), manual assessment (iii.) and operational-usability (v.).
Some authors [
The validation phase of legal modeling by domain legal experts { driven by
operational usability assessments { is also mentioned in the methodologies
referring to ontological expert knowledge evaluation. For example, the Methodology
for Modeling Legal Ontologies (MeLOn) [
Although the legal formal framework target of this work's analysis (i.e., the DAPRECO Knowledge Base) refers and is strictly bound to a legally validated ontology (i.e., the PrOnto) an argument for its legal validity cannot follow only from the validity of the ontology of reference. It requires a more comprehensive analysis and we believe that both qualitative evaluation methodologies and certain criteria from the CCCF are required. Ontologies are in fact about concepts, data, and entities and any validation strategy of them is inevitably about assessing the legal qualities of those objects. Formal models for legal compliance, such as the DAPRECO Knowledge Base, model also the logical and deontic structure of a legal text, its temporal aspects and, as when the formalism allows multiple con icting interpretations, as the DAPRECO Knowledge Base does, it includes structural elements to allow defeasible reasoning. The validation assessment should take these elements into account.
Thus, the necessity of an integrated approach, which additionally should also acknowledge an operational-usability assessment, since the legal validity of the DAPRECO Knowledge Base logic formul have to be validated by people who are not experts in logic. 3
This work leverages on our previous work [
Empirical validation using tailored constructed questionnaire is a very
useful quantitative indicator of user acceptance [
Does the deontic modality (obligation/permission/other) of the forq1 mula coincide with the modality of the GDPR articles? q2 Does the formula capture all the important legal concepts? q3 Does the formula capture all the important legal relations? q4 Is the interpretation given by the model correct? q5 Is the interpretation complete? q6 Is the interpretation to the point?
They are tailored to check for the following legal validation qualities: Accuracy (q1); Completeness (q2 q4); Consistency (q5); and Conciseness (q6).
We now give two examples of human-readable representations accompanying the questionnaire. Both examples will be referred later in this paper.
\Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data"
Without entering into any detail on its modeling (the reader can refer to [
Names like nominates and PersonalData are either chosen by the IT expert in
charge of the modelling in logic formalism or are taken from the PrOnto [
Listing 1. LegalRuleML representation of formula 1 (snippet). <r u l e m l : R u l e c l o s u r e=" u n i v e r s a l "> < r u l e m l : i f> <r u l e m l : E x i s t s> <ruleml:And> <ruleml:Atom> <r u l e m l : R e l i r i =" rioOnto:RexistAtTime " /> <r u l e m l : V a r k e y r e f=" : a 1 " /> <r u l e m l : V a r key=" : t 1 ">t1</ r u l e m l : V a r> </ ruleml:Atom> . . . <ruleml:Atom> <r u l e m l : R e l i r i =" prOnto:DataSubject " /> <r u l e m l : V a r k e y r e f=" :w " /> </ ruleml:Atom> . . .
</ ruleml:And> </ r u l e m l : E x i s t s> </ r u l e m l : i f> <r u l e m l : t h e n> <r u l e m l : E x i s t s> <ruleml:And> <ruleml:Atom> <r u l e m l : R e l i r i =" rioOnto:RexistAtTime " /> <r u l e m l : V a r k e y r e f=" : e a " /> <r u l e m l : V a r k e y r e f=" : t 1 " /> </ ruleml:Atom> . . .
</ ruleml:And> </ r u l e m l : E x i s t s> </ r u l e m l : t h e n> </ r u l e m l : R u l e>
In [
PersonalDataProcessing (:ep) isBasedOn Purpose (:epu) THEN it must happen that, in at least a situation, { At time :t1, Controller (:y) is Obliged to AbleTo (:ea)
Controller (:y) Demonstrate (:ed) GiveConsent (:ehc)
The words capitalized and in bold are concepts from the PrOnto ontology. The words in bold non-capitalized are relations introduced by the IT expert. Although this format still requires some mental e ort to be read, it is at least human-processable;
The human-readable representation is built from the output of the parser through a manual post-processing. That of Article 7.1 is shown in Table 2.
There is a processing, which has a purpose authorized by a consent given by a data subject, and that is what a processor, whom a controller controlling the personal data nominates, does on personal data of the data of the data subject.
Whenever there is a processing, which has a purpose authorized by a consent given by a data subject, and that is what a processor, whom a controller controlling the personal data nominates, does on personal data of the data of the data subject then the controller is obliged to able to demonstrate that \data subject gave consent".
Example 2. We refer to Article 5.1(a) of the GDPR, which is worded as follows: \Personal data shall be a) processed lawfully, fairly and in a transparent manner in relation to the data subject (`lawfulness, fairness and transparency');"
The output of the re-interpretation of the RIO formula (here omitted for reasons of space), transform the LegalRuleML as follows:
IF, in at least a situation, { At time :t1, the following situation exists: (All of the following (:a1)) 1. The PersonalDataProcessing (:ep) is performed by Processor (:x) over the
PersonalData (:z) 2. Controller (:y) nominates (:edp) Processor (:x) PersonalData (:z) is relating to DataSubject (:w)
The Controller (:y) is controlling PersonalData (:z) THEN it must happen that, in at least a situation, { At, time t2, Controller (:y) is Obliged to (All the following (:a2)) 1. Controller (:y) Implement (:ei) Measure (:em) 2. The fact Measure (:em) is the cause of the fact lawfulness (:el) 3. The fact Measure (:em) is the cause of the fact fairness (:ef ) 4. The fact Measure (:em) is the cause of the fact transparency (:et) 5. Controller (:y) Describe (:ed) Implement (:ei) Thing (:t1) is greater than, or at least equal to, Thing (:t2) PersonalDataProcessing (:ep) respects the principle of lawfulness (:el) PersonalDataProcessing(:ep) respects the principle of fairness (:ef ) PersonalDataProcessing (:ep) respects the principle of transparency (:et)
Task PersonalDataProcessing (:ep) is RelatedTo DataSubject (:w)
Ontological [[IPneRrseolantailodnaTtoa]]tshhea[lDlbatea[ Psurbocjeecsts]ed(`][ Llaawwffuulllny,esfsa]i,rl[yFaainrndesins] aantrda[nTsrpaanrsepnatrmenacnyn]'e)r; Concepts Other Onto- [Processor]; [Measure]; [Nominates]; [IsBasedOn]; [Implement]; [Describe] logical Concepts Context Overall Meaning
There is a processing done by a processor on personal data of the data subject, and a controller, which is who controls the personal data; he nominates the processor. The controller describes (how) he implements a measure.
Whenever there is a processing done by a processor on personal data of the data subject, and a controller, which is who controls the personal data and who nominates the processor. Then before that moment must be that the controller describes (how) he implements a measure that is what causes lawfulness, fairness and transparency of the data processing related to the data subject. 4
Can a legal validator reading the human-readable representation give useful
feedback to the modeller? Answering this question is the goal of this paper.
Rephrased, the goal is to show that the \Validation" phase of the methodology
presented in [
The starting point is the human-readable representation of Articles 5.1(a) and 7.1 of the GDPR. The \Check" action (see Figure 1) has been implemented by gathering a set of four validators, all jurists knowledgeable of data protection law, and by asking them to answer questions q1 q6 in Table 1.
Evaluators were told to compare the meaning of the formul , as expressed in the human-readable representation of the RIO logic, with the legal interpretation that them legal experts would give to the original articles in the GDPR.
We also asked them a few questions meant to reveal how much
understandable for them is the human-readable format, before they start using it. General
understandability of the format has been discussed elsewhere [
Questions q1 q6 (see Table 1) are yes/no questions, but in the questionnaire we additionally invited our checkers to motivate their answers and to pinpoint further whatever observation they valued meaningful. We collected eight documents with such written answers and comments which we reviewed and summarized. The following table resumes the ndings, wherein we report the comments whenever the answer to the question was `no', indicating that someone found some pertinent issue.
For instance, all experts easily understood and con rmed the deontic modality and agreed on the formulas be capturing all the legal concepts and relations (see Table 5). But is from the analysis conferred to the provided comments that we are able o er a broader spectrum, for they refer to the above surveyed criteria and also to other (non-surveyed) related criteria. Comments { in Completeness like \it was complex to capture the legal concepts within the structure of the formula"; comments in Consistency like \It refers to the implementation and description of a measure that it is hard to understand; \It is redundant and restates concepts already present at previous articles", and comments in Conciseness like \'Obliged to be able to' sounds weirds" { clearly indicate uneasiness about the way in which the formula have been structured; such comments may lead to a better formalization, for instance, stating certain contextual facts as a common premise valid for all the GDPR's articles without repeating them each time in each article.
One evaluator, in particular, has mentioned \Interchanged roles for the controller and the processor" in Consistency. Even if that is stated in the Context of the human-readable table, the evaluator was probably induced in error/confused by the excess of information provided. Further analysis is of course required. Extracting from the non-structured comments valid input for the IT expert has to be left as future work, as we comment in the following section. 5
This paper leverages a methodology that advocates an interdisciplinary validation of a representation of the GDPR articles in a logic formalism (i.e., RIO logic) to pursue quality, accountability, and transparency within. One important output of the methodology is the production of feedback derived from the involvement of legal experts, while assessing the quality of the legal interpretation that IT experts may instill in the formalization of the GDPR. This work has gathered evidence that such step is feasible. As a proof-of-concept, a small number of legal experts has been asked to answer six questions with the purpose of collecting comments about how two logic formul , modelling Articles 5.1a and 7.1 of the GDPR, are complete, accurate, concise, and consistent in reecting the legal meaning of the articles. Several comments have been collected. Although a thorough analysis thereof requires more time { an involvement of a larger group of expert checkers is also advisable{ we were able to identify a few issues of relevance using which the IT expert can review the formalization work.
Several challenges await us in the near future. We need to improve scalability in producing a human-readable representation of the RIO formul : it is currently done manually, starting from the pre-processed version. This is already more readable than the original LegalRuleML version and give us con dence that the work to produce a natural language analysis break-up table can be automatized. This step done, a forth bringing process will consist in streamlining the validation of the RIO formalization of the GDPR as a whole. This likely requires to set up an application where the work of the IT expert can be suitably translated in to the human-readable format and o ered for on-line checking to a group of legal testers, which may also vary, providing feedback that the IT expert can take into consideration until a good quality of legal interpretation is assessed for the formul .
Concomitantly, there is a need to de ne together with the legal experts a more
complete set of qualities and possibly a few metrics, which we can quantify and
de ne criteria on the legal quality of the formalization. In Section 2 we pointed
out possible metrics, and in this paper we have assessed a few (completeness,
consistency, conciseness in Section 4), but a wide and systematic investigation
of the state-of-the-art in this topic has not been done yet. The quadrant criteria
presented in [