The work suggests using a network of semantically clear interconnected activities for a formal yet flexible definition of policies in data archives and data infrastructures. The work is inspired by needs of EUDAT Collaborative Data Infrastructure and the case of long-term digital preservation but the suggested policy modelling technique is universal and can be considered for all sorts of data management that require clearly defined policies linked to machine-executable policy implementations.
Problematics of advanced long-term digital
preservation [
One of the problems that long-term digital preservation aims to address is having clear policies for the entire data lifecycle from data ingestion by archive or by e-infrastructure service, through years-long data management with sensible data checks, transformations and moves, to data access and data dissemination to the end users.
One can argue that without clear data policies and
means of their validation there is no such a thing as the
long-term digital preservation, even in cases when a
technology foundation used for an archive or an
einfrastructure is sound and well-supported. At the end of
the day, every technology evolves – and at a brisk pace
compared to relatively long time when many data assets
are going to be useful, so data policies and means of their
expression should be semantically clear and in a way
more permanent than technology that underpins data
management. A strong case for policy-driven digital
preservation, with extensive references to the prominent
projects and popular methodologies was made in [
In practice, quite a few data archives and einfrastructures end up in a situation when they have got a sound technology for managing data bits, also acquire a decent number of users (which is a popular measure used by funders for their judgement on the e-infrastructure success) but do not have a reasonable data policy, let alone any machine-assisted reasoning over it. The users’ trust in the archive or the e-infrastructure may be enough for their daily use but there can be a substantial conceptual and technological gap in regards to data policies formulation, expression and execution.
Some larger projects and e-infrastructures are aware
of this gap and do make efforts to close it by working on
data policies implementation. An example of such
einfrastructure is EUDAT [
The prime candidate for applying data policies in
EUDAT is B2SAFE service [
This work presents an alternative approach to those
mentioned and is based on Research Activity Model [
The main advantage of this alternative approach is its high modularity which allows modeling policy elements and using them as building blocks for the semantically clear representation of a whole policy. The modularity of policy design is especially important in data infrastructures that commonly aggregate data coming from different user communities, often having their own business models, technical requirements, data formats and data lifecycles which makes it difficult to design and adequately express the crosswalks between communityspecific data policies and those for the data infrastructure. Another advantage of the suggested approach is its ability to address the conceptual gap between policy formulation and policy implementation, as it may not be easy to translate a high-level policy (often in a textual form) into machine-executable policy.
The modularity should allow high levels of inheritance and reuse of policy elements; it also helps to solve specific problems of policy formulation and validation when textually the same policy can be executed in different ways leading to different states of data archive, for which situation we provide an example. The conceptual gap between policy formulation and policy implementation is addressed by a possibility to define policy-related Activities as “black boxes” with (initially) only interfaces defined; this can be hopefully done by policy makers themselves without entirely delegating this policy design phase to policy implementers (software developers).
Implementation of a sensible data policy is a challenging task even within the boundaries of a particular organization. In a situation when the organization is using a collaborative data infrastructure along with its own organization-specific IT services, the implementation of a data policy is going to be even more intricate and is likely to rely on loosely coupled services. An approach to data policy modelling suggested in this work is going to address this challenge, along with the alleviation of the earlier mentioned problems of the policy elements reusability and the policy application results predictability.
The work is inspired by needs of EUDAT
Collaborative Data Infrastructure [
Conceptual challenges of data policy modelling are discussed first, specifically the problem of policy decomposition into policy elements, then an example is given of how Activity Model can be used for policy modelling. This is followed by suggestions on what IT architecture for data policy management will be required to support the suggested modelling techniques.
Data policy is often created as a conventional textual document that contains certain statements about what should or should not be done with data, with implied or sometimes explicit logical “ANDs” and “ORs” that glue statements together in an aggregated policy. This composite nature of policies is why it seems natural to break down the policy document into granular statements, model each statement using some formalism and then execute the statements using some IT solution.
One of the most advanced efforts on data policy
decomposition was performed by SCAPE project [
SCAPE stopped short of the actual implementation of
control policies, so when EUDAT [
Having well-defined control policies or practical policies is not enough though for semantically clear modelling of a data policy as a whole, as the application (execution) of a policy composed of granular machineexecutable statements may lead to quite different outcomes depending on the order in which granular policies are applied.
The problem of policy decomposition is in fact
interrelated with the problem of policy validation. To
illustrate this, let us consider a simple case when there is
a couple of easily identifiable policy statements
contained in the same policy document which we want to
decompose and validate through execution of two
granular policies. Let the statements in a composite
policy (perhaps, but not necessarily so, added one to
another through some policy update by different policy
managers) be:
[
If a certain file of type RAW is more than X gigabytes in size but becomes less than X when converted in JPG then, depending on the higher-level guiding policy and on the order in which these granular policies are applied in the actual service implementation, the result of the combined application of the two granular policies can be any of the following: 1. File is moved as RAW in storage A and remains stored in A as RAW. 2. File is moved as RAW in storage A then converted in JPG and remains stored in A. 3. File is converted in JPG and stored in B. 4. File is moved as RAW in storage A and remains stored in A as RAW; also a copy of it converted in JPG is stored in B.
This is to illustrate that validation of the data policy implementation is hard as any of the listed outcomes may be considered being right or wrong depending on the validator’s point of view.
Also let us take into account that policy validation can be based on some statistical selection of samples (so that problematic boundary cases of RAW data sized only slightly over X gigabytes threshold may not be selected in a sample and hence go unnoticed), or that a policy validation procedure allows some tolerance towards small amount of failed policy checks (so that even if a few files have ended up somewhere that a particular policy interpretation considers to be a wrong place, this does not trigger a policy violation alert).
So even if the data policy can be, seemingly successfully, decomposed into granular policies that are easy to define and validate as machine-executable statements, the actual result of the policy implementation does not necessarily match the intentions of policy designers or policy managers, as the backwards process of the policy composition – assembling it from the granular policies (policy elements) – can be performed with substantial variations. 2.2 Possible responses to the challenge of granular policies insufficiency
One possible response to the outlined challenge could be setting up an elaborated policy governance framework, i.e. well-defined business processes that allow human agents (policy managers) to look after the policy implementation, i.e. accumulate and analyse feedback from the environment where the policy is applied and supply the result of this analysis as updated requirements to software developers who work on the actual software implementation of the policy. This approach requires a good organizational culture and a substantial human resource involved in data policy management and in policy implementation; documented requirements will serve as an interface between policy managers and policy implementers. Some “magic” should happen in between so that high-level policy definitions translate into actual policies implementation in software code, this is why policy validation is likely to demand extensive software testing with specific policyrelated test cases.
Another possible response is having an elaborated
means of expression for the entire data policy (a
sophisticated policy modelling language): both for the
definition of granular policies and for the definition of
logic than binds the granular policies into the whole. An
example of this approach is RuleML [
The third possible response is that a certain formalism is used for the expression and, where necessary, recomposition of granular policies (policy elements) and for their assembling in the whole, with that formalism being reasonably friendly to machines as well as to humans. The humans – policy managers themselves or a not-so-skilled modeller – can use the formalism for a flexible policy definition that can be fairly easily modified depending on the true policy intentions and on the feedback received from the archive or e-infrastructure where the policy is implemented. The role of software developers is then to implement an engine for the formalism (quite similarly to the second approach). The machine just executes the policy expressed using that formalism.
The differences amongst approaches are presented in Table 1; in essence, they are different “weights” (different levels of demand) for the skills of policy managers, policy modellers and policy implementers.
The preferable approach could easily be the third one as it empowers policy modelers themselves with reasonable means of policy expression and therefore can reduce overheads and risks of communicating a policy from policy managers through modelers to implementers. A remote analogy of the third approach could be the proliferation of SQL language that, despite its sophistication, has become a lingua franca of not only software engineers but is widely used by logistics and even sales departments is all sorts of business.
The formalism to be used for data policy expression
should not be something as developed as SQL though,
neither should it be purely textual: it can be based on the
idea of “building blocks” with possible graphical
representation of them, hence providing an
easy-tooperate semantic wrapper for machine-executable
statements. On the other hand (unlike SQL which allows
the actual data manipulation), these “building blocks” for
data policy definition are likely to remain only a wrapper
to the actual machine-executable implementations of
granular policies which will be inevitably specific to a
particular service even within the same archive or
einfrastructure. As an example, for EUDAT B2SAFE [
This work strongly prefers the third approach and
suggests considering Activity Model [
Activity Model [
The main “building block“ of the Activity Model is an “activity cell” represented by Figure 1 with its aspects (that can be thought of as incoming and outcoming relations) explained in Table 2.
The full RDF serialization of the Activity Model is
published in [
Compared to modelling data policies with workflows,
the suggested approach based on the definition of
policyrelated Activities should allow more loosely coupled
implementations of policy management IT solutions. As
an example, the “data engineering” part of policy
implementation represented by Generic Data
Management Activity can be performed on a software
platform fully controlled by a specific user community or
organization (e.g. a research institution), the operation
(the actual execution of control statements) represented
by Control Activity can be performed by collaborative
data infrastructure (e.g. by EUDAT CDI [
If the policy was modelled by an executable workflow, it would require the presence of all three aspects: data engineering, reasoning and execution – in the same workflow likely operated by a single universal workflow engine. This would mean not only an operational limitation but a conceptual / modelling limitation, too, as all the participants (stakeholders) of policy implementation would have to adhere to the conceptual framework and the format required by the workflow engine. Modeling with interconnected Activities as semantic wrappers to particular implementations leaves more freedom to conceptualize and to operate data policies that are going to be executed by loosely coupled IT services.
Depending on a particular operational environment (software platform where policies are executed), other parts of the Activity Model, e.g. its Inputs, Outputs, or Conditions may require additional semantically clear extensions. However, it is unclear at the moment whether these potentially required extensions should be a part of the universal Activity Model profile for data policies, or it is better to introduce them as necessary, as parts of policy execution engine implementations on particular software platforms. 3.3 Examples of the Activity Model data policies profile application
The role of the suggested model extensions will be clearer by giving an example of their application to the modelling of a particular policy. The example will be a policy with two granular statements about data movements depending on data size and data format that were considered in Section 2.1.
We will need to define first a File Characterization Activity: @prefix am: <http://.../stuff/ActivityModel#> . @prefix ampp: <http://.../ActivityModel#PolicyProfile> .
GDMA_FileChar a ampp:GenericDataPolicyActivity GDMA_FileChar am:hasInput File GDMA_FileChar am:hasOutput FileSize
GDMA_FileChar am:hasOutput FileFormat GDMA_FileChar am:hasOutput File GDMA_FileChar am:hasScope ImageFiles
GDMA_FileChar am:hasCondition ServiceInstance
GDMA_FileChar am:hasActor CertainScript GDMA_FileChar am:hasEffect FileCharLog In short, GDPA_FileChar activity takes a file as an input and produces values for the file size and file format (which can be semantically clearly defined as necessary – e.g. with measurement units and format IDs in a file type registry) as outputs; the initial file is passed over as another output. To derive the file size and format, the activity uses CertainScript (which again can be semantically clearly defined as necessary – e.g. with references to a software repository). As an additional outcome (better defined not as Output but as Effect) of the file characterization activity, we get theFileCharLog log file. The scope of activity is defined as ImageFiles (so that other kinds of files can be handled by differently defined Characterization Activities; what “ImageFiles” actually means can be clearly defined with e.g. a reference to a certain taxonomy entry). The Condition is defined as ServiceInstance (which means that Actor:CertainScript operates in some particular IT service environment).
Mapping of Activity to a particular software implementation can be performed using Activity ID and a reference to a repository with a clear software identity, e.g. a software versioning repository.
The graphic representation of this Characterization Activity (which, in the ideal world, can be designed in a certain authoring tool with graphical user interface and producing the above RDF as a serialization) is illustrated by Figure 2.
The problem of the policy composition out of two granular policies outlined in Section 2.1 can be addressed with the help of other classes of activities that we introduced earlier: Logical Switch and Control. For the sake of simplicity (as we are going just to illustrate it how the policy modelling can be done) we will not be defining all aspects for these activities, e.g. we can omit Scope or Effect but they may be required in a real policy modelling situation.
The Logical Switch activity will take File, FileSize and FileFormat as Inputs, a particular logic of handling file moves to either storage A or B, as well as file conversion, will be Condition. The Activity yields a list of particular control statements (like “move File to storage A”, “Convert file in JPG format”) as Output. The shape of such defined Logical Switch activity is illustrated by Figure 3.
The semantically clear definition of a Logical Switch Activity gives an idea of how we suggest to address the problem of a policy composition from granular policy statements. The hope is, if the logic of producing control statements is made explicit, as well as the control statements themselves, this will eliminate the ambiguity of a policy composed of granular policy statements.
A good question is what formalism, if any, will be adequate for the expression of logic in the Condition of the Logical Switch. The short answer is: it depends on the policy engine implementation. In an extreme case, this Condition can be just a mandatory textual explanation (commentary) of the logic implemented by the Actor (which is omitted in the Figure 3), i.e. by an executable function or a procedure or a script for a particular IT platform. Alternatively, rules modelling language or workflow templates (and appropriate engines for them) can be used – yet, in this case, the actual usage of these modelling languages or workflow templates would be limited to the policy logic enwrapped in the Logical Switch Activity, allowing freedom for different implementations of other types of Activities involved in the policy definition.
How to express control statements in the Output is subject to particular implementations, too. The only consideration which is important for the moment – important both from conceptual and from implementation perspectives – is having the list of control statements as a clearly defined interface between Logical Switch Activity and Control Activity.
Control Activity takes the list of control statements as Input and makes platform-specific function or procedure or script calls that implement the control statements. Actors for Control Activity are particular functions / procedures / scripts and the Effects of it are log and error files or messages – whatever is used for traceability in a particular implementation. Condition is, similarly to the file characterization activity definition, a particular software platform or IT service where Actors operate. Figure 4 presents an example of a diagrame for the Control Policy.
Generic Data PolicyActivities (such as data characterization) can be combined with Logical Switch Activities and Control Activities in a chain or a network of activities. For our example, the resulted chain is illustrated by Figure 5. It represents the full model of a certain data policy expressed as a chain of semantically clear activities with interfaces between them, as well as interfaces to activity implementations in particular IT services or software platforms.
It is worth mentioning once again that every aspect in the Figure 5 diagrame (such as File, Size, Format, Script or Log) should be thought of not as a particular artefact or a value but as a semantic wrapper of an artefact or a value. As a particular model serialization, these semantic wrappers can be RDF statements about artefacts or values.
In real data policy modelling situations, it may be necessary to define more than one instance of each Activity type; as an example, there could be two Data Characterization Activities defined (one for the file size and another for the file format) in place of one in our example. Nevertheless, even differently defined Activities could be combined in a semantically clear network representing the same data policy.
If Activities in Figure 5 are clearly defined and sensibly combined in the Activity network, this eliminates any ambiguity in policy definition and execution exemplified by two interfering granular policies discussed back in Section 2.1 so that the actual result of the policy implementation becomes predictable and can be formally validated.
One of the strengths of the suggested model is a combination of its reasonable expressivity with its high flexibility as it is based on the idea of composition of activities that can be a) modelled differently b) implemented differently and c) operated (executed) differently. In the above example, scripts for file characterization and scripts for policy execution can be implemented using different software and operated by different components of the same service, or by different services, or even by different e-infrastructures.
The actual chain or network of activities, as well as definition of each of them (i.e. definition of all semantic wrappers) could be done in a certain authoring tool with a graphic user interface and RDF as a model serialization format. Development of such a tool has been beyond resources available for this conceptual work; however, such a tool is worth mentioning as one of the elements of an IT architecture that can support data policies formulation, execution and validation.
The proposed IT architecture is presented by Figure 6 with the most essential components and information flows (that would constitute a core operational platform for data policy management) designated as filled-in boxes and arrows; more advanced components and flows are designated as dashed boxes and arrows with a blank background.
As already suggested, having policy Activities authoring tools with GUI and possibility to serialize Activity networks in a semantically explicit format such as RDF is essential for good levels of adoption of the suggested approach and therefore such authoring tools should be a part of a sensible IT architecture for data policy management. In addition, what is required is a repository where policy designs can be stored and retrieved from.
Figure 6 IT architecture for activity-based policy management
Activity network interpretation engine picks up Activity network from the authoring tools or repository and executes them. In order to execute activity networks in a particular IT environment (software platforms and services), a mapping engine is required that maps Activities and their aspects (such as Conditions or Outputs) to configuration files and executable scripts.
In addition to this generic mapping engine, specific engines for logical conditions and control statements can be implemented. Effects repository stores Effect aspects of each Activity; it is a generalization of logging service and contains semantically clear tracks of Activities execution. Policy search interface can be designed for searching and sharing data policies.
For the purposes of data archive or data infrastructure audit, a policy validation engine is required that talks to policy search interface and to Effects repository. The actual validation can be based on matching graphs of artefacts resulted from policies execution with graphs of Activities in the policy design.
The problem of data policy modelling with reasonable crosswalks between high-level (read: textual) policies and their machine-executable implementations has yet to find a satisfactory solution. The challenges of policy design and implementation are even bigger when collaborative data infrastructures are operated in combination with the in-house software platforms.
The problem of semantically clear crosswalks and the
problem of data policy implementation across
organization-specific and external IT services can be
addressed by adoption of certain policy modelling
techniques and tools. Activity Model [
This work has introduced extensions to the Activity Model in order to make it fit for the task of data policy modelling. An example of using the Activity Model for the definition of a particular data policy has been given, and a possible IT architecture has been considered that can support data policy management based on Activity networks.
This work is supported by EUDAT 2020 project that receives funding from the European Union’s Horizon 2020 research and innovation programme under the grant agreement No. 654065. The views expressed are those of the author and not necessarily of the project.