Current digital preservation is strongly biased towards data objects: digital files of document-style objects, or encapsulated and largely self-contained objects. To provide authenticity and provenance information, comprehensive metadata models are deployed to document information on an object's context. Yet, we claim that simply documenting an objects context may not be sufficient to ensure proper provenance and to fulfill the stated preservation goals. Specifically in e-Science and business settings, capturing, documenting and preserving entire processes may be necessary to meet the preservation goals. We thus present an approach for capturing, documenting and preserving processes, and means to assess their authenticity upon re-execution. We will discuss options as well as limitations and open challenges to achieve sound preservation, specifically within scientific processes.
Digital preservation (DP) traditionally has a predominantly data-centric view on both its operations as well as the objects it is dealing with. Digital objects considered for preservation are usually (turned into, as far as possible) selfcontained, static objects such as images resulting from scans, classical documentstyle objects, data sets, but also information packages containing software and other in principle dynamic objects as encapsulated files. Furthermore, objects are usually ”removed” from an operational life-cycle into an archival life cycle, ingested into designated repositories for long-term maintenance, from which they are removed and re-inserted into a potentially new life-cycle when needed in an operational manner again, only to be re-ingested as new objects after completion of their new life as now new archival objects. Consequently, also cost estimation and investments are largely based upon aggregated data item-level information, adding up processing costs, storage costs and others.
We claim that this traditional view is hitting severe limitations as we are
observing a set of interesting changes in the preservation community: Most
importantly, preservation is expanding beyond the traditional cultural heritage
community. While originating in the sciences, recognizing the need to maintain
the investment made into data collected electronically (as evident by the early
and strong commitment of institutions such as NASA, leading to the infamous
OAIS model) the key drivers, expertise and know how and development has come
from and taken place in the cultural heritage community. The key characteristic
of this community is the dedication to preservation of information as a, or even
the, primary mission, resulting in a holistic understanding of the scope of the
problem and its long-term implication beyond individual technical or
organizational issues. Yet, more recently we see a range of other communities facing the
need for digital preservation: back to the origins of DP, science as a whole is
becoming increasingly dependent on data as the core facilitator in virtually all
scientific disciplines, leading to trends defined as data-driven science, e-Science,
Big Data [
This paper starts in Sec. 2 with a loose collection of observations on changes
in the DP community, highlighting three areas of focus that we deem
important, namely a shift towards risk management, viewing DP in the context of
e-Governance frameworks, and the shift towards the preservation of processes
rather than data objects. This will be followed by a more detailed look at two
key aspects, namely process preservation and a framework for evaluating the
quality of re-execution of preserved processes in Sections 3 and 4, respectively
(largely adopted from [
The observed expansion in stakeholder communities has some interesting implications for the DP community: first of all, we are experiencing yet another clash of languages and cultures: after partially successfully consolidating the viewpoints of archival and library communities, merging the viewpoints of the museums community and even succeeding in getting computer science to listen and communicate on an increasingly shared level of mutual understanding, DP is currently being recognized, interpreted and contributed to by a whole range of new key players with completely diverse interpretation of the concepts widely accepted in the traditional DP community. Long-term may be as short as 7 years, preservation and loss is not necessarily measured on the level of the need for maintaining an object, but as best effort vs. risk trade-off, specifically not happening ”at all costs” wherever possible, with deletion-as-early-as-permissible being a key factor, Where DP is serving a business purpose, objects are rarely being perceived as frozen and deposited into an archive. Rather, they need to be maintained in an operational environment. Catch phrases such as business continuity capture a lot about the thinking behind this. More importantly, however, they also help to identify approaches and solutions to serve these (but also more traditional) DP needs that stem from different backgrounds: life cycle management of entire IT environments, redundancy, (IT) security, e-governance structures and methods have been developed, deployed, tested, customized and improved over long time spans in these communities. These solutions may prove valuable contributions to the DP community at large.
This integration of yet another heterogeneous set of communities poses severe challenges to a rather tightly-knit network of DP researchers and practitioners that has just started to evolve into a very young community of its own, with it’s own jargon, events, and commonly understood basis of generic concepts. Yet, it also offers huge benefits. Apart from contributing new competences and tested solutions that can be adapted to serve more generic DP needs, it also broadens the basis amongst which to share the costs of research and development in DP. It allows us to integrate know-how from different groups and grows the market of where this know-how can be used.
In terms of new areas of activity, at least three major trends can be observed: Preservation as a cost/benefit trade-off: The primary focus so far has been on preserving objects because they need to be preserved. Current thinking seems to be moving towards obsolescence as a risk, and DP as a risk mitigation strategy. While this concept is anything but new to the DP community, the key difference is on the juxtaposition of risk and benefit, and a much more pronounced and explicit willingness to sacrifice availability of objects when the investments necessary to maintain them would likely outweigh the business benefits or legal sanctions. Although the same principles are applicable in traditional settings as well, this focus shift forces a more explicit formulation of benefits/value and a clearer specification of risk and costs, especially for more immediate/short term actions. This also will call for the application of existing frameworks for risk identification as well as cost/benefit estimation, with potentially strongly diverging valuations in the non-heritage domains.
DP as capabilities and maturity evaluations: Another important shift
we may see coming is a shift from DP being something happening in a data
archive or repository setting, i.e. a designated institution or department
handling ”old” objects, and being audited on its performance via any set of audit
and certification routines to ensure proper preservation at the highest possible
level. Rather, we may view DP as a set of capabilities that an institution has, as
part of many other operational capabilities, and that are integrated with more
routine processes. We are thus currently investigating opportunities of
integrating DP in an eGovernance framework. Reference models such as TOGAF [
Preserving processes rather than (only) data: The third, major area of activity currently, and probably the most relevant with respect to semantic technologies, relates to the shift from preserving static objects to the preservation of entire process chains. This shift is motivated by two core considerations: first of all, in many new settings, it is not static artifacts, but the need to be able to re-run processes in an authentic manner that are the key DP requirement. While the preservation of data as documentation may be sufficient to provide evidence about processes having been run they are no replacement for preserving the actual process.
But even when the focus is on the actual data, it may be advisable to preserve the process chain the data was subjected to. In an e-Science setting, while preserving the data is an essential first step for any sustainable research efforts, the data alone is often not sufficient for later analysis of how this data was obtained, pre-processed and transformed. Results of scientific experiments are often just the very last step of the whole process, and to be able to correctly interpret them by other parties or at a later point in time, also these processes need to be preserved.
Thus, specifically in an e-Science setting, preserving processes together with the data helps us to meet two goals at the same time: on the one hand, the processes are essential aspects of representation information, allowing to trace the various (pre-)processing steps applied to the data, any bias that might have been introduced, or errors stemming from faulty processing. On the other hand, it also allows us to re-run these processes on new data, learning how models and views evolved, and to discover discrepancies from earlier analyzes.
We thus need to go beyond the classical concerns of Digital Preservation
research, and consider more than the preservation of data. The following section
takes a closer look at some of the activities centering around process preservation,
covering both process capture as well as evaluation of re-executed processes –
and the resulting requirements at the level of process capture. We will choose
examples from the e-Science/Data curation domain as an exemplary setting.
The following sections are largely adopted from a position paper summarizing
our considerations to data quality aspects in data curation for a recent workshop
by the National Science Foundation [
Curation of business or E-Science processes requires capturing the whole context of the process, including enabling technologies, different system components on both hardware and software levels, dependencies on other computing systems and services operated by external providers, the data consumed and generated, and more high-level information such as the goals of the process, different stakeholders and parties. The context of information needed for preserving processes is considerably more complex than that of data objects, as it not only requires dealing with the structural properties of information, but also with the dynamic behavior of processes. Successful curation of an eScience process requires capturing sufficient detail of the process, as well as its context, to be able to re-run and verify the original behavior at a later stage, under changed and evolved conditions. We thus need to preserve the set of activities, processes and tools, which all together ensure continued access to the services and software which are necessary to reproduce the context within which information can be accessed, properly rendered and validated.
To address these challenges, we have devised a context model to
systematically capture aspects of a process that are essential for its preservation and
verification upon later re-execution [
To move towards more sustainable E-Science processes, we recommend
implementing them in workflow execution environments. For example, we currently
use the Taverna workflow engine [
Implementing such a research workflow in a system like Taverna yields a complete and documented model of the experiment process – each process step is defined, as is the sequence (or parallelism) of the steps. Further, Taverna requires the researcher to explicitly specify the data that is input and output both of the whole process, as well as of each individual step. Thus, also parameter settings for specific software, such as the parameters for a machine learning tool or feature extraction, become explicit, either in the form of process input data, or in the script code.
Figure 3 shows an example of the music classification experiment workflow modeled in the Taverna workflow engine. We notice input parameters to the process such as the URL of the MP3 contents and the ground truth, and also an authentication voucher which is needed to authorize the use of the feature extraction service. The latter is a bit of information that is likely to be forgotten frequently in descriptions of this process, as it is rather a technical requirement than an integral part of the scientific process transformations. However, it is essential for allowing re-execution of the process, and may help to identify potential licensing issues when wanting to preserve the process over longer periods of time, requiring specific digital preservation measures.
During an execution of the workflow, Taverna records so-called provenance
data, i.e. information about the creation of the objects, on the data
transformation happening during the experiment. Taverna uses its proprietary Janus
format, an extension on the Open-Provenance Model[
A critical aspect of re-using digital information in new settings is its
trustworthiness, especially its authenticity and faithful rendering (with rendering being any
form of representation or execution and effect of a digital object, be it rendering
on a screen, an acoustic output device, or state changes on ports, discs etc.).
Establishing identity or faithfulness is more challenging than commonly assumed:
current evaluation approaches frequently operate on the structural level, i.e. by
analyzing the preservation of significant properties on the file format level in case
of migration of objects. Yet, any digital object (file, process) is only perceived
and can only be evaluated properly in a well-specified rendering environment
within which faithfulness of performance need to be established. In emulation
settings, this evaluation approach is more prominently present, yet few
emulators support the requirements specific to preservation settings. We thus argue
that, actually, migration, emulation and virtually all other approaches to
logical/structural data preservation need to be evaluated in the same way, as they
are virtually no different from each other as all need to be evaluated in a given
rendering/performance environment. [
We also devise a framework for evaluating whether two versions of a
digital object are equivalent [
An essential component of the framework is the identification at which levels
to measure the faithfulness of property preservation, as depicted in Figure 4. A
rendered representation of the digital object has to be extracted on (a) suitable
level(s) where the significant properties of the object can be evaluated. For some
aspects, the rendering of an object can be performed based on its representation
in specific memories (system/graphics/sound card/IO-buffer), for others the
respective state changes at the output port have to be considered while for yet
others the actual effect of a system on its environment needs to be considered,
corresponding to delineating the boundaries of the system to be evaluated. (Note
that identity on a lower level does not necessarily correspond to identity at higher
levels of the viewpath - in some cases significant effort is required to make up for
differences e.g. on the screen level when having to emulate the visual behavior
of cathode ray screens on modern LCD screens [
A key challenge in this context will be to come up with a comprehensive model of what information to capture for specific types of processes and preservation requirements, as well as guidelines on how to do this. 5
This paper presents a loose collection of some trends observed in digital preservation research, specifically a shift toward a more risk/cost/benefit based approach to DP, the framing of DP in IT governance principles, and specifically the necessity to preserve entire processes rather than only data. With the growing importance of preserving entire processes rather than sets of homogeneous, static (and usually quite simple) objects, the requirements on techniques to capture, document and reason across increasingly complex sets of context meta-information is growing. This requires new approaches to context capture and representation.
Still, the considerations above cover only a small subset of the quite significant research challenges that continue to emerge in the field of digital curation. We thus strongly encourage the community to contribute to an effort of collecting and discussing these emerging research questions in a loosely organized form. To this end, following the Dagstuhl Seminar on Research Challenges in Digital Preservation1, a Digital Preservation Challenges Wiki2 has been created, where we invite contributions and discussion. As a follow-up to the Dagstuhl seminar, 1 http://www.dagstuhl.de/de/programm/kalender/semhp/?semnr=10291 2 http://sokrates.ifs.tuwien.ac.at a workshop on DP Challenges3 will be held at iPRES 2012 in Toronto focusing on the elicitation and specification of research challenges. 3 http://digitalpreservationchallenges.wordpress.com/