This paper addresses on how to assess the maturity of a digital archive using the resources of the eArchiving Capability Maturity Model (eACMM) and the eArchiving Reference Architecture (eRA), both initially developed by the E-ARK project 1 and later adopted by the European Union eArchiving Initiative 2 .
Existing maturity models are beneficial, but often miss the supporting mechanisms to realize their full potential. This paper seeks to bridge this gap by demonstrating how Enterprise Architecture (EA) [1,2] management can enhance the usability and effectiveness of maturity models, fostering better decision-making and targeted improvements in organizational structures. The goal is to produce visual aids that can facilitate the maintenance and enhancement of digital archive implementation maturity by providing blueprints that encapsulate pertinent information about the maturity model. Furthermore, the paper aims to enable stakeholders to identify and rectify gaps through automatically generated visual representations, as already attempted by [3]. These blueprints provide a complete overview, facilitating a more intuitive grasp of the maturity model's current status and areas for growth.
One limitation identified in the literature is the lack of tools currently available for representing the maturity gaps. Usually, this is done manually, but automatic approaches are already being pointed as a future solution to obtain better assessments [4]. While maturity models provide valuable insights, their full potential remains unrealized without adequate support mechanisms.
Central to this process is the usage of the tool ATLAS 3 that supports various features of EA management. Specially relevant to our focus was its capability to generate views providing a visual synthesis of complex data, making it easier to analyze and identify discrepancies within the models. By leveraging ATLAS, the paper demonstrates how a systematic approach can be used to visualize and thereby enhance the maturity models under consideration. A dataset containing real digital archives implementations that used the eRA validates our maturity assessment proposal. It is important to emphasize that this paper is closely aligned with the eArchiving initiative, the eRA and the eACMM. eACMM is composed by a set of requirements represented as structured information, while eRA and the cases were represented in an ArchiMate model as a structured collection of elements, relationships and viewpoints.
For short, the solution encompasses the following stages done in the EAM tool: a) importing data, b) classifying the eACMM in the eRA model, c) computing the eACMM capability level for each element of the real-world case represented by the eRA model, and d) visualizing the eACMM capability levels organized by capability.
The Design Science Research (DSR) method, as outlined by Peffers et al. [5], is employed, in this paper, due to its alignment with the expected enterprise architecture (EA) deliverables of our work, which are conceptually similar to DSR artifacts. These artifacts represent an organization's structure, components, and interactions, providing a holistic perspective of its processes, information systems, technology, and more. Moreover, DSR is chosen for its suitability in understanding and advancing existing EA models. Its cyclical nature allows for iterative refinement of artifacts based on feedback from experts and practitioners, ensuring that the resulting viewpoints to be both theoretically rigorous and practically effective.
The paper is organized as follows. Section 2 presents the background concepts. Then, Section 3 describes and compares the EA tools that are of potential use in our paper. Afterwards, Section 4 details our solution. Section 5 validates the solution using the available datasets from E-ARK project. Finally, Section 6 concludes the paper and identifies future work.
The E-ARK project (European Archival Records and Knowledge Preservation) was a multinational initiative aimed at standardizing digital archiving methods across Europe. Running from 2014 and cofunded by the European Commission, it developed guidelines, open technical products, and integrated archiving infrastructure to enhance the availability, access, and use of archival data. Its results were adopted by the present eArchiving Initiative, which is influencing the European landscape in relation to the concern of digital preservation in the domain of digital archives.
This paper uses the dataset of the "eArchiving Referece Architecture v2.0", which is publicly available at: https://kc.dlmforum.eu/earchiving-ra20/ and the maturity model eACMM, both developed by E-ARK. These components already exist and are confirmed in the domain of digital archives and digital preservation. The focus is on enhancing their usability and effectiveness through the development of visual features.
Maturity models are widely used and recognized in both business and academics due to their simplicity and effectiveness. They can assist stakeholders grasp the present degree of maturity of a certain element in a meaningful way, allowing them to clearly identify strengths and weaknesses that need to be improved, and so prioritize what has to be done to achieve the next level. This may be used to demonstrate the outcomes of that endeavor, allowing stakeholders to determine whether the results justify the work or expenditure.
According to Proença and Borbinha [6], a maturity model outlines a progression toward a more structured and systematic approach to business operations. It is a commonly employed method for evaluating specific facets of organizations and businesses, such as business processes. A maturity model represents commonly accepted and validated knowledge, thus conducting a maturity assessment against and established maturity model helps stakeholders pinpoint strengths but also areas needing improvement, allowing them to prioritize actions to reach higher maturity levels. These assessments can vary from straightforward self-assessment questionnaires to comprehensive evaluation methods.
Proença and Borbinha [7] discuss how a maturity model can be a useful tool for assessing various domains of concerns and charting organized routes to better business practices. Typically, maturity models have various "maturity levels," usually five (but this might vary): Initial, Managed, Defined, Quantitatively Managed, and Optimizing. Maturity models give businesses with: (1) auditing and benchmarking metrics; (2) progress evaluation versus objectives; and (3) insights into strengths, shortcomings, and opportunities, which help in strategic decision-making and project management. Maturity is described as the process of managing and directing an entity's growth, meeting organizational goals, and transitioning from an initial to a more mature condition. Mature companies use different indications than less mature ones. Maturity models guide the progression from a starting condition to a targeted maturity state, offering a framework for evaluation, benchmarking, and continuous improvement.
The eACMM, that is a e-ARK's product, considers five maturity levels, which are, from the lowest to the highest: (1) Initial, (2) Basic, (3) Intermediate, (4) Advanced and (5) Optimizing, as depicted in Figure 1. It also focuses that on six fundamental capabilities for digital preservation: Pre-Ingest, Ingest, Access, Preservation and accessibility planning, Data Management, and Archival Storage. This method allows businesses to analyze their information governance practices around digital preservation. The eACMM focuses on the most significant references for digital archiving, particularly those that are being enhanced as part of the old eArchiving Building Block and the present eArchiving Initiative.
Reference architectures and maturity models are two key ideas in EA [2] that strive to improve organizational efficiency and effectiveness. Reference architectures provide a standardized framework for designing and implementing business processes and IT infrastructure, whereas maturity models examine an organization's processes and structures for sophistication and competence. Understanding the relationship between these two ideas is critical for achieving an organization's strategic objectives and operational excellence.
Pereira and Sousa [8], discuss how reference architectures, such as the Zachman Framework [9], TOGAF (The Open Group Architecture Framework) 4 , and others offer blueprints for the structure and operation of the IT and business processes in an organization [10]. These references define the important components and their linkages, ensuring that the organization's procedures are in accordance with its strategic goals. Reference architectures thus provide uniform languages and sets of standards to simplify the understanding of complicated realities, promote consistency, and ensure interoperability across domains.
Luftman [11] used the Capability Maturity Model Integration (CMMI) and the Business IT Alignment Maturity model to assess the current state of the processes of an organization, and provides a road map for continuous improvement. Models like that make possible to evaluate the effectiveness, efficiency, and agility of business and IT processes, identifying strengths and areas for improvement. They can guide organizations in progressing through various maturity levels, from initial and ad hoc practices to optimized and continuously improving processes.
The symbiotic relationship between reference architecture and maturity models is addressed Pereira and Sousa [12]. Reference architectures can serve as a foundation upon which maturity models can be applied, as they can provide the structured environment necessary for the effective maturity assessment. For instance, the use of a comprehensive reference architecture ensures that all components of the organization that are modeled can be considered during the maturity assessment, thereby providing a holistic view of the capabilities of the business.
Van Steenbergen et al. [13] discuss on how maturity models leverage reference architectures to identify gaps and weaknesses in the current system is presented. By mapping maturity model assessments to the reference architecture, organizations can pinpoint specific areas that require improvement and develop targeted strategies to enhance those areas. This alignment ensures that improvements are implemented in a systematic and cohesive way, consistent with the organization's overarching strategic direction. For example, the Business IT Alignment Maturity model may be linked to the architectural domains of TOGAF (business, data, application, and technology), indicating a clear route to greater levels of alignment and maturity.
Gonzalez-Perez et al. [14] emphasize that reference models facilitate the implementation of maturity models by providing predefined templates and best practices. These characteristics are critical for businesses with lower maturity levels, where procedures may be unclear or poorly defined. Reference models provide a starting point and a set of standards that may be customized to meet the organization's particular needs. This organized strategy reduces the risks associated with ad hoc deployments and guarantees that gains last in the long run.
According to Proença and Borbinha [15], the EA domain exists to address the challenge of ensuring alignment between IT and business. EA offers advice on harmonizing business and IT, but adopting its approaches might be difficult. This study seeks to provide a maturity model for EA in companies as a governance tool for analyzing and improving their existing status. Maturity models enable reengineering linked to the EA life cycle through benchmarking and road map planning. This model is built on design science research and current literature, and it identifies essential success elements while also reviewing previous maturity models. The suggested model, which was tested in a multistep procedure, fills gaps in existing models by including best practices and key success criteria. EA entails developing models that represent an enterprise's organizational structure, business processes, information systems, and infrastructure in order to manage and maintain alignment across time. Despite existing approaches, companies lack tools for assessing their present EA practices and identifying changes, making it difficult to improve their EA management practices.
In conclusion, the relationship between reference architectures and maturity models in the EA context is integral to achieving organizational excellence. Reference architectures provide the structural framework and common language necessary for effective maturity assessments, while maturity models offer a road map for continuous improvement. Together, they enable organizations to systematically enhance their processes and capabilities, ensuring alignment with strategic objectives and fostering long-term success [16,17].
This section reports to EA tools that are available to capture, transform and generate related data. EA is defined by TOGAF [18] as "the structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time". Also, Greefhorst and Proper [19] consider three perspectives for architecture: "regulation-oriented, design-oriented, and knowledge-oriented, where the first corresponds to the prescriptive perspective, the second corresponds to the descriptive perspective, and the third corresponding to the high-level design decisions of the system". The EA way of working are proposed by many standards that are widely used in industry, e.g. TOGAF, DODAF, MODAF, etc., and are recognized as a lingua franca between clients, suppliers, consultants and/or researchers. The usage of tools aids the creation of EA artifacts, such as catalogs, matrices and diagrams, avoiding expending large amounts of time and effort to create them.
From the data summarized in Table 1, it is observable that ATLAS in comparison to other tools such as ADOIT 5 , Archi6 , ARIS 7 , Enterprise Architect8 , LeanIX9 , MEGA 10 , and BizzDesign11 , offers features particularly suited to the maintenance of EA models and the creation of on-the-fly views of the model. ATLAS was used with a specific focus on digital preservation efforts, such as the E-ARK project, and is primarily used to generate graphical representations that aid in the analysis and decision-making processes in complex architectural environments.
ATLAS was chosen for the E-ARK project because of its capabilities to model and manage digital preservation procedures, and pragmatically, because it was a known tool for the research team. It assisted with critical activities such as blueprint design and effect analysis, both of which were required to assess digital archive maturity. However, it is crucial to highlight that the applicability of ATLAS varies based on the project objectives, and this comparison reflects its specific application within the context of the E-ARK project.
This section demonstrates the solution, systematically increasing the level of information provided to ensure a thorough understanding of the concepts presented. This step-by-step approach enhances clarity and guides the reader through the intricacies of our method, facilitating a detailed exploration of how our solution addresses the research problem. As an overview of the solution, Figure 2 depicts how the three steps are executed to generate the EA artifacts.
Prior to commencing the analysis of the solution, it is crucial to provide a comprehensive description of the dataset. The dataset is fundamentally derived from the project's reference architecture, which encompasses 616 elements across various classes. The Table 2 provides a detailed breakdown and analysis of these dataset elements.
The initial step is to import the dataset into our ATLAS tool, and then create our metamodel based on the classes in the dataset. However, it is possible to add properties to these classes within the dataset. In addition to the convenience of using a Business Actor like Joe, the tool allows you to add properties to an object, such as Joe's age. In ATLAS, the information about Joe object is stored, which is the Business Actor type and that he is, e.g., 40 years old. This method manipulates and adds properties to metamodel classes using this analogy, allowing us to create blueprints. In this scenario, the Boolean property "isAssessed" is added to all metamodel classes to indicate which objects in the dataset were or were not covered by the maturity model. The paper also included a property named "environment" to distinguish which features are included in each case study and even the reference architecture. In addition to the classes found in the dataset, the Assessment class is generated, which has attributes like maturity level, capability level, and capability function. Objects in this class can be associated with objects in any other class in the metamodel, thus providing information on the relationship between the assessment made and the element covered by that assessment.
Our solution is based on the construction of blueprints and can be categorized into two distinct branches. One branch focuses on conducting a compliance analysis between real cases and the eRA, while the other branch centers on providing an overview of the eACMM. In the demonstration of the first branch, a comparison between RODA case with eRA is done. Although the solution have the capability to analyse five real cases: RODA, ARS, ESSArch, NAE and NAF. The solution covers all these cases by inserting initial parameters when the blueprint is started. In this case, when blueprints are opened, for the first part of the solution, a pop-up window will appear, allowing us to choose an option from the list of real cases mentioned above. For example, for the following demonstrations below, the RODA parameter has been chosen. This means that the elements to be compared with the reference architecture will be those present in the RODA architecture.
In this study, various case studies were used to demonstrate the actual implementation of the eACMM maturity model in conjunction with EA models. The case studies chosen include RODA, ARS, ESSArch, NAE, and NAF. These are the identical cases used in the reports that will help us validate. RODA (Repository of Authentic Digital Objects) is a digital preservation platform that ensures long-term access to digital content. It covers the entire digital archiving process, including ingest, archival storage, and access to digital objects.
The ARS (Archival Repository System) focuses on the management of archival records, ensuring that ingest, archival storage, and preservation processes are compliant with digital preservation standards.
ESSArch is a digital preservation system designed to maintain the integrity and accessibility of digital archives over time.
The NAE (National Archives of Estonia) focuses on ingesting and preserving large volumes of government records.
NAF (National Archives of Finland) handles both the preservation and dissemination of government records and historical data.
The purpose of the eACMM is to evaluate the maturity of digital preservation processes. It focuses on three major aspects of an archive: ingest, archival preservation, and dissemination. These processes are crucial in ensuring the long-term preservation and accessibility of digital records. The eACMM evaluates an organization's capabilities in managing these processes using a set of capability levels, ranging from basic to optimizing. Each level assesses an organization's ability to implement and maintain effective digital preservation workflows.
Our suggested approach makes use of EA models, specifically ArchiMate, to improve these procedures by offering clear, pictorial depictions of the systems in question. These models aid in detecting weaknesses and potential areas for enhancement in the adoption of digital preservation best practices. Through mapping digital preservation systems' capabilities to the eACMM, it is guaranteed that organizations can systematically advance their maturity in all important areas. For instance, depicting the data flow through the ingest process, from submission to archival storage, using ArchiMate models, emphasizes decision-making points and roles. Inefficiencies or potential issues in the preservation workflow can be then identified and changes that are consistent with greater capability levels can be suggested, by evaluating the implemented flow against the eACMM maturity criteria.
As previously mentioned, our solution includes a branch dedicated to conducting a compliance analysis between a given real case and the E-ARK reference architecture. This analysis involves assessing both sides to determine their alignment and compatibility. Starting with the most basic subset of this solution view. The figure 3 illustrates the business functions present in both the reference architecture and the scenario. The left box displays all business functions within the reference architecture, while the right box shows those present in the scenario. This visual comparison allows for a quick assessment of compliance, identifying which elements align and which do not between the two sets. In the next stage, a box is located in the middle to indicate which elements have been reviewed, and therefore, are included in the maturity model. At this point, merely adding a box increases the amount of information provided by the plan. In the middle box, it is revealed the parts that are examined by the maturity model, which are the model's major components. Figure 4 illustrates the process.
In the third step of our process, the box housing the assessed elements based on their capability are considered, and created an alternative blueprint version categorized by capability levels. As part of enhancing clarity and usability, color coding is incorporated, specifically employing green and red hues within the scenario elements box. These colors serve a crucial role in swiftly identifying whether an element has undergone evaluation or not. A red indication adjacent to an element signifies its non-assessment status, whereas a green indication denotes that the element has been thoroughly evaluated. This strategic integration of color coding not only streamlines the assessment process but also enhances the overall accessibility and interpretation of the blueprint and its respective components. Algorithm 1 depicts the pseudo code for construction of this process to designing the final blueprints. The pseudo-code is adapted to the blueprint version that is organized by the capability levels, but it can be easily adapted to the maturity level version. On the one hand, our proposal have to check what capacity function an assessment that is associated with a dataset element has. The solution is checking what is written in the property "CapabilityFunction" from an assessment that is associated to an object from our dataset. To adapt it to the maturity level blueprint version a check about what are the number written on the "MaturityLevel" property from an assessment associated to an object is required. It will look like this: 𝑜𝑏𝑗𝑒𝑐𝑡.𝐴𝑠𝑠𝑒𝑠𝑠𝑚𝑒𝑛𝑡.𝑀 𝑎𝑡𝑢𝑟𝑖𝑡𝑦𝐿𝑒𝑣𝑒𝑙.
Figures 5 and 6 present the generated blueprints that include that third step, in the leftmost box all the elements belonging to the reference architecture. In the middle box, these same elements are evaluated, i.e., they are linked to the maturity model. In the right-hand box is depicted the elements that are part of the scenario, in this case the RODA scenario. It is shown that although there are only part of the elements present in the reference architecture, they correspond, and in the small colored boxes the elements evaluated in the scenario also conform to the reference model.
In this case, the scenario is in line with the reference architecture. Therefore, the organization would only recommend adding the elements that the reference architecture box contains and the real case does not. The elements that should be assessed are also in conformity. Figure 7 exemplifies the final solution blueprint covering all the layers that the scenario contains.
This section introduces an alternative blueprint emphasizing the maturity context of eACMM's capabilities. Each capability is depicted as a series of nested boxes, illustrating its evolution across various capability levels. This hierarchical representation offers a detailed view of how each capability progresses in terms of maturity within the E-ARK framework. Algorithm 2 depicts the pseudo-code trying to demonstrate the logic of the implementation of this part. Due to the code becoming repetitive, only the case in which the object is populated in the Pre-Ingest sub-container will be exemplified, however it can be generalized to Pre-ingest, Ingest, Access, Preservation, Data Management and Archival Storage. Figure 8 shows not only one class but as many as the ones contained in our maturity model. One business process and two business functions are observed. It is also relevant how the elements are distributed by their capability level. Figure 9 depicts the global result of the blueprint, here intentionally shown only as a low detailed glimpse view.
This section details the validation process undertaken to assess the effectiveness and reliability of our proposed solution. Our validation criterion centers on a comparison with definitive reports 12 13 generated by the E-ARK project team, considered authoritative and universally accepted as correct. The core of our validation hinges on achieving identical results to those outlined in this benchmark document. Our approach systematically scrutinizes the outputs generated by our solution against the outcomes documented in the E-ARK report. This thorough analysis ensures not only technical accuracy but also alignment with established standards and methods stipulated by E-ARK. By demonstrating that our solution consistently produces results congruent with those in the authoritative report, this paper underscore its reliability and applicability within the context of compliance analysis for reference architectures.
Moreover, this validation phase serves as a cornerstone in affirming the credibility of our solution. The convergence of our findings with those of the E-ARK report validates not only the technical robustness of our method but also its potential to contribute significantly to advancing the field's understanding and implementation of compliance analysis within reference architecture frameworks.
This paper presents an advancement to the field of EA by addressing the critical need for effective tools/view models to enhance and maintain maturity models. By developing visual tools using the ATLAS platform, this research provides a novel approach to synthesizing complex data into comprehensible blueprints. These visual representations enable stakeholders to identify and rectify gaps, fostering a more intuitive understanding of the maturity model's current state and areas requiring improvement. The alignment with the E-ARK project underscores the practical applicability of this research, demonstrating its relevance and potential impact. By integrating these visual tools with the E-ARK reference architecture and maturity model, the paper addresses the existing gaps between them and enhances the overall usability and effectiveness of maturity models.
Ultimately, this work aims to empower stakeholders to make more informed decisions and implement targeted improvements within their organizational structures. The comprehensive approach outlined in this paper not only maximizes the utility and impact of maturity models but also contributes to the broader goal of advancing EA practices. This paper represents a pivotal step towards more effective management and enhancement of maturity models, ensuring they fulfill their potential as valuable tools for organizational development and success.
In the end, we realized that two tasks could have been done differently, which is now identified as future work: the assessments could have been imported by interpreting ATLAS Forms, creating a questionnaire that generates the same data as the manually importation that was done. Consequently, by filling in a questionnaire, the maturity assessment process would resemble the process that was used to generate the reports used to validate our solution. In addition, the second part of the solution, which had a different focus and did not allow the manipulation of colored flags, could have included a compliance analysis for the selected practical cases.
This work was funded by national funds through Fundação para a Ciência e a Tecnologia (FCT) (No: UIDB/50021/2020 (INESC-ID)), and the INESC-ID project E-ARK CSP (PR09033) and the eArchiving Common Services Platform -CNECT/LUX/2021/OP/0077 NUMBER -LC-01905904.
DA x RA compliance (w/ assessment) by maturity l