=Paper=
{{Paper
|id=Vol-341/paper-1
|storemode=property
|title=An Approach for Enterprise Interoperability Measurement
|pdfUrl=https://ceur-ws.org/Vol-341/paper1.pdf
|volume=Vol-341
|dblpUrl=https://dblp.org/rec/conf/caise/ChenVD08
}}
==An Approach for Enterprise Interoperability Measurement==
https://ceur-ws.org/Vol-341/paper1.pdf
An Approach for Enterprise Interoperability
Measurement
David Chen1, Bruno Vallespir1, Nicolas Daclin2
1
IMS-LAPS/GRAI, University of Bordeaux
351, Cours de la liberation, 33405 Talence, France
2
Centre de Recherche LGI2P, Nîmes, France
david.chen@ims-bordeaux.fr
Abstract. Developing or improving enterprise interoperability implies that the
degree of interoperability can be measured, and causes identified and analysed.
This paper aims at presenting an enterprise interoperability measurement
approach developed within the frame of the two main European IST projects in
this field: INTEROP NoE and ATHENA IP. Having given the basic concepts
on enterprise interoperability, the paper focuses on presenting three kinds of
enterprise interoperability measurements: interoperability potentiality,
interoperability compatibility and interoperability performance. Future works
and perspective are discussed as part of conclusion.
Keywords: Enterprise interoperability, interoperability measurement.
1. Introduction
Interoperability is still a vague concept and has many definitions and connotations to
different people in different sectors and domains. Starting from a pure software
problem in the middle of 90’s, interoperability is taking on a wider meaning to cover
the many knowledge spaces, dimensions and layers of single and collaborating
enterprise. Since a decade, although some efforts have been made to develop
enterprise interoperability, especially in Europe [2], [9], [14] where several research
projects have been launched under FP5/FP6, there is still no an overall satisfactory
solution on interoperability. Research in this area is still fragmented. Most of
researches and developments are focused on the technology aspect to solve
interoperability problems. Few approaches are developed to evaluate the degree of
interoperability. Although some works have been performed in this domain [3], [10],
[11], [12], [17], however it is difficult to define metrics, mainly due to the difficulty to
identify the attributes to characterise the interoperability. Thus, developing
interoperability measurement is becoming an important challenge.
This paper is concerned with the research undertaken within INTEROP NoE and
ATHENA IP during the period of 2003-2007. The objective is to present enterprise
interoperability measurement concepts and principles. After having given basic
2 Proceedings of MoDISE-EUS 2008
concepts on enterprise interoperability in section 2, the paper discusses in detail in
section 3 the three basic interoperability measurements (potentiality, compatibility
and performance). The three measurements proposed are consistent to and built on the
conceptual interoperability framework presented in section 2. An illustration example
is outlined in section 4 to show how to identify interoperability barriers through an
interoperability compatibility measure. Future works are given as part of conclusion
at the end of the paper. It is to note that the approach proposed in the paper should be
considered as a basis. Detailed and operational methodology still needs to be
developed to guide users to carry out these measurements.
2. Basic concepts and definition
Generally speaking, interoperability is the capability for two (or more) systems to
exchange information [13] and to use reciprocally their functionality [19]. It has been
considered that: (i) Enterprises are not interoperable because of barriers to
interoperability; (ii) Barriers are incompatibility of various natures at different
enterprise levels; (iii) Barriers common to all enterprises can be identified and tackled
[4], [14].
2.1 Interoperability barriers
Three categories of barriers (conceptual, technological and organisational) are
identified as follows [14]:
- The barriers of a conceptual nature which relate to the syntactic and semantic
differences of information to be exchanged. These barriers concern the modelling at
the high level of abstraction (ex.: enterprise models of a company) as well as the
level of the programming (ex.: low capacity of semantic representation of XML).
- Technological barriers relating to the incompatibility of information technologies
(architecture & platforms, infrastructure…). These barriers concern the standards to
present, store, exchange, process and communicate data and information through the
use of software systems.
- The barriers of an organisational nature which relate to the definition of
responsibility and authority so that interoperability can take place under good
conditions. These can be seen as ‘human technologies’ or ‘human factors’ and are
concerned with human and organisation behaviours which can be obstacles to
interoperability.
2.2 Interoperability concerns
This section defines the interoperations that can take place in the various areas of
concerns of the enterprise [14].
- The interoperability of data: It refers to make work together different data models
(hierarchical, relational, etc.) and of the different query languages, to find and share
information coming from heterogeneous bases which can moreover reside on
Proceedings of MoDISE-EUS 2008 3
different machines with different operating systems and data bases management
systems.
- The Interoperability of services: It is concerned with identifying, of composing and
making function together various services/applications (designed and implemented
independently) by solving the syntactic and semantic differences as well as finding
the connections to the various heterogeneous data bases.
- The interoperability of processes: it aims to make various processes work together.
Generally in a company, several processes run in interactions (in series or parallel). In
the case of the networked enterprise, it is also necessary to study how to connect
internal processes of two companies to create a common process.
- The interoperability of business: It refers to work in a harmonised way at the levels
of organization and company in spite of for example, the different modes of decision-
making, methods of work, legislations, culture of the company and commercial
approaches etc. so that business can be developed and shared between companies.
2.3 Interoperability approaches
According to ISO 14258 (Concepts and rules for enterprise models), there are three
basic ways to relate entities together to establish interoperations [15]:
- Integrated approach: there exists a common format for all models. This format must
be as detail as models. The common format is not necessarily a standard but must be
agreed by all parties to elaborate models and build systems.
- Unified approach: there exists a common format but only at a meta-level. This meta-
model is not an executable entity as it is in the integrated approach but provides a
mean for semantic equivalence to allow mapping between models and systems.
- Federated approach: there is no common format. To establish interoperability,
parties must accommodate ‘on the fly’. Using federated approach implies that no
partner imposes their models, languages and methods of work.
Fig. 1 describes formally the basic concepts defined above and their relationships.
Interoperability
knowledge
provides
Business Solution Conceptual
concerns removes
Interoperability Interoperability
Process Technological
concern barrier
uses
Interoperability
Service approach Organisational
Data Integrated Unified Federated
Fig. 1. Basic concepts of enterprise interoperability [6]
4 Proceedings of MoDISE-EUS 2008
3. Interoperability measurement
The fact that interoperability can be improved means that metrics for measuring
interoperability can be defined. Measuring interoperability allows a company
knowing its strengths and weaknesses to interoperate and to prioritize actions to
improve the interoperability. Current state-of-the-art focuses on solving some specific
interoperability problems at technological level; few researches have been done to
deal with interoperability measuring problem. Existing approaches mainly focus on
maturity measure issues [7], [10], [11], [17], [18]. No approaches have been found in
the literature on interoperability compatibility measure and interoperability
performance measure. At the current stage of research, three kinds of interoperability
measurement can be considered: (i) interoperability potentiality measure; (ii)
interoperability compatibility measure, and (iii) interoperability performance measure
[5], [8], [14]. These measures take into account of existing works in this area and are
based on the concepts of enterprise interoperability presented in section 2.
3.1 Interoperability potentiality measure
The interoperability potentiality is concerned with a set of characteristics that have
impact on the ability to interoperate with a third partner. The objective of this measure
is to evaluate the potential of a system to accommodate dynamically to overcome
possible barriers when interacting with a partner.
We define the potentiality as the fact that an enterprise possesses intrinsic
attributes related to interoperability, which make it easier to interoperate with other
enterprises, in the eventuality of a partnership. In other words, potentiality is an intra-
enterprise evaluation without the need to know the interoperating partner. The main
goal is to increase the capability of interoperation and decrease the risk to meet
problems during a partnership. Generally speaking, through this measure the
potentiality for an enterprise to interoperate with a unknown partner can be accessed,
such as for example, the use of standards, the flexibility of its organization, the
openness of its ICT infrastructure, the existence of its enterprise models, etc.
Today few methods are developed for measuring interoperability potentiality.
Existing approaches mainly focus on maturity measure, for examples the LISI (Levels
of Information Systems Interoperability) proposed a maturity model for measuring
interoperability in five levels of maturity [3]: isolated, connected, functional, domain,
enterprise; Other approaches based on LISI proposed similar models [7], [18];
ATHENA project has also elaborated for the manufacturing enterprise the EIMM
(Enterprise Interoperability Maturity Model) to address interoperability issues at all
levels of the company [1].
The proposed enterprise interoperability potentiality model concerns the
evaluation of an enterprise according to the three categories of barriers that impact the
development of interoperability, and the four areas of concern where interoperability
takes place, i.e. business, processes, services and data. For each category of barriers
and each concern, five levels are defined to characterize the potentiality [8]:
(1) Isolated: total incapacity to interoperate;
(2) Initial: interoperability requires strong efforts that affect the partnership;
Proceedings of MoDISE-EUS 2008 5
(3) Executable: interoperability is possible but the risk of encountering problems is
high;
(4) Connectable: interoperability is easy even if problems can appear for distant
partnership;
(5) Interoperable: which considers the evolution of levels of interoperability in the
enterprise, and where the risk of meeting problems is weak.
Fig. 2 gives interoperability maturity model illustrated at the business level.
Potentiality
Level Conceptual potentiality Organizational potentiality Technological potentiality
Isolated, Initial, Executable, Isolated, Initial, Executable, Isolated, Initial, Executable
Business Connectable, Interoperable Connectable, Interoperable Connectable, Interoperable
Fig. 2. The enterprise interoperability potentiality model at business level
It is to note that the maximum potentiality does not imply full interoperability. Indeed,
the use of standard tools by an enterprise does not ensure that a partner will use the
same ones. Hence, problems of interoperability can still appear.
3.2 Interoperability compatibility measure
The interoperability compatibility measurement has to be performed during the
engineering stage i.e. when systems are re-engineered in order to establish
interoperability. In contrary to interoperability potentiality measure, the
interoperability compatibility measure can only be performed when the two
partner/system of the interoperation is known. The measure is done with respect to the
identified barriers to interoperability. The highest compatibility means there is no
barrier to interoperability. The inverse situation means the poorest compatibility for
interoperability.
As an example, Fig. 3 shows an illustration of barriers identified at the moment
when company A and company B wish to establish interoperability. In the figure,
‘+++’ means that there exist an important barrier between the two enterprises, ‘+’
means weak barrier, ‘++’ is in between, and ‘-‘ means that there is no barrier.
Company A Company B
Iop CONCEPTUAL TECHNOLOGICAL ORGANISATIONAL Iop
concerns concerns
Company A BUSINESS +++ ++ - BUSINESS
Company B
PROCESS - ++ +++ PROCESS
SERVICE +++ ++ - SERVICE
DATA + +++ + DATA
Fig. 3. Compatibility measure matrix example
Identifying interoperability barriers is only concerned with those ‘things’ that need to
be shared and exchanged between two systems/companies. Interoperability requires a
6 Proceedings of MoDISE-EUS 2008
common basis for those elements. Typically, not all of the information managed by
two systems is shared. Therefore, interoperability requires identifying the shared
elements and possible barriers between these elements.
The compatibility measure can be performed with the help of a questionnaire. For
example, regarding the three categories of interoperability barriers, following
questions can be asked:
Conceptual compatibility: Syntactic: is the information to be exchanged
expressed with the same syntax? Semantic: does the information to be exchanged
have the same meaning?
Organizational compatibility: Persons: are authorities/responsibilities clearly
defined at both sides? Organization: are the organization structures compatible?
Technological compatibility: Platform: are the IT platform technologies
compatible? Communications: do the partners use the same protocols of exchange?
The measure can also be valued. If an incompatibility is detected, the coefficient 1
is assigned to the interoperability concern and the barrier that are considered.
Conversely, the coefficient 0 will be given when none incompatibilities is detected.
Following this rule, the compatibility matrix (Fig. 4) can be built.
Barriers Conceptual Organizational Technology
syntactic semantic authorities organization platform communication
Concerns responsibilities
Business 1 1 0 1 0 0
Processes 1 1 1 1 1 1
Services 1 0 0 0 1 0
Data 0 0 0 1 1 1
Fig. 4. The valued compatibility measurement matrix
3.3 Interoperability performance measure
The performance measurement is to be done during the test or operation phase of two
interoperate enterprises. Criteria such as cost, delay and quality can be used to
measure the operational performance. Each kind of measurement can be evaluated
with local coefficients in order to get a global coefficient ranging from “poor
interoperability” to “good interoperability”.
3.3.1 Time of Interoperation
The time of interoperation corresponds to the duration between the date when
information is requested and the date when the requested information is used. The
time of interoperation can be decomposed in several periods of time. Fig. 5 proposes a
decomposition of the time of interoperation (adapted from Kasunic [11]).
Proceedings of MoDISE-EUS 2008 7
Fig. 5. Decomposition of the time of interoperation
The request time represents the duration between the date when a request is sent and
the date when the request is received by the partner. The treatment time of the request
corresponds to the time to treat the request. The return time corresponds to the
duration between the date when the requested information is sent back and the date
when the information is received. The time to use represents duration of latency, i.e.
the duration between the date when the information is received and the date when the
information is exploited.
Thus, the real value of the time of interoperation can be defined as the sum of all
the periods of time composing this one. This real value can be noted as follow [2]:
tin eff = ∆treq + ∆ttreat + ∆tret + tuse
Where:
• tineff, represents the real value of interoperation time
• ∆treq, represents the request time
• ∆ttreat, represents the time of treatment of the request
• ∆tret, represents the return time
• ∆tuse, represents the time to use
The assessment of the time of interoperation corresponds to the comparison
between the real value of the time of interoperation and the time expected by the
partners. If the time measured is longer than expected, then there is a deficiency in
terms of time of interoperation.
3.3.2 Quality of Interoperation
The quality of interoperation takes in consideration three kinds of quality: (1) the
quality of exchange, (2) the quality of use and, (3) the quality of conformity.
The quality of exchange draws up if the exchange is correctly performed i.e. if
information sent to a partner succeeds. The quality of use represents the number of
information received by a partner in comparison with the number of information
requested. A number of information received superior (difficulty to teat all
information) or minor (shortage of information) to the number of information
requested means a deficiency. The quality of conformity corresponds to the
exploitation of the information i.e. if the information received is exploitable or not.
8 Proceedings of MoDISE-EUS 2008
Thus the quality of interoperation can be defined as the sum of the three kinds of
quality. The quality of interoperation can be noted as follow [2]:
qin = ∆qex + ∆qut + ∆qconf
Where:
• qin, the quality of interoperation
• ∆qex, the quality of exchange
• |∆qut|, the absolute value of use
• ∆qconf, the quality of conformity
The assessment of the quality of interoperation corresponds to the comparison
between the real value of the quality of interoperation and the quality expected by the
partners. A difference means a deficiency in terms of quality of interoperation.
3.3.3 Cost of Interoperation
The cost of interoperation is defined by the costs induced by the removing of the
barriers and the modification of the systems to obtain a satisfying time and quality of
interoperation. It is defined as [2]:
C =C +C
in ex ut
Where:
• Cin, the cost of interoperation
• Cex, the cost of exchange, i.e. the cost to exchange information
• Cut, the cost needed to make the information exchanged usable
The assessment of the cost of interoperation corresponds to the comparison
between the real value of the cost of interoperation and the cost expected by the
partners. If the cost measured is higher than the expected cost, it means a deficiency
in terms of cost of interoperation.
4. An illustration example
This section presents an illustration example based on a Carrier-Shipper Scenario to
show the use of the compatibility measure to detect possible interoperability barriers.
The case was provided by SAP and used in ATHENA A8 project [2]. In the scenario,
a set of needs and objectives for new solutions have been defined from the point of
view of an SME shipper. For examples, (Semi-) automatic integration of Carrier
Services, data and process mapping, user interface, predefined and easy configurable
adapters, and configuration etc. Fig. 6 shows the internal process of the three
companies and cross companies interoperations between the three processes. The
targeted interoperations take place at the all four interoperability concerns (business,
process, service, and data).
Proceedings of MoDISE-EUS 2008 9
Carrier A Shipper Carrier B
Sales
Sales Order
Order
Calculate
Calculate Rate
Rate
Delivery
Delivery
Calculate
Calculate Rate
Rate
Picking
Picking
Generate Generate
Generate Routing
Routing
Generate Routing
Routing
Code Code
Code
Code Packing
Packing
Generate Generate
Generate Label
Label
Generate Label
Label Shipment
Shipment
Fig. 6. Scenario mapped with interoperability barriers [2]
To analyze existing situation and identify possible barriers between the three
companies, interoperability compatibility measure has been performed.
Fig. 7 shows an illustration example of the measure. Tools and methods used in
two companies in the four areas of interoperability concerns were identified and
compared, and their compatibilities assessed. For example, different data
models/formats with different semantics are used by the two companies, resulting in
important conceptual barriers at data level. A template was developed in the project to
document more in detail the barriers encountered and potential solutions that may be
used to overcome the barriers. An example of this template is shown table 1.
Fig. 7. Compatibility measure illustration example
10 Proceedings of MoDISE-EUS 2008
Table 1. Excerpt of the template description [2]
Template elements Description
Interoperability concerns Data, Service
Interoperability barriers Conceptual barrier - Incompatible syntactic and
semantic representation of data at each interacting
partner
Interoperability problem Different models adopted by the companies
makes data exchange difficult as enterprises
cannot exchange their data automatically
ATHENA solutions identified - Conceptual solutions: Annotation of proprietary
models according to common ontology to allow
data reconciliation
- Technical solutions: A3 tools, WSDL Analyzer
5. Discussion
The proposed work is related to existing researches in the following ways:
Existing works [11], [3], [17] etc. mainly focus on the technology aspect and
maturity measure. The approach proposed extends the interoperability measurement
to interoperability compatibility measure and interoperability performance measure.
The interoperability potentiality measure concept proposed in the paper generalizes
and extends the maturity concept to potentiality concept which includes not only
maturity aspect but also other aspects such as openness, flexibility, configurability,
adaptability (not dealt explicitly in the paper) etc.
The proposed potentiality measure model allows extending the LISI maturity
model (dedicated to information systems) to cover the four areas of enterprise
interoperability concerns (data, service, process, business) and three categories of
interoperability barriers (conceptual, technological, organizational).
The interoperability compatibility measure can be done only if the two systems in
question are known. At the current stage, the measure is performed using
questionnaire to collect the information on the systems and applications used at the
both sides and possible incompatibilities determined by experts of the domain. For
example, if two ERP systems use two different terms to designate customer orders
(ex. Order and Command) then there is semantic incompatibility.
The interoperability performance measurement is not only concerned with the
technology aspect (IT systems) but also human and organizational ones. For example,
the time measure covers not only possible data/information transmission time by IT
systems which is quite small, but also delay caused by human factors (such as rigid
and centralized organization, long human reaction delay etc.). The interoperability
concept itself considered in the paper takes into account both IT and human aspects.
The calculation of metrics for interoperability potentiality and compatibility
measurements is done through human judgment and evaluation. Knowledge-based
system can be built for these measures in the future.
Proceedings of MoDISE-EUS 2008 11
The use of these measures depends on the context and objective of the companies.
Generally speaking, the potentiality measure can be used at any time in a company to
evaluate its interoperability potential. A questionnaire can be elaborated to help
collect relevant information. The compatibility measure can be used in an
interoperability project: at the beginning to determine existing interoperability degree
and identify existing barriers between two enterprise systems; and at the end of the
project to measure the interoperability degree achieved and improvement. The
performance measure is used after the project when two systems are in interoperation
so that operational performance (time, quality, and cost) can be accessed.
Interoperability solutions can be categorized according to their ability to overcome
interoperability barriers and linked to interoperability problems identified by these
measures. An interoperability solution repository can be built based on the conceptual
interoperability framework presented in section 2.
6. Conclusions
This paper has presented basic concepts and approaches of enterprise interoperability
measurement. The paper also contributed to establish a science base of enterprise
interoperability considered in the roadmap for enterprise interoperability by the
European Commission [16]. The basic concepts of enterprise interoperability
presented in section 2 served as basis by European Standardisation Committee CEN
TC310/WG1 to elaborate an international standard in collaboration with ISO TC184
SC5/WG1: CEN/ISO 11354 (Framework for Enterprise Interoperability).
The three kinds of enterprise interoperability measurements allows considering the
three aspects of interoperability evaluation: measuring the set of intrinsic properties of
the system for interoperability (potentiality measure); detecting barriers between two
particular enterprises (compatibility measure); and performance evaluation during the
operational phase (performance measure). The three measures are complementary and
consistent with respect to enterprise interoperability concepts.
Concerning interoperability potentiality measurement, the approach presented
mainly focused on the maturity measure. Other system properties that have impact on
interoperability such as openness, flexibility to change and to adapt, configurability
(not only for IT system, but also organizational structure), etc. need to be investigated
and explicitly considered.
Relating to compatibility measurement, it is to note that the measure can be used
in an interoperability engineering project to evaluate both existing interoperability
degree at the beginning of the project and the achieved degree at the end of the project
so that improvement of interoperability can be assessed.
For interoperability performance measurement, the measures proposed at this
stage of research are rather straightforward and need to be tested in more industrial
cases for refinement and validation. It is also to note that the criteria used (time,
quality and cost) are also used in other approaches to evaluate industrial system’s
performance in general.
12 Proceedings of MoDISE-EUS 2008
Acknowledgement
This work is mainly based on the works performed within the INTEROP NoE
(Contract n° 508011) and ATHENA IP (Contract n° 507849) funded by the European
Commission under the IST 6th Framework Programme. The authors grant
acknowledgements to INTEROP and ATHENA consortiums, especially to members
of worWP DI (Domain Interoperability) of INTEROP NoE and ATHENA A8.
References
1. ATHENA Integrated Project, Framework for the Establishment and Management
Methodology, Deliverable DA1.4, 2005.
2. ATHENA Integrated Project, Guidelines and Best Practices for Applying the ATHENA
Interoperability Framework to Support SME Participation in Digital Ecosystems,
Deliverable DA8.2, January 2007.
3. C4ISR, Architecture Working Group (AWG), Levels of Information Systems
Interoperability (LISI), 30 March 1998.
4. Chen, D., Daclin, N.: Framework for enterprise interoperability, IFAC TC5.3 workshop
EI2N (2006), Bordeaux, France.
5. Chen, D., Daclin, N., Barrier driven methodology for enterprise interoperability,
PROVE2007, In Proc. Establishing The foundation of Collaborative Networks, Guimarães,
10-12 Oct. 2007, pp. 453-460.
6. Chen, D., Shorter, D. Framework for Manufacturing Process Interoperability - CEN/ISO
11354, Standardisation workshop in conjunction to I-ESA 2008, Berlin, 25-28 March 2008.
7. Clark, T., Jones, R., Organizational Interoperability Maturity Model for C2., Department of
Defense, Canberra, Australia, 1999.
8. Daclin, N., Interopérabilité des systèmes industriels – Contribution à l’élaboration d’une
méthodologie, Thèse de doctorat (In French), Université de Bordeaux, Décembre 2007.
9. EIF, European Interoperability Framework for PAN-European EGovernment services, IDA
working document - Version 4.2 – January 2004.
10.Hamilton, J., Rosen, J.D., Summers, P.A.: Developing Interoperability Metrics, in joint
command and control interoperability: cutting the gordian knot, Chapter 6 (2004)
11.Kasunic, M., Anderson, W.,: Measuring systems interoperability: challenges and
opportunities, Software engineering measurement and analysis initiative (2004)
12.IAC, Interoperability Strategy Concepts, Challenges, and Recommendations, Industry
Advisory Council (IAC), Enterprise Architecture SIG, April 3, 2003.
13.IEEE, A compilation of IEEE standard computer glossaries, standard computer dictionnary,
1990.
14.INTEROP, Enterprise Interoperability-Framework and knowledge corpus - Final report,
INTEROP NoE, FP6 – Contract n° 508011, Deliverable DI.3, May 21st 2007.
15.ISO 14258 - Industrial Automation Systems – Concepts and Rules for Enterprise Models,
ISO TC184/SC5/WG1, 1999-April-14 version
16.IST, European Commission, Enterprise interoperability research roadmap, (Editors: M.S. Li,
R. Cabral, G. Doumeingts, and K. Popplewell), Final Version (Version 4.0), 31 July 2006.
17.Mick R., Defining and Measuring Interoperability, ARC INSIGHTS, 43, October 2004
18.Tolk, A., Beyond Technical Interoperability – Introducing a Reference Model for Measures
of Merit for Coalition Interoperability. In Proc. of the 8th International Command and
Control Research and Technology Symposium (ICCRTS), Washington, June 17-19, 2003.
19.Vernadat, F., B., Interoperable enterprise systems: architecture and methods, INCOM’2006
– 12th IFAC/IFIP/IFORS/IEEE/IMS Symposium, 2006.