In the D-Grid project BIS-Grid we developed the BIS-Grid Workflow Engine in order to utilize a common WS-BPEL workflow engine for scientific workflow execution in WSRF-based Grid infrastructures. The BIS-Grid Workflow Engine itself is built on the Grid middleware UNICORE 6 to benefit from its security mechanisms and to automatically gain interoperability with UNICORE 6-based Grid infrastructures. As beside UNICORE 6 the Grid middleware Globus Toolkit 4 is prevalent in the D-Grid infrastructure, we decided to examine the interoperability of the BIS-Grid Workflow Engine with Globus Toolkit 4. The interoperability with Globus Toolkit 4 concerns basically the support of its Grid Security Infrastructure (GSI) by UNICORE 6. In this paper we describe to what extend GSI is already supported by UNICORE 6 and what is missing. We furthermore give some details about our implementation for the BIS-Grid Workflow Engine to allow Globus Toolkit 4 service invocations within a WS-BPEL workflow execution. Contact: gusch@informatik.uni-kiel.de
In BIS-Grid1, a project within the German Grid initiative
DGrid, we developed the so called BIS-Grid Workflow Engine2,
which utilizes the Web Services Business Process Execution
Language (WS-BPEL)[
The BIS-Grid-specific services are mainly implemented as stateful WSRF services in which a state is represented by a WSRF instance. For this paper it is sufficient to know that for each workflow execution one WSRF instance is located in UNICORE 6
compute jobs / file transfers
UNICORE Atomic Services uses uses workflow deployment / execution other UNICORE 6 services
BIS-Grid-specific services
other UNICORE 6 services
W
S
B
P
E
L
w
o
r
k
fl
o
w
e
n
g
i
n
e
Fig. 2. Aspects regarding GT4 interoperability by the BIS-Grid Workflow
Engine
and one corresponding WS-BPEL workflow instance in the
WSBPEL workflow engine. It is technically assured that incoming and
outgoing messages within a workflow execution are automatically
mapped to and handled by the correct WSRF instances and
corresponding WS-BPEL workflow instances. For more details
about the BIS-Grid Workflow Engine architecture and the handling
of instances please refer to [
The decision to built on UNICORE 6 enables the BIS-Grid Workflow Engine to execute scientific workflows in UNICORE 6based Grid infrastructures. As the Grid middleware Globus Toolkit 4 (GT4) is also a prevalent middleware in the D-Grid infrastructure, we decided to extend the BIS-Grid Workflow Engine respectively UNICORE 6 by the interoperability with GT4. Both Grid middleware provide (WSRF) services built on similar SOAP stacks and service containers, but they differ in the applied security mechanisms. Thus, establishing the interoperability with GT4 comes with the need that UNICORE 6 supports its security mechanism called Grid Security Infrastructure (GSI). Therefore, two interoperability aspects have to be considered, see Figure 2.
First, the invocation of services deployed in UNICORE 6 by a GT4 client. Second, the invocation of GT4 services within a workflow execution. We decided in BIS-Grid to focus on the second aspect mainly due to its major relevance compared to the first aspect and its less effort required for implementation.
Copyright c 2011 for the individual papers by the papers’ authors. Copying permitted only for private and academic purposes.
In Section 2 we give a brief introduction into GSI. Details about
GSI support in UNICORE 6 and our reasons to focus on GT4
service invocations within a workflow execution are discussed in
Section 3. Implementation details to enable GT4 service invocations
in the BIS-Grid Workflow Engine are presented in Section 4.1. After
discussing related work in Section 5 we conclude the paper with an
outlook to future work in Section 6. Additional details about GT4
interoperability in the BIS-Grid Workflow Engine can also be found
in [
Security in Globus Toolkit 4 is based on the so called Grid Security Infrastructure (GSI) which itself utilizes many existing security standards, see Figure 3. It provides three kinds of message protection, based on WS-Security, WS-SecureConversation and Transport Layer Security (TLS)3, in which different mechanisms for authentication, authorization and delegation are supported, Delegation utilizes so called X.509 proxy certificates. Technically, a proxy certificate is a self-signed certificate which is initially derived from a personal X.509 end entity (user) certificate (EEC) issued by a trusted certificate authority (CA). Such a (first) proxy certificate allows the creation of an arbitrary number of succeeding proxy certificates, see Figure 4.
A proxy certificate is used as an user’s authentication credential
that can be delegated to a GT4 resource allowing it to act on behalf
of an user using its identity. Proxy certificates have the advantage
that the private key of a user certificate is always kept locally,
but proxy certificates lacks of a revocation mechanism. Thus, a
proxy certificate has usually a short lifetime to minimize misuse
in case of its interception. Finally, to allow a basic management
of delegated credentials GT4 provides a delegation service. For
additional information about GSI, please refer to [
As already mentioned, GT4 interoperability in the BIS-Grid Workflow Engine implies to support GSI by UNICORE 6. Thereby, the following interoperability aspects are relevant.
1. GT4 client: A GT4 client invokes a service of the BIS-Grid
Workflow Engine respectively UNICORE 6. 2. GT4 service invocation: The BIS-Grid Workflow Engine respectively UNICORE 6 invokes a GT4 service (within a workflow execution).
Thus, the technical issues concerning the two interoperability aspect are:
Support of WS-Security, WS-SecureConversation and TLS for message protection.
The UNICORE 6 security stack is based on standard X.509 certificates and TLS. Support of WS-SecureConversation and proxy certificates for message protection and delegation, for example, are not covered by UNICORE 6 security mechanisms. However, a UNICORE 6 client can generate a proxy certificate which is added to the header of a SOAP message. This proxy certificate is extracted from the SOAP message within UNICORE 6 and attached to the corresponding internal WSRF instance. It is obvious that central authentication and delegation mechanisms of GSI are not supported by UNICORE 6. And it needs no further technical details to conclude that supporting the first interoperability aspect (GT4 client) results in a major refactoring and extension of the UNICORE 6 security stack. Consequently, we skipped that aspect mainly due to limited project resources. But we can utilize the UNICORE 6 mechanism to create and store proxy certificates in the second interoperability (GT4 service invocation) aspect, which is described in the following.
Regarding the second interoperability aspect (GT4 service invocation) the BIS-Grid Workflow Engine respectively UNICORE 6 acts as a GT4 client. Generally, a GT4 client must support GSI message protection mechanisms as well as proxy certificates4. GT4 already offers client libraries to invoke a GT4 service. Thus, in our approach we built on those GT4 client libraries when a GT4 service has to be invoked within a workflow execution. The needed proxy certificate can be fetched from the internal WSRF instance of the workflow execution, which was previously attached by a UNICORE 6 client. As the invocation of GT4 services within the BIS-Grid Workflow Engine has a major relevance for the project and its implementation effort is significantly less than to support GT4 clients, we implemented this approach. Implementation details are described in the following Section. 4 The support of the GT4 delegation service and grid-mapfile is located on the Grid middleware side. SAML assertions are omitted as they are generally not supported in the D-Grid infrastructure.
Incoming and outgoing messages are processed in UNICORE 6 through handler chains. Such a handler chain can be modified during runtime, which allows a flexible message handling for receiving and sending messages. The BIS-Grid Workflow Engine has a default outgoing handler chain to invoke external UNICORE 6 services5, see Figure 5. The default handler chain is modified at runtime to invoke an external GT4 service, which is described in Section 4.1. This GT4 handler chain was further extended to support the GT4 delegation service, which is described in Section 4.2.
The configuration of external service invocations and its
credentials is located in the so called BIS-Grid Deployment
Descriptor. Figure 6 depicts the configuration of credentials. This
information is used to distinguish between an external UNICORE 6
service and an external GT4 service invocation and to use the correct
credentials. For more details about the BIS-Grid Deployment
Descriptor and especially the configuration of external GT4 service
invocations, please refer to [
GT4 handler chain The default handler chain of the BIS-Grid Workflow Engine ends with the handler OutMessageHandler, see Figure 5. This handler supports TLS and X.509 certificate based authentication and is responsible for the invocation of an external UNICORE 6
service. For a GT4 service invocation the OutMessageHandler is exchanged by the so called AxisMessageHandler at runtime. Its name indicates that the GT4 SOAP stack is based on Axis6. The AxisMessageSender uses the Java GT4 client libraries to configure and execute a GT4 service invocation. Furthermore, the GT4 client libraries itself uses a GSI specific Axis client handler chain, see Figure 7.
Even if UNICORE 6 and GT4 have different security concepts, both implementations are based on some standard Java libraries as WSS4J and XMLSec, but in different and incompatible version. This library version conflicts prevents running both handler chains in one Java virtual machine. Therefore, we implemented two solutions to avoid this conflict. In the first solution we implemented a Java classloader that replaces the default Java class loader in the GT4 handler chain starting with the AxisMessageHandler. In the second solution the AxisMessageHandler uses an RMI call to invoke the GT4 specific part of the GT4 handler chain running in its own Java virtual machine. The Java virtual machine for the corresponding RMI server is started and stopped by an additional section in the UNICORE 6 start-stop-script. 4.2
Support of Globus Toolkit 4 Delegation Service In order to allow a more flexible delegation mechanism GT4 offers a delegation service for basic credential management. A credential is usually represented by a proxy certificate. Technically, the delegation service is implemented as stateful WSRF service. Each delegated proxy certificate is attached to one WSRF instance identifiable by an unique resource key. This resource key can be used, for example, to reference a credential respectively a proxy certificate used for data staging activities in a job submission. If the WSRF instance is destroyed the corresponding proxy certificate is deleted, too.
As first step we examined how the mechanisms works to delegate a proxy certificate as credential via the GT4 delegation service. For this reason we executed a job submission including file staging via the GT4 command line tool globusrun-ws with enabled debug mode and analyzed the exchanged SOAP messages. Furthermore, globusrun-ws -S …..
GT4 DelegationFactoryService
GT4 ManagedJobFactoryService we inspected the source code of globusrun-ws. A proxy certificate derived from a user certificate was previously generated with the GT4 command line tool grid-proxy-init. Regarding the delegation service we identified the following mechanism, cp. Figure 8: 1. The WSRF resource property delegationFactoryEndpoint of the GT4 job submission service called ManagedJobFactoryService is fetched to get the endpoint for the associated GT4 delegation (factory) service called DelegationFactoryService. 2. The endpoint for the DelegationFactoryService is used to get the WSRF resource property CertificateChain. In our case the host certificate of the GT4 resource was returned. 3. The proxy certificate of the user is used to generate a new proxy certificate signed with the public key of the host certificate. 4. The new proxy certificate is delegated to the GT4 resource via the DelegationFactoryService. It returns the resource key DelegationKey, which identifies the corresponding WSRF instance. The DelegationKey is automatically added to the job description as credential reference, before the job is submitted via the ManagedJobFactoryService. 5. After the job execution is finished the delegated proxy certificate is deleted by destroying the corresponding WSRF instance.
Each WS-BPEL workflow engine is capable to create and handle the SOAP messages exchanged in this scenario. But to the best of our knowledge no existing WS-BPEL workflow engine supports the generation of proxy certificates as described above. Thus, we developed a solution to locate the proxy generation into the GT4 handler chain. We assume that a proxy certificate were previously attached to the corresponding WSRF instance of the workflow execution. Based on the WS-BPEL workflow perspective to following mechanism is applied: 1. The WS-BPEL workflow fetches the WSRF resource property delegationFactoryEndpoint of the ManagedJobFactoryService. 2. The WS-BPEL workflow fetches the WSRF resource property CertificateChain respectively the host certificate from the corresponding DelegationFactoryService by using dynamic binding based on the delegationFactoryEndpoint. 3. The WS-BPEL workflow invokes the DelegationFactoryService with the host certificate as credential. 4. The GT4 handler chain of the BIS-Grid Workflow Engine detects the invocation of a GT4 delegation service and generates a new proxy certificate based on the proxy certificate from the WSRF instance and signed with the public key of the host certificate in the SOAP message. Afterwards, the host certificate in the SOAP message is replaced by the new proxy certificate. 5. The GT4 handler chain uses the modified SOAP message for the invocation of the DelegationFactoryService to get the DelegationKey which is returned to the WS-BPEL workflow. 6. The WS-BPEL workflow adds the DelegationKey to the job description as credential reference and submits the job to the ManagedJobFactoryService. 7. The WS-BPEL workflow deletes the delegated proxy certificate by destroying the corresponding WSRF instance after the job execution is finished.
This approach allows a flexible management of delegated proxy certificates as credentials within a WS-BPEL workflow whereas the point of delegating or destroying a credential can be chosen by the workflow designer. 5
The general suitability for WS-BPEL to execute scientific
workflows, especially in WSRF-based infrastructures, is shown in
several publications [
To the best of our knowledge, no comparable solution exists to invoke the GT4 delegation service within WS-BPEL. 6
In this paper we discussed the interoperability of the BIS-Grid Workflow Engine with the Grid middleware Globus Toolkit 4 (GT4) which results in a discussion about how to enable UNICORE 6 to support the Grid Security Infrastructure (GSI) used in GT4. For this reason we examined the use of GT4 clients and the invocation of GT4 services. Due to its major significance and less effort in realization we implemented a solution for the BIS-Grid Workflow Engine that allows the invocation GSI-secured GT4 services within WS-BPEL as well as the delegation of proxy certificates via the GT4 delegation service.
Currently, we use the recommended lifetime (about 11 days) when a new proxy certificate is generated for credential delegation. We intend to allow an arbitrary proxy certificate lifetime based on additional information added to the SOAP message beside the host certificate when the GT4 delegation service is invoked.