Grid Workflow Workshop 2011 GWW 2011 Interoperability of the BIS-Grid Workflow Engine with Globus Toolkit 4 Guido Scherp 1 and Wilhelm Hasselbring 1 1 Software Engineering Group, University of Kiel, 24098 Kiel, Germany UNICORE 6 ActiveBPEL ABSTRACT In the D-Grid project BIS-Grid we developed the BIS-Grid Workflow UNICORE Atomic Services compute jobs / file transfers Engine in order to utilize a common WS-BPEL workflow engine for WS-BPEL workflow engine scientific workflow execution in WSRF-based Grid infrastructures. uses The BIS-Grid Workflow Engine itself is built on the Grid middleware BIS-Grid-specific services UNICORE 6 to benefit from its security mechanisms and to workflow deployment / execution automatically gain interoperability with UNICORE 6-based Grid uses infrastructures. As beside UNICORE 6 the Grid middleware Globus other UNICORE 6 services Toolkit 4 is prevalent in the D-Grid infrastructure, we decided to examine the interoperability of the BIS-Grid Workflow Engine with other UNICORE 6 services Globus Toolkit 4. The interoperability with Globus Toolkit 4 concerns basically the support of its Grid Security Infrastructure (GSI) by UNICORE 6. In Fig. 1. General Architecture of the BIS-Grid Workflow Engine this paper we describe to what extend GSI is already supported by UNICORE 6 and what is missing. We furthermore give some details about our implementation for the BIS-Grid Workflow Engine to allow Globus Toolkit 4 service invocations within a WS-BPEL workflow execution. Contact: gusch@informatik.uni-kiel.de Fig. 2. Aspects regarding GT4 interoperability by the BIS-Grid Workflow Engine 1 INTRODUCTION In BIS-Grid1 , a project within the German Grid initiative D- and one corresponding WS-BPEL workflow instance in the WS- Grid, we developed the so called BIS-Grid Workflow Engine2 , BPEL workflow engine. It is technically assured that incoming and which utilizes the Web Services Business Process Execution outgoing messages within a workflow execution are automatically Language (WS-BPEL)[12], a widely accepted standard for service mapped to and handled by the correct WSRF instances and orchestrations, to execute scientific workflows in WSRF[11]-based corresponding WS-BPEL workflow instances. For more details Grid infrastructures. Mainly due to a lack of supported security about the BIS-Grid Workflow Engine architecture and the handling mechanisms, common WS-BPEL workflow engines are usually of instances please refer to [7, 8]. not interoperable with Grid middleware and Grid clients. Thus, The decision to built on UNICORE 6 enables the BIS-Grid we implemented a Grid wrapper for arbitrary WS-BPEL workflow Workflow Engine to execute scientific workflows in UNICORE 6- engines through specific BIS-Grid-specific services for the Grid based Grid infrastructures. As the Grid middleware Globus middleware UNICORE 6, see Figure 1. Based on this architecture Toolkit 4 (GT4) is also a prevalent middleware in the D-Grid we were able to use solely standard WS-BPEL elements for infrastructure, we decided to extend the BIS-Grid Workflow Engine workflow execution that in principle allows a free choice of any respectively UNICORE 6 by the interoperability with GT4. Both WS-BPEL compliant workflow engine. Furthermore, the BIS-Grid- Grid middleware provide (WSRF) services built on similar SOAP specific services in UNICORE 6 that wrap a WS-BPEL workflow stacks and service containers, but they differ in the applied engine could be implemented by solely using UNICORE 6 built-in security mechanisms. Thus, establishing the interoperability with mechanisms. GT4 comes with the need that UNICORE 6 supports its security The BIS-Grid-specific services are mainly implemented as mechanism called Grid Security Infrastructure (GSI). Therefore, stateful WSRF services in which a state is represented by a WSRF two interoperability aspects have to be considered, see Figure 2. instance. For this paper it is sufficient to know that for each First, the invocation of services deployed in UNICORE 6 by a GT4 workflow execution one WSRF instance is located in UNICORE 6 client. Second, the invocation of GT4 services within a workflow execution. We decided in BIS-Grid to focus on the second aspect 1 http://www.bisgrid.de mainly due to its major relevance compared to the first aspect and 2 http://bis-grid.sourceforge.net/ its less effort required for implementation. Copyright c 2011 for the individual papers by the papers’ authors. Copying permitted only for private and academic purposes. 1 Guido Scherp and Wilhelm Hasselbring 3 GSI SUPPORT IN UNICORE 6 As already mentioned, GT4 interoperability in the BIS-Grid Workflow Engine implies to support GSI by UNICORE 6. Thereby, the following interoperability aspects are relevant. 1. GT4 client: A GT4 client invokes a service of the BIS-Grid Workflow Engine respectively UNICORE 6. 2. GT4 service invocation: The BIS-Grid Workflow Engine respectively UNICORE 6 invokes a GT4 service (within a workflow execution). Fig. 3. Overview of GSI (from [14]) Thus, the technical issues concerning the two interoperability aspect are: • Support of WS-Security, WS-SecureConversation and TLS for message protection. • Support of proxy certificates for delegation. • Support of the GT4 delegation service for credential management respectively proxy certificate management. Fig. 4. Chain of Proxy Certificates (from http://www.globus.org/security/overview.html) • Support of SAML and grid-mapfile for authorization. The UNICORE 6 security stack is based on standard X.509 In Section 2 we give a brief introduction into GSI. Details about certificates and TLS. Support of WS-SecureConversation and proxy GSI support in UNICORE 6 and our reasons to focus on GT4 certificates for message protection and delegation, for example, service invocations within a workflow execution are discussed in are not covered by UNICORE 6 security mechanisms. However, Section 3. Implementation details to enable GT4 service invocations a UNICORE 6 client can generate a proxy certificate which is in the BIS-Grid Workflow Engine are presented in Section 4.1. After added to the header of a SOAP message. This proxy certificate discussing related work in Section 5 we conclude the paper with an is extracted from the SOAP message within UNICORE 6 and outlook to future work in Section 6. Additional details about GT4 attached to the corresponding internal WSRF instance. It is obvious interoperability in the BIS-Grid Workflow Engine can also be found that central authentication and delegation mechanisms of GSI are in [9]. not supported by UNICORE 6. And it needs no further technical details to conclude that supporting the first interoperability aspect (GT4 client) results in a major refactoring and extension of the 2 GRID SECURITY INFRASTRUCTURE UNICORE 6 security stack. Consequently, we skipped that aspect mainly due to limited project resources. But we can utilize the Security in Globus Toolkit 4 is based on the so called Grid UNICORE 6 mechanism to create and store proxy certificates in the Security Infrastructure (GSI) which itself utilizes many existing second interoperability (GT4 service invocation) aspect, security standards, see Figure 3. It provides three kinds of message which is described in the following. protection, based on WS-Security, WS-SecureConversation and Regarding the second interoperability aspect (GT4 service Transport Layer Security (TLS)3 , in which different mechanisms invocation) the BIS-Grid Workflow Engine respectively for authentication, authorization and delegation are supported, UNICORE 6 acts as a GT4 client. Generally, a GT4 client must Delegation utilizes so called X.509 proxy certificates. Technically, a support GSI message protection mechanisms as well as proxy proxy certificate is a self-signed certificate which is initially derived certificates4 . GT4 already offers client libraries to invoke a GT4 from a personal X.509 end entity (user) certificate (EEC) issued by service. Thus, in our approach we built on those GT4 client libraries a trusted certificate authority (CA). Such a (first) proxy certificate when a GT4 service has to be invoked within a workflow execution. allows the creation of an arbitrary number of succeeding proxy The needed proxy certificate can be fetched from the internal WSRF certificates, see Figure 4. instance of the workflow execution, which was previously attached A proxy certificate is used as an user’s authentication credential by a UNICORE 6 client. As the invocation of GT4 services within that can be delegated to a GT4 resource allowing it to act on behalf the BIS-Grid Workflow Engine has a major relevance for the project of an user using its identity. Proxy certificates have the advantage and its implementation effort is significantly less than to support that the private key of a user certificate is always kept locally, GT4 clients, we implemented this approach. Implementation details but proxy certificates lacks of a revocation mechanism. Thus, a are described in the following Section. proxy certificate has usually a short lifetime to minimize misuse in case of its interception. Finally, to allow a basic management of delegated credentials GT4 provides a delegation service. For additional information about GSI, please refer to [14]. 4 The support of the GT4 delegation service and grid-mapfile is located on the Grid middleware side. SAML assertions are omitted as they are generally 3 Also known as Secure Sockets Layer (SSL) not supported in the D-Grid infrastructure. 2 Interoperability of the BIS-Grid Workflow Engine with Globus Toolkit 4 Fig. 5. Default UNICORE 6 handler chain of the BIS-Grid Workflow Engine (from [9]). Fig. 7. GT4 handler chain (from [9]). service. For a GT4 service invocation the OutMessageHandler is exchanged by the so called AxisMessageHandler at runtime. Its name indicates that the GT4 SOAP stack is based on Axis6 . The AxisMessageSender uses the Java GT4 client libraries to configure and execute a GT4 service invocation. Furthermore, the GT4 client libraries itself uses a GSI specific Axis client handler chain, see Figure 7. Fig. 6. Credential description in the BIS-Grid Deployment Descriptor (from Even if UNICORE 6 and GT4 have different security concepts, [9]). both implementations are based on some standard Java libraries as WSS4J and XMLSec, but in different and incompatible version. This library version conflicts prevents running both handler chains 4 INVOCATION OF GLOBUS TOOLKIT 4 in one Java virtual machine. Therefore, we implemented two SERVICES WITHIN THE BIS-GRID WORKFLOW solutions to avoid this conflict. In the first solution we implemented ENGINE a Java classloader that replaces the default Java class loader in the GT4 handler chain starting with the AxisMessageHandler. In Incoming and outgoing messages are processed in UNICORE 6 the second solution the AxisMessageHandler uses an RMI call through handler chains. Such a handler chain can be modified during to invoke the GT4 specific part of the GT4 handler chain running runtime, which allows a flexible message handling for receiving and in its own Java virtual machine. The Java virtual machine for the sending messages. The BIS-Grid Workflow Engine has a default corresponding RMI server is started and stopped by an additional outgoing handler chain to invoke external UNICORE 6 services5 , section in the UNICORE 6 start-stop-script. see Figure 5. The default handler chain is modified at runtime to invoke an external GT4 service, which is described in Section 4.1. This GT4 handler chain was further extended to support the GT4 4.2 Support of Globus Toolkit 4 Delegation Service delegation service, which is described in Section 4.2. In order to allow a more flexible delegation mechanism GT4 The configuration of external service invocations and its offers a delegation service for basic credential management. A credentials is located in the so called BIS-Grid Deployment credential is usually represented by a proxy certificate. Technically, Descriptor. Figure 6 depicts the configuration of credentials. This the delegation service is implemented as stateful WSRF service. information is used to distinguish between an external UNICORE 6 Each delegated proxy certificate is attached to one WSRF instance service and an external GT4 service invocation and to use the correct identifiable by an unique resource key. This resource key can be credentials. For more details about the BIS-Grid Deployment used, for example, to reference a credential respectively a proxy Descriptor and especially the configuration of external GT4 service certificate used for data staging activities in a job submission. If the invocations, please refer to [8, 9]. WSRF instance is destroyed the corresponding proxy certificate is deleted, too. 4.1 GT4 handler chain As first step we examined how the mechanisms works to delegate The default handler chain of the BIS-Grid Workflow Engine ends a proxy certificate as credential via the GT4 delegation service. For with the handler OutMessageHandler, see Figure 5. This this reason we executed a job submission including file staging via handler supports TLS and X.509 certificate based authentication the GT4 command line tool globusrun-ws with enabled debug and is responsible for the invocation of an external UNICORE 6 mode and analyzed the exchanged SOAP messages. Furthermore, 5 Standard web services are supported, too. 6 http://axis.apache.org/axis/ 3 Guido Scherp and Wilhelm Hasselbring 1. The WS-BPEL workflow fetches the WSRF resource property delegationFactoryEndpoint of the ManagedJob- FactoryService. 2. The WS-BPEL workflow fetches the WSRF resource globusrun-ws GT4 GT4 -S ….. DelegationFactoryService ManagedJobFactoryService property CertificateChain respectively the host certificate from the corresponding Delegation- getRP: (staging)delegationFactoryEndpoint FactoryService by using dynamic binding based on <(staging)delegationFactoryEndpoint> the delegationFactoryEndpoint. getRP: CertificateChain 3. The WS-BPEL workflow invokes the Delegation- FactoryService with the host certificate as RequestSecurityToken: credential. CreateManagedJob: … ... 4. The GT4 handler chain of the BIS-Grid Workflow Engine detects the invocation of a GT4 delegation service and generates a new proxy certificate based on the proxy certificate from the WSRF instance and signed with the public key of the host certificate in the SOAP message. Afterwards, the host Fig. 8. Utilization of the GT4 delegation service in a job submission. certificate in the SOAP message is replaced by the new proxy certificate. 5. The GT4 handler chain uses the modified SOAP message for the invocation of the DelegationFactoryService to get the DelegationKey which is returned to the WS-BPEL we inspected the source code of globusrun-ws. A proxy workflow. certificate derived from a user certificate was previously generated 6. The WS-BPEL workflow adds the DelegationKey to the with the GT4 command line tool grid-proxy-init. Regarding job description as credential reference and submits the job to the delegation service we identified the following mechanism, cp. the ManagedJobFactoryService. Figure 8: 7. The WS-BPEL workflow deletes the delegated proxy certificate 1. The WSRF resource property delegationFactory- by destroying the corresponding WSRF instance after the job Endpoint of the GT4 job submission service called execution is finished. ManagedJobFactoryService is fetched to get the endpoint for the associated GT4 delegation (factory) service This approach allows a flexible management of delegated proxy called DelegationFactoryService. certificates as credentials within a WS-BPEL workflow whereas the point of delegating or destroying a credential can be chosen by the 2. The endpoint for the DelegationFactoryService is workflow designer. used to get the WSRF resource property Certificate- Chain. In our case the host certificate of the GT4 resource was returned. 3. The proxy certificate of the user is used to generate a new proxy 5 RELATED WORK certificate signed with the public key of the host certificate. The general suitability for WS-BPEL to execute scientific 4. The new proxy certificate is delegated to the GT4 resource via workflows, especially in WSRF-based infrastructures, is shown in the DelegationFactoryService. It returns the resource several publications [15, 4, 16, 5, 10, 13, 1, 2, 3]. Regarding the key DelegationKey, which identifies the corresponding complexity to invoke stateful WSRF services some approaches as WSRF instance. The DelegationKey is automatically [15, 10, 2, 3] suggest extensions to WS-BPEL7 or its predecessor added to the job description as credential reference, before the BPEL4WS. Consequently, both a WS-BPEL/BPEL4WS workflow job is submitted via the ManagedJobFactoryService. editor and a WS-BPEL/BPEL4WS workflow engine must be 5. After the job execution is finished the delegated proxy provided that supports those language extensions. A language certificate is deleted by destroying the corresponding WSRF extension for BPEL4WS, for example, is presented in [2, 3] that instance. allows the invocation of stateful, WSRF-based GT4 services. To support these new language elements an existing workflow editor Each WS-BPEL workflow engine is capable to create and handle and workflow engine were extended. This approach hampers a the SOAP messages exchanged in this scenario. But to the best of migration to newer versions of the workflow editor or workflow our knowledge no existing WS-BPEL workflow engine supports engine. However, we reused the solution to utilize the GSI specific the generation of proxy certificates as described above. Thus, Axis client handler chains of the GT4 client libraries for GT4 service we developed a solution to locate the proxy generation into the invocations. GT4 handler chain. We assume that a proxy certificate were previously attached to the corresponding WSRF instance of the workflow execution. Based on the WS-BPEL workflow perspective 7 The standard provides an extension mechanism to define language to following mechanism is applied: extensions. 4 Interoperability of the BIS-Grid Workflow Engine with Globus Toolkit 4 In [6] the utilization of conventional workflow technology as WS- [3]Tim Dörnemann, Matthew Smith, and Bernd Freisleben. Composition and BPEL for scientific simulations is discussed. A discussion about the execution of secure workflows in wsrf-grids. Cluster Computing and the Grid, IEEE International Symposium on, 0:122–129, 2008. interoperability with GSI is not included. [4]Wolfgang Emmerich, Ben Butchart, Liang Chen, Bruno Wassermann, and Sarah To the best of our knowledge, no comparable solution exists to Price. Grid Service Orchestration Using the Business Process Execution Language invoke the GT4 delegation service within WS-BPEL. (BPEL). Journal of Grid Computing, 3(3-4):283–304, September 2005. [5]Onyeka Ezenwoye, S. Masoud Sadjadi, Ariel Cary, and Michael Robinson. Orchestrating WSRF-based Grid Services. Technical report, School of Computing and Information Sciences, Florida International University, April 2007. 6 CONCLUSION AND FUTURE WORK [6]Katharina Görlach, Mirko Sonntag, Dimka Karastoyanova, Frank Leymann, and Michael Reiter. Conventional Workflow Technology for Scientific Simulation, In this paper we discussed the interoperability of the BIS-Grid pages 1–31. Guide to e-Science. Springer-Verlag, März 2011. Workflow Engine with the Grid middleware Globus Toolkit 4 (GT4) [7]Stefan Gudenkauf, Felix Heine, Andre Höing, Jens Lischka, Holger Nitsche, and which results in a discussion about how to enable UNICORE 6 to Guido Scherp. BIS-Grid Deliverable 3.1: Specification. Technical report, BIS- support the Grid Security Infrastructure (GSI) used in GT4. For Grid, December 2007. [8]Stefan Gudenkauf, Andre Höing, Dirk Meister, Holger Nitsche, and Guido this reason we examined the use of GT4 clients and the invocation Scherp. BIS-Grid Deliverable 3.4: Final Version of the WS-BPEL Engine. of GT4 services. Due to its major significance and less effort in Technical report, BIS-Grid, May 2009. realization we implemented a solution for the BIS-Grid Workflow [9]Stefan Gudenkauf, Andre Höing, and Guido Scherp. BIS-Grid Deliverable 3.5: Engine that allows the invocation GSI-secured GT4 services within GT4 interoperability. Technical report, BIS-Grid, April 2010. [10]Frank Leymann. Choreography for the Grid: towards fitting BPEL to the resource WS-BPEL as well as the delegation of proxy certificates via the GT4 framework: Research Articles. Concurr. Comput. : Pract. Exper., 18(10):1201– delegation service. 1217, 2006. Currently, we use the recommended lifetime (about 11 days) [11]OASIS. Web services resource framework (wsrf). http://www.oasis- when a new proxy certificate is generated for credential delegation. open.org/committees/wsrf/. We intend to allow an arbitrary proxy certificate lifetime based on [12]OASIS. Web Services Business Process Execution Language Version 2.0. http://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.html, 04 2007. additional information added to the SOAP message beside the host [13]Wei Tan, Liana Fong, and Norman Bobroff. BPEL4Job: A Fault-Handling Design certificate when the GT4 delegation service is invoked. for Job Flow Management. In ICSOC ’07: Proceedings of the 5th international conference on Service-Oriented Computing, pages 27–42, Berlin, Heidelberg, 2007. Springer-Verlag. [14]The Globus Security Team. Globus toolkit version 4 grid security infrastructure: REFERENCES A standards perspective. Technical report, 2005. [1]Asif Akram, David Meredith, and Rob Allan. Evaluation of BPEL to Scientific [15]Yong Wang, Chunming Hu, and Jinpeng Huai. A New Grid Workflow Description Workflows. In CCGRID ’06: Proceedings of the Sixth IEEE International Language. In SCC ’05: Proceedings of the 2005 IEEE International Conference Symposium on Cluster Computing and the Grid, pages 269–274, Washington, on Services Computing, pages 257–260, Washington, DC, USA, 2005. IEEE DC, USA, 2006. IEEE Computer Society. Computer Society. [2]Tim Dörnemann, Thomas Friese, Sergej Herdt, Ernst Juhnke, and Bernd [16]Bruno Wassermann, Wolfgang Emmerich, Ben Butchart, Nick Cameron, Liang Freisleben. Grid Workflow Modelling Using Grid-Specific BPEL Extensions. chen, and Jignesh Patel. Sedna: A BPEL-Based Environment for Visual Scientific 2007. Workflow Modeling, chapter 0, pages 427–448. Springer, 2006. 5