Services are loosely coupled interacting software components. Since two or more services are usually composed to one software system, the behavior of an implemented service should not di er to its speci cation. Therefore we propose an approach to test, if the implementation contains unspeci ed behavior. Due to the interacting nature of services this is a nontrivial task.
In the paradigm of service-oriented computing (SOC) [
Best practices propose that systems should be speci ed prior to their implementation. A speci cation of a service (e. g., an abstract WS-BPEL process) describes its interface and the possible interactions with other services. Furthermore it already contains all relevant internal decisions of the control ow (e. g., the criteria whether or not a credit should be approved) and data about nonfunctional properties. In contrast, implementation-speci c details (e. g., whether amounts are represented as integers or oats) are usually not yet de ned.
Ideally, an implementation should conform to its speci cation. This means, all speci ed properties should be implemented. But it is obviously not always possible to verify the conformance in a formal proof. Furthermore veri cation is excessively elaborate and therewith often too expensive. Thus, it is customary to test the implementation.
As Dijkstra stated, program testing can be a very e ective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. To nevertheless be able to make a statement about the correctness of an implementation, a test suite with a signi cant number of test cases is necessary.
Testing services received much attention both in academia and industry. However, the communicating nature of services is often neglected. Instead of taking the new characteristics of SOC into account, existing work on testing classical software systems was slightly adapted to cope with the new languages used to implement services. Consequently, these approaches only take the internal behavior of a service into account, but ignore the interactions with other services. Others in turn are restricted to stateless remote-procedure calls. Both might lead to undetected errors.
However, communication is an essential part of the speci cation of interacting
services and consequently it is crucial to check it during conformance testing.
That means, any interaction derived from the speci cation also has to be a correct
interaction of the implementation and vice versa. In [
This paper is organized as follows. Section 2 describes how a service implementation can be tested using partners. Section 3 recalls the approach to generate test cases and introduces an example protocol we use to demonstrate the approach. Section 4 provides the main contribution of this paper with the elaboration of test cases that are able to detect unspeci ed behavior of an implementation. Section 5 concludes the paper and gives directions to future work. 2
An adequate software test needs to check all speci ed properties. Due to the fact that protocols are an essential part of service speci cations, they likewise have to be considered during service testing. A protocol describes in which state a service can send or receive which messages (and also which messages are not allowed in that state). In this way, all possible interactions with other services are de ned which leads to an implicitly speci ed set S of all partners. Thereby a partner is again a service. If the speci ed service is implemented correctly, a partner always interacts deadlock-freely with it. In contrast non-partners violate the protocol; that is, during interaction the implemented service might deadlock. For this paper, we assume \correct interaction" between two services means deadlock freedom of their composition. We are well aware that there are other possibilities for de ning \correct interaction". Nevertheless, deadlock freedom will certainly be art of any de nition, so this paper can be seen as a step towards a more sophisticated setting.
We claim that like a function call is the most natural test case to test a
function of a classical program, a partner service is likewise the most natural
test case for a given service [
specified, yet unimplemented partners (undesired)
S
I implemented, yet unspecified partners (undesired)
To calculate S we can transform the speci cation into a formal model from
which a description (operating guideline) of all partners can be generated [
The general test procedure for interacting services can be sketched as follows. We have a test suite containing the test cases P 2 S and another one containing the test cases Q 2= S. The service to be tested is deployed in a testing environment together with the two test suites. To process both test suites, the contained test cases are executed one after the other. Thereby, each test case interacts by message exchange with the service under test. The testing environment then is responsible for logging and evaluating exchanged messages. We thereby assume the test environment is able to detect whether the implementation terminates properly; for instance, whether a BPEL process instance has completed successfully.
A test run with a service P 2 S fails if the test run terminates, but the implementation did not reach a nal state. Then a deadlock has occurred and the implementation is incorrect. P can serve as a witness that the implementation excludes at least this speci ed partner service (SnI 6= ;). A test run that does not terminate, or terminates, but leaves the implementation in a nal state, is inconclusive; that is, the run neither witnesses an error nor can proof overall correctness. Note that our approach is currently centered around deadlock freedom. In this setting, in nite runs are considered correct, because they do not contain a deadlock. Furthermore, the test environment cannot detect in nite runs and has to abort the test at some point, yielding an inconclusive result. But even if, for each partner service P 2 S, the run terminates and the implementation reaches a nal state, we cannot conclude that the implementation does not exclude a speci ed partner service. We only know, that there exists a successful run for each speci ed partner. Since during testing the partner service (the tester) cannot control all internal decisions of the service. Hence, an incorrect part of the implementation may not executed during the test at all. However, a systematic test approach with a signi cant number of test cases can increase the likelihood of detecting an error.
The set S contains a large number of many services (usually even in nitely
many). It is therefore not practical to test a service with every P 2 S. In [
Canonically, a test run with a non-partner service Q 2= S fails, if the test run terminates and the implementation is in a nal state. Thereby, Q is a witness that the implementation supports undesired partner services (InS 6= ;). Test runs of the composition of the implementation and the test case that neither terminate nor leave the implementation in a nal state are inconclusive.
In Sect. 4 we will explain how the non-partner can be derived from the speci cation. This is a nontrivial task, because not all partner services Q 2= S are suitable for testing. As demonstrated by the following example, there exist some Q0 2= S which can fail even trough the implementation is correct. 3
To formally reason about the speci cation of a service, an exact mathematical
model is needed. We use open nets [
offer reject accept invoice ?o
!r !a ?i ω !invoice ∨ !offer
From this model, an operating guideline (OG) [
Two test cases are depicted in Fig. 2(c) and 2(d). While the rst test case sends the invoice only after receipt of an acceptance message, the second exploits asynchronicity and sends the invoice right after sending the o er. In both cases, the OG's annotations are ful lled. 4
As mentioned earlier, the operating guideline can used to generate the test suite containing all necessary partners the test if SnI = ; holds. To be able to make a statement about these InS deviations (i.e., unspeci ed partners), a di erent approach is needed. This is due to the fact that a service that is not characterized by the does not necessarily yield a deadlock. For example, a service without the ability to receive a rejection message (see Fig. 3(a)) may deadlock with the buyer service if the latter rejects the o er. If, however, the o er is accepted, no deadlock occurs. Hence, any result of this test case would be inconclusive.
To this end, the operating guideline of the speci ed protocol cannot be used to derive InS-test cases. To characterize all services that are expected to deadlock when composed to the implementation, we need to reinterpret the nal states of the protocol: If we complement the set of nal markings, then the operating guideline of this anti-protocol (called the anti operating guideline) characterizes partners that interact without deadlock with the anti-protocol. That is; the interaction terminates in a nal state of the anti-protocol. This in turn is a deadlock of the original protocol. Hence, the anti operating guideline characterizes partners that are expected to deadlock with the original protocol.
For the example of Fig. 2(a), any marking but [!] would be a nal marking of the anti-protocol. Note that the structure of the anti-operating guideline of Fig. 2(b) is very similar to the original operating guideline (cf. Fig. 3(b)) and we see that the interactions that would lead to the a nal state in Fig. 2(b) (e.g. !o er ?accept !invoice) lead to the only state without nal in the annotation in Fig. 3(b).
Three test cases characterized by the anti operating guideline are depicted
in Fig. 3(c){3(e): the rst two exploit the fact that premature termination will
lead to a deadlock with a conformant implementation, whereas the last test case
avoids a valid nal marking by sending another o er.
?reject
Several works exist to systematize testing of Web services (see [
To the best of our knowledge, none of the existing testing approaches take
stateful business protocols implemented by Web services into account, but mainly
assume stateless remote procedure calls (i. e., request-response operations), see for
instance [
Finally, several tools such as the Oracle BPEL Process Manager [
We presented an approach to generate test cases from a speci cation protocol
and show how they can be used to check whether an implementation introduces
unspeci ed behavior. This test suite is not complete in the sense that we cannot
detect all deviations from the speci cation. The reason for this lies in the nature
of services in which decisions may not always be enforceable by the environment.
To this end, we introduced anti operating guidelines to characterize SnI test
cases that must deadlock when composed to a compliant implementation. The
results of this paper are de ned in terms of open nets and are independent of
a concrete protocol speci cation language. With existing translation, e.g. from
BPEL to open nets and vice versa [
In future work, we plan to propose a test case coverage criterion to select
a small number of test cases which still can detect the same errors that are
detectable with the complete set of test cases. Indications are that this criterion
should be similar to the criterion we proposed for InS test cases [