When developing an existing service further, new functionalities can be added and existing functionalities can be changed or removed. Consequently, also test cases have to be added to or removed from the existing test suite. In this paper, we present an idea how a test suite can be adjusted to these changes. Thereby, we focus on modifications concerning the communication behavior of a service.
Testing is an effective instrument to detect errors in software. However, for thorough testing the number of required test cases increases rapidly with growing software complexity. To reduce the effort for testing, test cases are generated and executed by tool support as much as possible.
In the paradigm of service-oriented computing (SOC) [
The development of a service is rarely finished with its initial release. In most cases, maintenance follows. This includes both, correction (i.e., fixing of discovered bugs) and integration of enhancements and new features. To adequately test the new service version, also the test suite must be revised: New test cases have to be added and existing test cases may be removed. Using the updated test suite, all (new) parts of the new version are taken into account during testing. Further, one can be sure that a failed test case indicates a bug in the implementation and is not caused by the specified changes.
In this paper, we present an idea how a test suite, generated by our approach [
The rest of this paper is organized as follows: Section 2 introduces basic formalisms.
Section 3 recalls the testing approach of [
Black box testing [
Figure 1(a) shows the abstract specification Q of a simple online shop Q. The shop expects a costumer to log in and to choose one of the payment methods: cash on delivery (cod), credit card (cc), or bank transfer (bt). However, for setting the payment method the login must be accepted first, otherwise the login is rejected and the user has to retry to login. The final marking of this net is the marking [!] which only marks the place !: the control flow reached its end and all message channels are empty.
A partner P for a given service S is again a service that interacts deadlock freely with S. That means, no deadlocks (except the final state) can occur in the composition of the both services. To model the communication behavior of a partner P , we use again open nets. Since we also abstract from the data, the net models an abstract partner P of P .
As an example we consider a partner R for the service Q (cf. Fig. 1(a)). Its abstract version R is depicted in Fig.1(b). To check deadlock freendom between Q and R it is sufficient to compose their abstract versions Q and R . For this purpose, we only need to merge interface places with the same label. In the composed system, these merged (a) Q (b) R (c) OGQ interfaces place become to ordinary places. In our example the composition of Q and R is free of deadlocks (except the final marking).
The set of all partners of a given service S can be characterized by its operating
guideline OGS [
Figure 1(c) depicts the operating guideline OGQ of the abstract specification Q (see Fig. 1(a)). In the graphical representation of operating guidelines, sending events are preceded by \!" and receiving events are preceded by \?". Initial states have an incoming arc from nowhere. Final states are double-lined. Since the reachability graph of R is a subgraph of OGQ (see the bolded part of OGQ ) and the Boolean formulas of the “touched” states are fulfilled, R is characterized by OGQ ; that is, R is a partner of Q. 3
To test whether the implemented service interacts correctly with its environment we create partner services as test cases. As they are derived from the specification they interact by design deadlock freely with the service under test (SUT). If, however, during testing a deadlock still occurs, we can conclude that the implementation contains (at least) one error.
The generation of test cases for a service S bases on the operating guideline OGS .
As there is usually a high number of abstract partners characterized by OGS , it is not
recommended to take every partner for testing. But among the test cases there is some
redundancy, such that is possible to to select a subset without reducing the test suite’s
quality [
As already mentioned in the introduction, it can be required to modify an existing service from time to time. By introducing enhancements and new features the specification is changed. This entails existing test cases to become invalid. That makes the test suite inconclusive. Further, after adding new functionalities, there are parts in the new version of the service which are not taken into account by the old test suite.
To ensure a thorough testing, the test suite needs to be revised. Instead of discarding all old test cases and deriving a new test suite from the scratch, we aim to keep as much old test cases as possible. This is desirable because the procedure for generating the test case (see Sect. 3) is not fully automatic. Thus, this strategy helps to reduce the manual effort. Further, it is reasonable to use the same data in the remaining test cases. By generating test cases from scratch, these data information would be lost.
First, we describe in the following how invalid test cases can be removed from the test suite. Afterward, we suggest how the new test cases can be found. Making the test suite conclusive. Integrating new functionalities can change the control flow and with it the communication behavior such that existing test cases become invalid. This is demonstrated by the following example. Figure 3 shows a modification Qnew for the specification in Fig. 1(a): If the credit card (cc) is chosen as payment method, a voucher is sent to the customer. Now, the composition of Qnew and R (see Fig. 1(b)) contains a deadlock because the voucher cannot be received from R . When testing the new version of the simple online shop with R the test fails even through the implementation is correct. In converse, if the implementation is incorrect the test can succeed.
To achieve a conclusive test suite for the new specification we discard the invalid test cases. They can be detected using the matching algorithm for operating guidelines (cf. Sect. 2). For each test case we check whether it matches with the operating guideline of the new specification. This can be done using the tool Cosme3. Non-matching test cases are invalid and have to be removed. After this procedure we have a conclusive test suite. Adding missing test cases. To cover new parts during testing, test cases have to be added to the test suite. For a systematic approach we make the following consideration: Let OGS be the operating guideline of a specification and OGSnew the operating guideline of the new specification. Let TS be the set of test cases characterized of OGS and TSnew be the set of the test cases characterized by OGSnew . When calculating the difference TSnew nTS we get exactly the set of test cases for testing the new parts.
Unfortunately, the result of the difference operation cannot be represented as an
operating guideline. Instead, it can be represented as an extended annotated service
automaton (EAA for short) [
Formally, an operating guideline can easily be transformed into an EAA by setting
the global Boolean formula to true; that is, there are no additional requirements for the
states. For EAAs all basic set operations (including the difference) are defined [
Thus, we are able to calculate the set TSnew nTS based from the corresponding operating guidelines. The result is an EAA that characterizes the test cases to be added.
In the next step, the (abstract) test cases needs to extract from the EAA and transformed into executable test cases. 5
The set of test cases to be added, can be only represented by EAAs, an extension of
operating guidelines. In contrast to operating guidelines, the extraction of test cases from
an EAA is a time-consuming task. In [
In this paper, we presented a concept to manage test suites for services in an iterative development process. With the contained test cases the communication behavior can be tested thoroughly. We demonstrated how invalid test cases can be detected and removed from the test suite and we principle show how new test cases can be added. Thereby, most steps are supported by existing tools.