Two problems are common in agile based software development approaches. First, software requirements change frequently and therefore they are difficult to maintain and document. Second, test case development takes time and effort and thus the testing process is often delayed. To provide solution to these problems, we suggest using Action-Triad method for capturing the software application requirements. This method can be used to create conceptual models that can be used as documentation in agile development. The method can also create requirements traceability matrix (RTM) and test cases. When the requirements change, the conceptual models, RTM, and test cases can be regenerated. The method is demonstrated here using a case study.
Agile practices advocate on iterative and test-driven software development strategy
where the focus is on working software and not on extensive documentation [1, 2].
Agile practices focus on capturing the requirements through user stories which are
constantly updated in the development cycle [
Kaner [7] mentions that designing good test cases is a complex art as the process of test creation is subjective and is based on testers’ domain knowledge. Also there is no clear method in writing test cases. For example Kaner mentions that if an application has 20 variables, then should we create one test case combining all variables or multiple test cases for each variable. In agile development, creating test cases is particularly challenging as the application changes frequently during the development process. Based on the above discussion, two questions can be raised in the context of agile practices: (1) how to document requirements in a standardized way so that the requirement changes are captured quickly? and (2) how to create test cases consistently and quickly prior to software application development? To answer these questions, this paper proposes an action based method for capturing requirements in a structured way and creating test cases based on these requirements. The structured requirements can be documented and changes in the requirements can be controlled through the use of the method. The method can be also used for creating test cases thus test cases can be created before the software application is developed. In section 2, this method is described and in section 3, a sample case study is presented. Section 4 is the discussion section where the benefits for practice are discussed. 2
A function represents activities that are performed by agents.
An object represents non-agents in the domain with which the agents act.
Objects can be tangible (e.g. Phone) or intangible (e.g. Web site).
Dimensions describe the objects or agents in measurable form.
Dimensions have instances that are generally expressed in text or numbers.
Software application requirements can be decomposed into a set of triads. These triads can have precedence (i.e. function of one triad needs to be performed earlier than the other). Additional concepts - instances, scenario, expected results, and requirement ID are identified that are relevant to software testing. A dimension can have multiple values or instances. Each instance can have a positive or a negative scenario. Positive scenario means that the action with the specific instance can be performed successfully. If the user cannot perform the action successfully with a specific instance then the scenario is negative. For example, if password is null then the null instance is considered as a negative scenario as the action login (dimension of which is password) cannot be completed successfully. Expected results indicate the outcome when an action is taken using a specific instance (e.g. null password should result in incomplete login).
When the action triads are processed by a software tool then two types of outputs are generated- a set of conceptual models and tables and set of test cases. Conceptual models are generally graphical representations (e.g. UML Use Case and BPMN process models) of the domain that need to be reflected in the Information Systems [9]. Conceptual models are used for documenting the features of the domain that needs to be reflected in the Information Systems [9]. [10] mention that conceptual models could be highly relevant in agile development methods as agile development requires efficient communication between the stakeholders. The conceptual models and their descriptions can be used as requirements documentation in agile development. Using the precedence of the triads, a high level BPMN process model can be created and using the agents and the action performed by them, UML Use Case model can be developed. The dimensions of the functions can be used to create test case steps and the instances can be used in these test cases. As a large combination of instances can create large number of test cases therefore an optimization engine (part of the software tool) could be used to come up with minimum number of test cases that can cover maximum combination of instances. As the action triad method captures the requirements ID, therefore a requirements traceability matrix (RTM) can also be generated. RTM maps the requirements with the test cases. A set of guidelines is proposed so that application requirements can be captured in to set of action triads. These guidelines are: Decompose an application to be developed to sets of action triads. Indicate the sequence of the action triads for the application. Ensure that each action triad is unique for an application under testing. Ensure that each dimension has an instance with at least one positive scenario. In this method, functions must have dimensions but agents and objects may not. 3
The user needs to provide a valid username and password to successfully login to the application.
After the user logs into the application, then she is taken to the browse 1.2 application page where the user can click on different reports.
On clicking the logout button the user gets an alert to logout. If the user 1.3 clicks on yes then she is logged out otherwise on clicking cancel the user is taken back to the browse application page.
A user has a valid username (John) and a valid password (1234!). 1.4 The screen shots of the interface of the application are shown in Figure 1.
1.1 The outputs of the action-triad method are described next. In Figure 2, high level Use Case and BPMN diagram are generated from the method. As the agent –user is associated with two actions therefore the Use Case shows these two actions. Login application is followed by the logout application and they are shown accordingly in the BPMN. It is to be noted that these models are high level as detailed level information (e.g. join operators in BPMN) is not captured in the method.
User
Login Application Logout Application
User
Login Application
Logout Application Fig. 3: Action path model for Login function Select valid LoginUsername Select valid LoginPassword Based on the triad information, the tool uses an optimization engine to create test cases with specific steps. A test case generally contains specific steps that testers need to perform to test a certain function of an application. Along with the steps, a test case contains test objective, expected results, and traceability (i.e. which steps correspond to specific part of the requirements) [11]. The engine optimizes the dimensions and their instances to create test cases. The dimensions (column 1 of table 3) become test case step descriptions and the instances (column 2 of table 3) become the values of these descriptions. As all the instances will not fit into one test case therefore the optimization engine will create multiple test cases ensuring that a pair of instances is covered in at least one test case. To make the test case step description readable, specific user actions (e.g. enter or click) are used in the step descriptions. In the tool, each dimension can be classified as: text, dropdown, and button. These keywords are linked to action words that users perform to test the application. For example, if text is selected as a dimension type then the keyword “Enter” will be used in the test case step description. Thus a test case step description could be “Enter John as LoginUsername” where John is an instance of the dimension LoginUsername (type is text). Similarly when dimension type is button then the keyword “click” is used in the step description.
For the case study described here, the action triad method generated 5 test cases (1 positive and 4 negative) for the login function. Table 4 shows two test cases of login function that are generated using the action-triad method. Each test case has a description, step number, step description, expected results, and traceability. The step description has specific actions with specific values (e.g. Enter “John” as LoginUserName). However, each test case is different as different combinations of instances are used. If one of the instances has a negative scenario then the test case is considered negative meaning the actions mentioned in the test case should not be successfully executed. If no instances have negative scenarios then the test case is considered as positive meaning the actions mentioned in the test case should be executed successfully.
Each test case starts with the instances of the dimensions (e.g. UserPassword and UserUsername) of the entities (e.g. user). This step is considered as a pre-requisite i.e. the condition that is required before the test case can be run. A pre-requisite step does not have expected results and traceability.
Each test case can have only one negative instance scenario. This is because from a tester’s perspective, if a test case has two or more negative instances (e.g. Username is null and password is null) then it is not possible to identify the exact cause of failure of the test (e.g. whether the test failed because password was incorrect or it failed because the username was incorrect). The action triad method also creates a standardized form of test cases called- Gherkin syntax (Figure 4). Gherkin syntax is a business readable language that describes the behavior of the software. The syntax includes status parameter and the values of status are either successful or unsuccessful. If a test case has a negative value of an instance then the status is unsuccessful or else the status is successful. Gherkin syntax (Figure 4) is a rearrangement of the test cases written in tabular format (Table 4). In some agile projects, user stories are written in Gherkin syntax and used to generate automated test scripts using test automation tools.
Feature: User Login Application As a User I want to Login Application Given UserPassword is '<UserPassword>' And UserUsername is '<UserUsername>' When User Login Application And LoginUsername is '<LoginUsername>' And LoginPassword is '<LoginPassword>' And LoginTrigger is '<LoginTrigger>' Then status of Login should be '<Status>' Examples: Scenario Outline: User attempts to Login Application with various input parameters | UserPassword | UserUsername | LoginUsername | LoginPassword | LoginTrigger | Status | | 1234! | John | John | 1234! | Login | Successful | | 1234! | John | John | Password | Login | Unsuccessful | | 1234! | John | Blank | 1234! | Login | Unsuccessful | | 1234! | John | JohnInvalid | 1234! | Login | Unsuccessful | | 1234! | John | John | Blank | Login | Unsuccessful |
Fig. 4: Gherkin syntax of the test cases for the login function In agile practices, software development is performed by teams that include business analysts, testers, and developers [1]. Due to short implementation cycles (sprints) in agile, the analysts face difficulty in developing requirements documentation. Further as the application requirements change, the documentation needs to be updated frequently. The testers in the agile team face the challenge of developing the test cases prior to the codes written by the developers.
The proposed action-triad method helps to resolve these challenges by capturing the requirements in a systemic way. The method generates conceptual models that can be used for requirements documentation. The RTM and test cases in multiple formats are also generated in this method. When the application requirements change, the conceptual models, RTM, and test cases can be regenerated.
The requirements documentation, RTM, and test cases are currently written manually but the use of the action-triad method can help the agile teams to develop these outputs automatically. Thus the software applications can be developed in shorter time. Currently, the action-triad method is applied to few real-world agile practice based projects using a software tool. To test the effectiveness of the method, it has to be applied to complex application development projects and modified as necessary.
State diagrams as models have also been used for creating automated test cases. A state diagram depicts the states that a system can assume and shows the events that cause and/or result from a change from one state to another [11]. Test cases are derived from the state diagrams by identifying valid and invalid state transitions [11]. However, there are two main challenges in using such models. First, developing such models is difficult due to the complexity of state diagrams. As the application gets complex, the number of states and the transitions grow rapidly creating explosion of states (a phenomenon called – state explosion [12]). Second, creating optimized test cases from these models is challenging because of large number of states and the frequent change in the functionalities of the application. A complex application will have a large number of states and transitions. Thus modeling such application using state diagrams is challenging and so is deriving the test cases from the diagrams. Alternatively, we suggest using ER based Action Triad method as there is no need of modeling transitions and only the higher state changes are modeled using triads. Decomposing an application into set of triads is still a modeling skill that modelers need to apply. However, once the triads are identified, the methodology helps to create the model elements consistently.