INTRODUCTION

Software testing is known as a key critical phase in the software development life cycle, which account for a large part of the development effort. A way of reducing testing effort, while ensuring its effectiveness, is to generate test cases automatically from artifacts used in the early phases of software development. Many test case generation techniques have been proposed [2], [4], [10], [11], [12], [15], [21], [22], [42], [47], [50], mainly random, pathoriented, goal-oriented and model-based approaches. Random techniques determine a set of test cases based on assumptions concerning fault distribution. Pathoriented techniques generally use control flow graph to identify paths to be covered and generate the appropriate test cases for those paths. Goal-oriented techniques identify test cases covering a selected goal such as a statement or branch, irrespective of the path taken. There are many researchers and practitioners who have been working in generating a set of test cases based on the specifications. Modeling languages are used to get the specification and generate test cases. Since Unified Modeling Language (UML) is the most widely used language, many researchers are using UML diagrams such as state diagrams, use-case diagrams and sequence diagrams to generate test cases and this has led to model-based test case generation techniques. In this paper, an approach with additional requirement prioritization step is proposed toward test cases generation from requirements captured as use cases [23], [24], [33]. A use case is the specification of interconnected sequences of actions that a system can perform, interacting with actors of the system. Use cases have become one of the favorite approaches for requirements capture. Test cases derived from use cases can ensure compliance of an application with its functional requirements. However, one difficulty is that there are a large number of functional requirements and use cases. A second research challenge is to ensure that test cases are able to preserve and identify critical domain requirements [5]. Finally, a third problem is to minimize a number of test cases while preserving an ability to reveal faults. For example, there are a lot of functional requirements in the large software development. Software test engineers may not be able to design test cases to cover important requirements and generate a minimum set of test cases. Therefore, test cases derived from large requirements or use cases are not effective in the practical large system. This paper presents an approach with additional requirement prioritization process for automated generation of abstract presentation of test purposes called test scenarios. This paper also introduces a new test case generation process to support and resolve the above research challenges. We overcome the problem of large numbers of requirements and use cases. This allows software testing engineer to prioritize critical requirements and reasonably design test cases for them. Also, this allows us to be able to identify a high percentage of each test case's critical domain coverage.

The rest of the paper is organized as follow. Section 2 discusses the comprehensive set of test case generation techniques. Section 3 proposes the outstanding research challenges that motivated this study. Section 4 introduces a new test generation process and technique. Section 5 describes an experiment, measurement metrics and results. Section 6 provides the conclusion and research directions in the test case generation field. The last section represents all source references used in this paper.

LITERATURE REVIEW

Model-based techniques are popular and most researchers have proposed several techniques. One of the reasons why those model-based techniques are popular is that wrong interpretations of complex software from non-formal specification can result in incorrect implementations leading to testing them for conformance to its specification standard [43]. A major advantage of model-based V&V is that it can be easily automated, saving time and resources. Other advantages are shifting the testing activities to an earlier part of the software development process and generating test cases that are independent of any particular implementation of the design [7]. The model-based techniques are method to generate test cases from model diagrams like UML Use Case diagram [23], [24], [33], UML Sequence diagram [7] and UML State diagram [5], [43], [22], [2], [21], [15], [32], [4]. There are many researchers who investigated in generating test cases from those diagrams. The following paragraphs show examples of model-based test generation techniques that have been proposed for a long time.

Heumann [23] presented how using use cases to generate test cases can help launch the testing process early in the development lifecycle and also help with testing methodology. In a software development project, use cases define system software requirements. Use case development begins early on, so real use cases for key product functionality are available in early iterations. According to the Rational Unified Process (RUP), a use case is used to describe fully a sequence of actions performed by a system to provide an observable result of value to a person or another system using the product under development." Use cases tell the customer what to expect, the developer what to code, the technical writer what to document, and the tester what to test. He proposed three-step process to generate test cases from a fully detailed use case: (a) for each use case, generate a full set of use-case scenarios (b) for each scenario, identify at least one test case and the conditions that will make it execute and (c) for each test case, identify the data values with which to test. Ryser [24] raised the practical problems in software testing as follows: (1) Lack in planning/time and cost pressure, (2) Lacking test documentation, (3) Lacking tool support, (4) Formal language/specific testing languages required, (5) Lacking measures, measurements and data to quantify testing and evaluate test quality and (6) Insufficient test quality. They proposed their approach to resolve the above problems. Their approach is to derive test case from scenario / UML use case and state diagram. In their work, the generation of test cases is done in three processes: (a) preliminary test case definition and test preparation during scenario creation (b) test case generation from Statechart and from dependency charts and (c) test set refinement by application dependent strategies.

RESEARCH CHALLENGES

This section discusses the details of research issues related to test case generation techniques and research problems, which are motivated this study. Every test case generation technique has weak and strong points, as addressed in the literature survey. In general, referring to the literature review, the following lists major outstanding research challenges. The first research problem is that existing test case generation methods are lack of ability to identify domain specific requirements. The study [5] shows that domain specific requirements are some of the most critical requirements required to be captured for implementation and testing, such as constraints requirements and database specific requirements. Existing approaches ignore an ability to address domain specific requirements. Consequently, software testing engineers may ignore the critical functionality related to the critical domain specific requirements. Thus, this paper introduces an approach to priority those specific requirements and generates an effective test case. The second problem is that existing test case generation techniques aim to generate test cases which maximize cover for each scenario. Sometimes, they generate a huge number of test cases which are impossible to execute given limited time and resources. As a result, those unexecuted test cases are useless. The last problem is to unable to identify suitable test cases in case that there are limited resources (e.g. time, effort and cost). The study reveals that existing techniques aim to maximum and generate all possible test cases. This can lead to unable to select necessary test cases to be executed during software testing activities, in case that there are limited resources.

PROPOSED METHOD

This section presents a new high-level process to generate a set of test cases introduced by using the above comprehensive literature review and previous works [43].

Figure 1 A Proposed Process to Generate Test Cases

From the above figure, the left-hand side process is a general waterfall process. We propose to add two additional processes: (a) requirement prioritization and (b) test case generation.

The requirement prioritization process aims to be able to effectively handle with a large number of requirements. The objective of this process is to prioritize and organize requirements in an appropriate way in order to effectively design and prepare test cases [16], [25], [37]. There are two sub-processes: (a) classify requirements and (b) prioritize requirements.

The classify requirement process primarily divides and classifies requirements into four groups [30]: (a) "Must-Have" (b) "Should-Have" (c) "Could-Have" and (d) "Wish". The "Must-Have" requirements are mandatory requirements that need to be implemented in the system. The "Should-Have" requirements are requirements that should be implemented if there are available resources. The "Could-Have" requirements are additional requirements that are able to be implemented if there are adequate resources. The "Wish" requirements are "would like to have in the future" requirements that may be ignored if there are inadequate resources. This paper introduces five factors to classify the above requirements, as follows:

Table 1 Requirement Classification Group Time Cost People Scope Success Must have Y Y Y N Y Should have Y Y Y N N Could have N N Y Y N Wish N N N Y N

From the above table, the following shortly describes a meaning of the above factors:

• Time -The requirement must be implemented in the current version or release of software. • Cost -There is an available of budget or fund to implement the requirement. • People -There is an available of human resources to develop and test the requirement. • Scope -The requirement can be removed out of the current version or release of software. • Success -The success of system development rely on the requirement. In addition, this paper secondary divides those requirements into two groups: (a) functional and (b) non-functional. The functional requirements can be categorized into two groups: (a) domain specific requirements and (b) non-domain specific requirements. The domain specific requirements are able to identify as database specific and constraints requirements. For example, database connection specific requirements and requirements for an interface with other systems. The non-functional requirements can be vary, such as performance, security, operability and maintainability requirements. The following displays the classify requirement tree:

Figure 2 A Classify Requirement Tree

From the above figure, we propose a ranking number for each requirement. This paper prioritizes "Must-Have" requirements as top three ranking and "Wish" requirements as last three ranking. The study [5] reveals that domain specific requirements should have higher priority than both of behavioral and nonfunctional requirements.

However, when the requirement is already classified, the next process is to prioritize those requirements. In the requirement prioritization process, this paper proposes to use a cost-value approach to weight and prioritize requirements. This paper also proposes to use the following formula: P(Req) = (Cost * CP)

(1) Where:

• P is a prioritization value.

• Req is a requirement required to be prioritized.

• Cost is a total estimated cost of coding and testing for each requirement. • CP is an user-defined customer priority value. This value is in the range between 1 and 10. 10 is the highest priority and 1 is the lowest priority. This value aims to allow customers to identify how important of each requirement is from their perspective. To compute the above cost for coding and testing, this paper proposes to apply the following formula:

Cost= (ECode*CostCode)+(ETest*CostTest)

(2) Where:

• Cost is a total estimated cost.

• ECode is an estimated effort of coding for each requirement. The unit is man-hours. • CostCode is a cost of coding that is charged to customers. This paper applies the cost-value approach to identify the cost of coding for each requirement group (e.g. "Must-Have", "Should-Have", "Could-Have" and "Wish"). The unit is US dollar. • ETest is an estimated effort of testing for each requirement. The unit is man-hours. • CostTest is a cost of testing that is charged to customers. The approach to identify this value is similar to CostCode's approach. The unit is US dollar. In this paper, we assumed the following in order to calculate CostCode and CostTest. Also, this paper assumes that a standard cost for both activities is $100 per man-hours.

• A value is 1.5 of ("Must-Have", "Should-Have") -this means that "Must-Have" requirements have one and half times cost value than "Should-Have" requirements. • A value is 3 of ("Must-Have", "Could-Have")this means that "Must-Have" requirements have three times cost value than "Could-Have" requirements. • A value is 2 of ("Should-Have", "Could-Have") -this means that "Should-Have" requirements have two times cost value than "Could-Have" requirements. • A value is approximately 3 of ("Could-Have", "Wish") -this means that "Could-Have" requirements have three times cost value than "Wish" requirements. Therefore, the procedure of requirement prioritization process can be shortly described below: 1. Provide estimated efforts of coding and testing for each requirement. 2. Assign cost value for each requirement group based on the previous requirement classification (e.g. "Must-Have", "Should-Have", "Could-Have" and "Wish"). 3. Calculate a total estimated cost for coding and testing, by using the formula (2). 4. Define a customer priority for each requirement. 5. Compute a priority value for each requirement by using the formula (1). 6. Prioritize requirements based on the higher priority value.

Once the requirements are prioritized, the next proposed step is to generate test scenario and prepare test case.

This section presents an automated test scenario generation derived from UML Use Case diagram. Our approach is built based on Heumann's algorithm [23]. The limitation of our approach is to ensure that all use cases are fully dressed. The fully dressed use case is a use case with the comprehensive of information, as follows: use case name, use case number, purpose, summary, pre-condition, post-condition, actors, stakeholders, basic events, alternative events, business rules, notes, version, author and date.

The proposed method contains four steps, as follows: (a) extract use case diagram (b) generate test scenario (c) prepare test data and prepare other test elements. These steps can be shortly described as follows:

1. The first step is to extract the following information from fully dressed use cases: (a) use case number (b) purpose (c) summary (d) pre-condition (e) post-condition (f) basic event and (g) alternative events. This information is called use case scenario in this paper. The example fully dressed use cases of ATM withdraw functionality can be found as follows:

EVALUATION

The section describes the experiments design, measurement metrics and results.

Experiments Design

A comparative evaluation method has proposed in this experiment design. The high-level overview of this experiment design can be found as follows: 1. Prepare Experiment Data. Before evaluating the proposed methods and other methods, preparing experiment data is required. In this step, 50 requirements and 50 use case scenarios are randomly generated. 2. Generate Test Scenario and Test Case. A comparative evaluation method has been made among the proposed test generation algorithm, Heumann's technique Jim [23], Ryser's method [24], Nilawar's algorithm [33] and the proposed method presented in the previous section. 3. Evaluate Results. In this step, the comparative generation methods are executed by using 50 requirements and 50 use case scenarios. These methods are also executed for 10 times in order to find out the average percentage of critical domain requirement coverage, a size of test cases and total generation time. In total, there are 500 requirements and 500 use case scenarios executed in this experiment. The following tables present how to randomly generate data for requirements and use case scenarios respectively.