The article describes the results of development tool for XMLmessages and messages packs generation for automated functional testing. The tool realizes described in international standard ISO/IEC/IEEE 29119-4:2015 testing method based on equivalence partitioning for XML-messages processing systems. Built-in declarative language gives to test developer ability to describe equivalence partitions for XML-messages with different structure and different content. The language allows to describe data sources for elements of XML-messages. For each element both limits and dependences can be set. Technique of using the language is developed. The tool allows to generate XML-messages based on XSD-schemas using elements values matched equivalence partitions description from selected data sources. It gives ability to create numerous different test cases for current state of normative-reference information in system under test.
In process of supporting the life cycle of responsible systems their development and fixing also continues. The priority task of using the new packet of system modifications is to recheck already existing and testing of new functionality.
The key statement in testing is the need to reproduce all possible situations on which earlier the system fulfilled correctly, according to the declared documentation. Such checks require the large volume of varied data and, most often, these data are created by testers manually. However, formation of test examples by human is a very labor-consuming task.
In a definite time the test basis allowing to launch checks in an automatic mode collects. But there is a problem of updating of data.
In the course of application of patches to system its status and a status of internal reference databases, and also formats of input/output messages can change. Such changes can exert a direct impact and on test data – old test sets, earlier correct for system, can be not the valid or incorrect. There is a task of updating these test sets under a new status of system, and the more data, the higher the labor input on carrying out such operations.
Thus, we select two main objectives, systematically originating when carrying out regression testing: 1. Formation of test sets with a high level of a covering of functionality of system; 2. Updating of data under new formats and new values in reference bases of system. 2
The general diagram of testing of the information system processing XML messages consists of several sequential stages (a figure 1): 1. compilation of test requirements on the basis of functional requirements; 2. preparation of test sets (input messages and reference responses to them) to test requirements; 3. submission of messages in system and a check of responses of system with standards; 4. report generation about a correctness of functioning of system by results of a check.
The strategy of partition of values of input parameters on groups which processing in
system, according to the description and documentation, shall be made equally can be
applied to the decision of the task of a high level of a covering of functionality and
result in similar results. Following the results of partition it is enough to select only on
one parameter value from each group to check all group. The group received as a
result of partition is called an equivalence class, and process of separation of such
groups – partition on equivalence classes. It is told about this strategy of formation of
input data in the standard [
In the systems constructed for operation with electronic XML messages rules of
formation of the specific message and a format of its contents, according to the
standard [
Automation of process of data preparation will allow to achieve the following results: • to reduce the general time for conducting testing; • to reduce labor input of operation of testers; • to increase a level of a covering of functionality of system.
The special purpose tool of generation of XML messages is developed for achievement of these purposes. Generation of data is made on the strategy of partition of certain parameters of the incoming document on equivalence classes and search of combinations of parameter values.
It is important to consider that response messages of system depend on type of the input message, and also its contents therefore they need to be created jointly.
For the description of dependences between input and reference output XML messages and their rules of formation, the special declarative LDL language is developed. This language also allows to define data source and equivalence classes for these data for each parameter (attribute or the tag) of the XML message. As in case of complete search of combinations of equivalence classes the messages which aren't found in actual practice or not bearing semantic loading in language can be created mechanisms for restrictions and exceptions of such combinations are provided. 4
The test set represents an array of XML messages, input and output to them. And input messages can be excellent on types, and respectively and responses to them will be different. Thus, within one start of generation it is necessary to create input messages, different on type, and reference output corresponding to them. Instrumental means of generation allows to carry project development of test sets. Such project gives an opportunity to define uniform settings of the environment of generation, the general for different LDL files. Also, for each subsequent start of generation, provides a choice of LDL files, necessary for formation of a test set.
In total process of formation of a test set provides two stages: 1. preparation for generation of test data (figure 2) – provides interaction of the user with the tool for formation set-up and the file in the LDL language in which rules of generation of messages will be put; 2. generation of test data – demands from the user only start, further is executed in an automatic mode.
At this stage setup of connections with databases of the tested system is made, paths to xsd-schemas and directories in which it is necessary to place results of generation are specified. Such tuning can is performed for each project separately, and for the tool of generation in general. 4.2
Tasks of the tester include determination of composition of a test set for verification of the test requirement. After detection of necessary type input messages the tester shall specify types output which are expected following the results of processing of the input message the tested system. 4.3
As earlier it was marked, contents of the response message can change depending on contents input. At this stage the user defines communications between parameters input and response messages and sets procedures of transmission or conversion of values of such parameters. 4.4
Data sources for parameters of messages are defined by the following stage. The data sources are (figure 3): • the values entered by the user; • generators of accidental parameter values; • procedures of data transformation; • value from the database.
After determination of data sources for parameters the user shall select from them those parameters of the input message which influence verification of the test requirement and to describe equivalence classes for such parameters. The description of an equivalence class is set by logical expression, defining what values can accept parameter.
Also at this stage inadmissible combinations of equivalence classes of parameters are designated. Such combinations won't take a part in the course of generation.
The total theoretically possible quantity of different input messages created by the tool of generation will be calculated by a formula 1.
Where – the estimated number of different input messages, – the number of parameters of the input message for which equivalence classes are defined, – quantity of equivalence classes for parameter data source, – quantity of the combinations excluded from generation. Actually generated input messages it can appear less as in data source of parameter there can be no values satisfying to logical expression of an equivalence class. 4.6
After carrying out preparation, the tester needs only to launch generation with the necessary parameters. On an input the tool of generation needs to give settings of the environment and LDL files, with the description of rules of formation of a test set (figure 4).
Thus, for updating of values in test messages, it will be necessary to carry out only repeatedly generation, without modification of LDL files or settings of the environment. In case of change of structure of input or response messages, it is necessary to bring into accord with schemas only LDL files on the basis of which in a consequence test sets will be generated similar previous. 5
In some systems there is a need for additional actions with the XML messages, most often, connected to support of information security. The functionality of the tool of generation of test data provides some of types of such actions. Additional procedures carry out collection of messages in software packages to the certain signs set by the user as procedure parameters and also a reversing of messages in the official envelopes.
Procedures are launched irrespective of generation process that allows to apply them to the data created not only the tool of generation, but also the person, or other system. 6
1. Practical application of the generator can be very wide. The generator can be set up under any system architecture processing XML messages and having the description of types of these messages in a format of XSD-schemas. The description in the LDL file of rules of formation of certain XML messages in the LDL language allows to resolve a problem of updating of data under new values in databases of the tested system. In case of modification of XS-schemas, it is necessary to change only the LDL file and to repeat generation that considerably reduces labor input of maintenance of urgent structure of XML messages. The problem of preparation of large volume of test data is solved by determination of sets of equivalence classes for each of message parameters. 2. Mastering of the main functionality of the tool of generation happens within one week. Formation of the LDL file for one type of the input message, can borrow of several minutes till several o'clock, depending on complexity of the made example and skills of the user. 3. The average time of generation makes about 20000 input messages, different on composition and semantic loading, in an hour together with reference messages, output on them. Such tool allows to provide a high level of a covering of each checked test requirement. The detail level can be set in oscillator adjustments. Practical application of the generator allowed to expand a test coverage of system considerably. In table 1 the volume of stored regression functional tests on types of messages in 10 years of manual formation of tests is provided: number of tests, number of input test messages, number of response (reference) messages. Table 2 shows the amount of the regression functional tests created by means of the tool of generation on types of messages is provided: number of tests, number of input test messages, number of response (reference) messages. This amount was received following the results of operation of the generator within working month without the training period.