The tool for XML-messages and messages packs generation for automated functional testing 1 2 2 Boris Pozin , Ilya Galakhov and Aleksandr Korotkov 1 EC-leasing, 125 Varshavskoye shosse, Moscow, 117405, Russia, NRU Higher School of Economics bpozin@ec-leasing.ru 2 EC-leasing, 125 Varshavskoye shosse, Moscow, 117405, Russia {igalakhov,akorotkov}@ec-leasing.ru Abstract. The article describes the results of development tool for XML- messages and messages packs generation for automated functional testing. The tool realizes described in international standard ISO/IEC/IEEE 29119-4:2015 testing method based on equivalence partitioning for XML-messages pro- cessing systems. Built-in declarative language gives to test developer ability to describe equivalence partitions for XML-messages with different structure and different content. The language allows to describe data sources for elements of XML-messages. For each element both limits and dependences can be set. Technique of using the language is developed. The tool allows to generate XML-messages based on XSD-schemas using elements values matched equiva- lence partitions description from selected data sources. It gives ability to create numerous different test cases for current state of normative-reference infor- mation in system under test. Keywords: XML-messages generation, equivalence partitions, XML-messages processing systems, automated functional testing 1 Introduction In process of supporting the life cycle of responsible systems their development and fixing also continues. The priority task of using the new packet of system modifica- tions is to recheck already existing and testing of new functionality. The key statement in testing is the need to reproduce all possible situations on which earlier the system fulfilled correctly, according to the declared documentation. Such checks require the large volume of varied data and, most often, these data are created by testers manually. However, formation of test examples by human is a very labor-consuming task. In a definite time the test basis allowing to launch checks in an automatic mode collects. But there is a problem of updating of data. In the course of application of patches to system its status and a status of internal reference databases, and also formats of input/output messages can change. Such 231 changes can exert a direct impact and on test data – old test sets, earlier correct for system, can be not the valid or incorrect. There is a task of updating these test sets under a new status of system, and the more data, the higher the labor input on carry- ing out such operations. Thus, we select two main objectives, systematically originating when carrying out regression testing: 1. Formation of test sets with a high level of a covering of functionality of system; 2. Updating of data under new formats and new values in reference bases of system. 2 Testing method The general diagram of testing of the information system processing XML messages consists of several sequential stages (a figure 1): 1. compilation of test requirements on the basis of functional requirements; 2. preparation of test sets (input messages and reference responses to them) to test re- quirements; 3. submission of messages in system and a check of responses of system with stand- ards; 4. report generation about a correctness of functioning of system by results of a check. Fig. 1. General diagram of testing of an information system. The strategy of partition of values of input parameters on groups which processing in system, according to the description and documentation, shall be made equally can be applied to the decision of the task of a high level of a covering of functionality and result in similar results. Following the results of partition it is enough to select only on one parameter value from each group to check all group. The group received as a 232 result of partition is called an equivalence class, and process of separation of such groups – partition on equivalence classes. It is told about this strategy of formation of input data in the standard [1] in more detail. Creation of data on such principle allows to achieve a high level of a covering of a functionality of system, but at the same time and is very labor-consuming. In the systems constructed for operation with electronic XML messages rules of formation of the specific message and a format of its contents, according to the stand- ard [3] most often are defined. When updating test sets for such systems there is a need to repeatedly check compliance of each type of messages to schemas and to enter new parameter values in case of changes in databases of system that involves big dead times and resources in case of manual formation of test data. 3 Automation of process of preparation of test data Automation of process of data preparation will allow to achieve the following results: • to reduce the general time for conducting testing; • to reduce labor input of operation of testers; • to increase a level of a covering of functionality of system. The special purpose tool of generation of XML messages is developed for achieve- ment of these purposes. Generation of data is made on the strategy of partition of certain parameters of the incoming document on equivalence classes and search of combinations of parameter values. It is important to consider that response messages of system depend on type of the input message, and also its contents therefore they need to be created jointly. For the description of dependences between input and reference output XML mes- sages and their rules of formation, the special declarative LDL language is developed. This language also allows to define data source and equivalence classes for these data for each parameter (attribute or the tag) of the XML message. As in case of complete search of combinations of equivalence classes the messages which aren't found in actual practice or not bearing semantic loading in language can be created mecha- nisms for restrictions and exceptions of such combinations are provided. 4 Process of formation of a test set by means of the tool of generation The test set represents an array of XML messages, input and output to them. And input messages can be excellent on types, and respectively and responses to them will be different. Thus, within one start of generation it is necessary to create input mes- sages, different on type, and reference output corresponding to them. Instrumental means of generation allows to carry project development of test sets. Such project gives an opportunity to define uniform settings of the environment of generation, the 233 general for different LDL files. Also, for each subsequent start of generation, provides a choice of LDL files, necessary for formation of a test set. In total process of formation of a test set provides two stages: 1. preparation for generation of test data (figure 2) – provides interaction of the user with the tool for formation set-up and the file in the LDL language in which rules of generation of messages will be put; 2. generation of test data – demands from the user only start, further is executed in an automatic mode. Fig. 2. Process of preparation for generation of test data. 4.1 Setup of a runtime environment of generation At this stage setup of connections with databases of the tested system is made, paths to xsd-schemas and directories in which it is necessary to place results of generation are specified. Such tuning can is performed for each project separately, and for the tool of generation in general. 4.2 Determination of types input and output messages Tasks of the tester include determination of composition of a test set for verification of the test requirement. After detection of necessary type input messages the tester 234 shall specify types output which are expected following the results of processing of the input message the tested system. 4.3 Installation of correlations between input and output messages As earlier it was marked, contents of the response message can change depending on contents input. At this stage the user defines communications between parameters input and response messages and sets procedures of transmission or conversion of values of such parameters. 4.4 Determination of data sources for parameters of messages Data sources for parameters of messages are defined by the following stage. The data sources are (figure 3): • the values entered by the user; • generators of accidental parameter values; • procedures of data transformation; • value from the database. Fig. 3. Types of data sources of parameter. 4.5 Partition of equivalence classes for necessary parameters After determination of data sources for parameters the user shall select from them those parameters of the input message which influence verification of the test re- quirement and to describe equivalence classes for such parameters. The description of an equivalence class is set by logical expression, defining what values can accept parameter. Also at this stage inadmissible combinations of equivalence classes of parameters are designated. Such combinations won't take a part in the course of generation. The total theoretically possible quantity of different input messages created by the tool of generation will be calculated by a formula 1. 235 ∏ (1) Where – the estimated number of different input messages, – the number of parameters of the input message for which equivalence classes are defined, – quantity of equivalence classes for parameter data source, – quantity of the com- binations excluded from generation. Actually generated input messages it can appear less as in data source of parameter there can be no values satisfying to logical expression of an equivalence class. 4.6 Generation of test data After carrying out preparation, the tester needs only to launch generation with the necessary parameters. On an input the tool of generation needs to give settings of the environment and LDL files, with the description of rules of formation of a test set (figure 4). Fig. 4. Diagram of operation of the tool of generation. Thus, for updating of values in test messages, it will be necessary to carry out only repeatedly generation, without modification of LDL files or settings of the environ- ment. In case of change of structure of input or response messages, it is necessary to bring into accord with schemas only LDL files on the basis of which in a consequence test sets will be generated similar previous. 5 Formation of packets of XML messages In some systems there is a need for additional actions with the XML messages, most often, connected to support of information security. The functionality of the tool of generation of test data provides some of types of such actions. Additional procedures carry out collec- 236 tion of messages in software packages to the certain signs set by the user as procedure parameters and also a reversing of messages in the official envelopes. Procedures are launched irrespective of generation process that allows to apply them to the data created not only the tool of generation, but also the person, or other system. 6 Conclusion 1. Practical application of the generator can be very wide. The generator can be set up under any system architecture processing XML messages and having the descrip- tion of types of these messages in a format of XSD-schemas. The description in the LDL file of rules of formation of certain XML messages in the LDL language al- lows to resolve a problem of updating of data under new values in databases of the tested system. In case of modification of XS-schemas, it is necessary to change on- ly the LDL file and to repeat generation that considerably reduces labor input of maintenance of urgent structure of XML messages. The problem of preparation of large volume of test data is solved by determination of sets of equivalence classes for each of message parameters. 2. Mastering of the main functionality of the tool of generation happens within one week. Formation of the LDL file for one type of the input message, can borrow of several minutes till several o'clock, depending on complexity of the made example and skills of the user. 3. The average time of generation makes about 20000 input messages, different on composition and semantic loading, in an hour together with reference messages, output on them. Such tool allows to provide a high level of a covering of each checked test requirement. The detail level can be set in oscillator adjustments. Practical application of the generator allowed to expand a test coverage of system considerably. In table 1 the volume of stored regression functional tests on types of messages in 10 years of manual formation of tests is provided: number of tests, number of input test messages, number of response (reference) messages. Table 1. Amount of stored regression functional tests. Amount Amount of output Message type Amount of tests of input messages messages Type 1 131 125 312 Type 2 194 657 2190 Type 3 117 398 3551 Type 4 472 1741 5946 Type 5 461 774 3864 Type 6 456 2033 4992 Total 1831 5728 23667 237 Table 2 shows the amount of the regression functional tests created by means of the tool of generation on types of messages is provided: number of tests, number of input test messages, number of response (reference) messages. Table 2. The amount of the regression functional tests created by means of the instrument of generation. Amount of input Amount of output Message type Amount of tests messages messages Type 1 1443 1351 3378 Type 2 1271 4328 14431 Type 3 663 2122 18903 Type 4 2778 10354 35198 Type 5 3184 5175 34279 Type 6 2847 12179 29877 Total 12186 35509 136066 This amount was received following the results of operation of the generator within working month without the training period. References 1. Software and systems engineering - Software testing - Part 4: Test technique. ISO/IEC/IEEE 29119-4:2015 2. Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979 3. W3C XML Schema Definition Language: http://www.w3.org/TR/tr-technology- stds#tr_XML_Schema