Testing is common practice in the realm of software engineering. Especially in agile approaches, where test-driven development can be seen as integral. Capa agents are developed under the agile Paose approach. Their internal components are implemented using Java reference nets which combine the semantics of P/T nets and Java. The existing testing methods for these kinds of nets are either difficult to learn or ill-suited for testdriven development and regression testing. In this work a tool chain is presented which allows testing of reference nets using regular Java classes. For this purpose an extension of the well-established JUnit framework is provided. All tools are designed to be easily understood by developers of Capa agents. This is achieved by a mixture of automatic code generation, repurposing other tools of the Paose approach, and employing a style of testing that is reminiscent of regular Java tests. While most of the code generating tools are limited to the context of Capa agents, regression testing is possible for any kind of reference net. The findings may help in the development of testing frameworks for other high-level Petri net formalisms.
Capa (Concurrent Agent Platform Architecture) allows for the construction of
software agents based on reference nets [
The tools presented in this work allow for the test-driven development of regression tests for Capa agent components. These tests are written using the JUnit framework which is already well-supported by many continuous integration softwares. Furthermore the techniques presented are at least partly applicable to general reference nets. As such it is of interest to ask if the ability to develop high-level Petri nets under a test-driven approach is useful in a more general case outside of Capa agents. This will be discussed in the first part of this work, before introducing the tool chain in the second. 2
We briefly introduce (Java) reference nets, the general structure of a Capa agent, some important aspects of Paose and the widely used JUnit framework. 2.1
Java reference nets, from here on simply referred to as reference nets, were first
introduced by Kummer [
Stub Classes Reference nets are themselves Java objects. To allow easy access to their interfaces one can use stub classes which are generated from stub files with Java-like syntax. They basically function as adapters and make net instances available for Java classes. Usage of stub classes actually allows to exchange every Java object by a reference net instance and vice versa.
The example shown in Figure 2 shows the stub syntax and resulting Java code for one of the standard interfaces used in Paose. The channel name “newExchange” is a standard name that is used multiple times. The specific channel instance is identified with the provided String id. Since this id will never change during runtime, it is declared in the method body. This simplifies the job of testers, as they do not have to know the channel instance of the net. Instead this task is shifted to the person writing the stub file. This is important because the correct implementation of the stub class depends on the use of the interface. Both, uplinks and downlinks, can be used to send and receive information, sometimes both at once. Since Java only allows for a single return value, channels might require multiple corresponding Java methods.
1 public void i n p u t 2 ( f i n a l Object ppo , f i n a l int ppid ){ 3 . . . 4 SimulationThreadPool . g e t C u r r e n t ( ) . 5 e x e c u t e (new Runnable ( ) { 6 public void run ( ) { 7 de . renew . u n i f y . Tuple inTuple ; 23451 b}(tShOrteribasijnk:egncvetwso=oiEd",xitcneihnstatpnCuigdhet a()sn{,noe,l "id; ) ; 1111012398 } do_S.ue.yi.tn.nTrscetuhnaprenolwcen=e.i dus,"aennt.iierfoeywnnE.ReTxwecuq.hpuaclenaegsl toel"u..ts, TyinnuTcphulepro;len)i z; e ( 2.2
The agent itself is accessible through the “receive” and “send” transitions which are inscribed with synchronous channels of the same name. Messages given as parameters via synchronization are Java objects of a specific type. Protocols are tied to an act of communication and exist only as long as that communication lasts. They are implemented as instances of reference nets and are held in the place labeled “conversations”.
Decision components as well are implemented as instances of reference nets. They are however instantiated when the agent is started and their lifetime is usually the full runtime of the agent itself. They are used to model proactive behavior of the agent, but also to implement services used by multiple protocols. The exchange transition (located between conversations and decision components) is used to synchronize uplinks of the “dcexchange” channel in protocols with uplinks of the “exchange” channel in decision components. The same channel can be used to allow communication between two decision components (not modeled in the figure). Individual instances of these uplinks are identified via additions to the channel name provided as String parameters. Examples of this can be seen in Figure 4.
Agent-, protocol- and decision component-nets, as well as their respective interfaces, are the main concern when black-box testing Capa agents. 2.3
Full comprehension of the Paose approach is not required in order to understand this work. Therefore this section will only address two tools which are part of the Paose development process and which are used during testing later on. Net Components are subnets which serve as templates for recurring functionality within the nets. For example there exist net components for the aforementioned “exchange” channel, as seen in Figure 4. Net Components consist of regular net elements and are only visually distinguishable, i.e. they are easily recognized by humans, but no meta information about them is kept in the net template. Agent Interaction Protocols (AIP) are extended UML sequence diagrams. They are used to model interactions between different agents1. An example can be seen later in Figure 7. It is possible to automatically generate net skeletons of protocol nets from the models. This is done by mapping the inscriptions on the diagram elements to Net Components which are then drawn and connected in the same order. 2.4
JUnit is a testing framework for Java applications [
Tests can be started, manually or automatically, from the IDE or from the command line using build tools. To execute the tests a controller class called test runner is used. Often test classes specify which runner is supposed to execute them.
The runner creates a TestResult object which is usually used to generate a test report. The design of the report depends on the implementation but often includes the number of failed and succeeding tests, the expended time and the stack trace in case of a failure.
JUnit4 heavily relies on reflectively manipulating tests by use of annotations. The @RunWith annotation is used to specify the runner class. @Test marks the tests themselves. Optionally a time limit may be set, which is useful if deadlocks 1 More specifically the interactions between agent roles. are a concern. @Before and @After are called before and, respectively, after each test. They are used for setting up the testing environment and returning it to its prior state after testing.
The core practices of Agile Modeling are introduced and it is shown that Petri nets can be used in conformance with them. It is also argued why Petri nets are especially useful as a modeling language in agile development. 3.1
Agile Modeling, as it is presented by Ambler [
Iterative and Incremental Modeling The first practice is apply the right artifacts. Petri nets cannot be considered a universal modeling language but are useful in modeling concurrent behavior. For these kinds of tasks, they are indeed the right artifacts. Similarly the practice iterate to another artifact is easy to fulfill if we assume that Petri nets are not our only means of modeling. If one is stuck during the modeling of a net, switching to a different modeling task may bring clarification.
The practices create several models in parallel and model in small increments
require a specific style of nets. More precisely they require nets that are limited
in their scope. This can be achieved by hierarchical net structures, as is done
in Renew with the nets-within-net approach or CPNTools with hierarchical
Coloured Petri Nets [
Teamwork The practices of this category are model with others, active
stakeholder participation, collective ownership and display models publicly. All of this
is part of the Paose approach and has been successfully done within the context
of a Petri nets-based software development environment. [
It has been have established that Petri nets can be used as a modeling language in agile development. To take this a step further it is shown that Petri nets are especially useful in agile approaches due to the nets being executable.
Research suggests that there are significant advantages keeping models and
code consistent [
The design / testing paradigm favored in eXtreme-Programming and other
agile approaches is test-driven development [
The idea of test-driven modeling has been proposed before.
Hawari et al. [
Zhang and Patel have successfully used similar techniques with executable
UML in industry software projects [
Walkinshaw and Derrick [
In the realm of Petri nets, testing is more associated with the techniques of verification or model checking. The presented approach is not competitive, but complementary to these. If the state-space becomes too large or the problem simply becomes undecidable due to added semantics, testing might be a good alternative. 5
Decisions that have been made regarding some aspects of the tools are clarified. Requirements and motivations that guided the development are explained. 5.1
The testing methods presented are specifically designed to satisfy the needs of
agile development. For this purpose three main points that had to be
incorporated were identified:
Test-driven Development Adapted to Petri nets, this means tests can be
written even before the net structure is known. In the field of testing this is known
as black-box testing [
Small-scale Unit- and System-wide Integration Tests Kent Beck
recommended when talking about eXtreme Programming :
“If the gap [between writing code and tests] is minutes, the cost of communicating
expectations between two people would be prohibitive.” [
Usability and Heterogeneous Skill Sets The importance of this last point
is difficult to quantify for a more general case. In academic software-projects
however, a significant discrepancy in the abilities of programmers has been
observed. As a joint Bachelor’s and Master’s project, both seasoned programmers
with several years of working experience, as well as beginners who have never
used a UNIX operating system before are working together [
To write tests the testing-framework JUnit is used. Its use was already suggested
in earlier works, however favoring a hybrid solution that relied on JUnit only
for starting and controlling tests. In the first approach the tests themselves were
written in the language of the test-objects, i.e. with reference nets [
Familiarity As mentioned earlier, some of the participants of academic Capa software projects have never seen Petri nets before, let alone used them for programming. However in order to even create something that is in need of testing, a certain degree of Java knowledge can be assumed. By relying solely on Java, tests are not much different from what a Java programmer would usually write, therefore shortening the time needed to learn the testing process. For developers that are completely new to programming with nets, we can also assume that the tests themselves are much less prone to failures and errors compared to the new language of Petri nets. Especially in the context of test-driven development, in which tests are written before the implementation, it is likely that the first attempts using nets are faulty, effectively defeating the purpose of writing these tests.
Support Features Renew provides some support features to developers, but not nearly as many as comparable IDEs for wide-spread high-level programming languages. The main draw of programming with reference nets, the concurrency, is completely irrelevant for the tests themselves. JUnit Functionality JUnit provides additional functionality, for example methods that are called before each individual test or expecting a test to fail due to an exception.
functionality is completely separated from its test. The net instance used can be exchanged for a different kind or even a Java class. Apart from technical advantages, this also better conforms to the goals of testdriven development. Theoretically, tests created this way could be written without any knowledge of Petri nets, preventing any chance of the tests being influenced by the implementation. 6
The MulanNetTest Plugin This Renew plugin was developed as part of a Bachelor’s thesis by the author. It was later expanded upon by adding support for the JUnit framework. All tools will be explained using the “WebChat” example. A use case can be seen in Figure 6. The first task when writing tests before the implementation is done, is to provide interfaces against which can be tested. For agents and protocols this is easy. Agents only provide their standard interfaces. Protocols can be generated from the AIP. Decision components however are entirely created by the developers.
To solve this, the AIP were extended to include both types of internal components. An example for the WebChat application can be seen in Figure 7. Depending on the outgoing and incoming arrows, as well as their inscriptions, net components are generated and connected in the order according to the diagram. From the model in Figure 7 four nets are generated: Two protocol nets and two decision component nets. The net ’Sender_DC’ can be seen in Figure 8. The commentary fields employ the new comment tool. It allows developers to append blue text to net elements which is also added to the elements’ meta information and can later be retrieved. In order to test the newly generated nets with JUnit, an adapter stub class has to be created. This is done automatically by mapping the standardized Net Component interfaces to code in net stub syntax. The Components are identified by naming the place containing the channel name after the interface type. For all places that have non-standardized names getter/setter methods are created. The result can be seen in Figure 9. The commentary is read from the place’s meta information and is also written into the final Java class. The stub generation is done within the IDE via a context menu, as seen in Figure 10. 6.3
To write the tests themselves the JUnit framework is employed and an adapter for this was created. First all newly added elements are explained. An in depth example is given in the next section.
Wincierz: Test-driven Development of Reference Nets 209 A e
o om n
n u n ig ah F c 1 2 3 4 5 6 7 8 9 / Receive t h e answer from t h e p r o t o c o l / void r e c e i v e A n s w e r ( Object o , int i d ){ S t r i n g s=" r e c e i v e A n s w e r " ; t h i s : exchange ( s , o , i d ) ; } Annotations The adapter mirrors JUnit4 ’s use of reflection. New annotations are @Repeat(int), which allows easy repetition of a test class, and @Concurrent Parameters(Object[][]), which is modeled after the @Parameters class used with the Parameterized test runner. It is used to define an array of arrays. For each entry of the super-array, a new test instance will be created and run concurrently. The sub-array values are reflectively injected into fields annotated with @ConcurrentParameter(int) according to the given adicity. This is also modeled after the regular JUnit functionality of the Parameterized runner. Thus users who have used it before, will hopefully understand the principle immediately.
RenewTestRunner The adapter is designed to be very close to regular JUnit in its use, as to allow easy adoption of its functionality. The core element is a custom test runner, which automatically starts a new Renew instance. The runner is designed to support the new annotations.
RenewTestClass All tests have to extend this class. This is more akin to the older JUnit version 3, where all tests had to extend a TestClass. It is necessary because of Renew ’s plugin based architecture. New instances of Renew are run in a new class loader to ensure the correct order of loading the plugins. The test runner cannot read the objects annotated with @ConcurrentParameters, therefore this task as well as the field injection is done by the test class itself upon being called reflectively. In addition the RenewTestClass provides convenience methods to synchronize testing phases. This is elaborated on in the example. 6.4
The code example shows a simple test class. The artifact to be tested is the “Sender_DC” net. During setup a new instance of the net is created (line 17/18), which is then wrapped in a stub adapter at the beginning of the test (line 24). The net instance is static because it will be used by the three instances of the test class with the different specified parameters.
The tests are synchronized using the finishPhase() method (lines 23,34,38). The test will wait until all instances of the test class have finished the current phase. The phases can be distinguished as executing and evaluation phases. Between each phase the simulation engine is halted / restarted. This ensures that during execution all test instances are active in the same part of the net, thus possibly provoking concurrency failures if existent.
Not all stub methods have been shown, but the methods can be easily matched to their respective channels in the net graphic by their names. In this case only the first half of the net is tested. A message is received from the web interface and given to the protocol net. The test concludes successfully, if both messages are the same. Since no functionality has been implemented, the test will fail after 3000 milliseconds. @Repeat ( t i m e s = 3) @RunWith( v a l u e = RenewTestRunner . c l a s s ) public c l a s s WebChatTest extends RenewTestClass { @ConcurrentParameters public Object [ ] [ ] params = {{ " house " , 1} , {" c a r " , 2} , {" p e t r i " , 3 } } ; @ConcurrentParameter ( v a l u e = 0) public S t r i n g message ; s t a t i c N e t I n s t a n c e i n s t a n c e ; @Before public void s e t u p ( ) {
Net net = Net . forName ( "Sender_DC" ) ; i n s t a n c e = net . b u i l d I n s t a n c e ( ) ; } @Test ( timeout = 3000) public void t e s t M e s s a g i n g ( ) { t h i s . f i n i s h P h a s e ( ) ; Sender_DCStub stub = new Sender_DCStub ( i n s t a n c e ) ; WebEventAction wea = new WebEventAction ( ) ; WebEvent we = new WebEvent ( ) ; VTSequence v t s = new VTSequence ( ) ; we . setData ( t h i s . message ) ; v t s . add ( we ) ; wea . s e t E v e n t s ( v t s ) ; wea . setName ( "" ) ; 34 35 36 37 38 39 40 41 42 43 44 45 } } stub .AGENTLET_DC_ACTION_HANDLE_WEB_EVENTSIn( wea , t h i s . i d ) ; t h i s . f i n i s h P h a s e ( ) ; stub . askForMessageIn ( Boolean .TRUE, t h i s . i d ) ; S t r i n g r e s u l t = stub . askForMessageOut ( t h i s . i d ) ; t h i s . f i n i s h P h a s e ( ) ;
A s s e r t . a s s e r t E q u a l s ( t h i s . message , r e s u l t ) ; 7
The code generation tools are useful in the context of Capa agents, but difficult to extend to more general cases. The unification mechanism of synchronous channels makes a mapping to Java methods problematic, as the number of required methods increases exponentially with the number of parameters, since any combination of receiving and giving Objects has to be taken into account. Other high-level Petri net formalisms with more specified interfaces might be more suitable.
The JUnit extension however allows the testing of all reference nets, provided a net stub is manually created. Writing the tests is about as difficult as testing regular Java classes and should not require special knowledge about Petri nets.
The testing methods have proven to be able to find semantical failures in Capa applications. Other properties, like liveness, are not possible to determine using testing. For this a combined approach with verification techniques might be a solution.
Also the JUnit extension in its current form is costly to use. To provide a clean environment, a new instance of Renew is started before each test. This takes about three to five seconds depending on the hardware on which the tests are run.
Despite some flaws the tools presented allow for easier testing and thereby higher quality of code. Automatic test execution will likely improve the integration process during development, although this has to be further observed in practice. The increased degree of testability improves the usefulness of reference nets not only as a programming, but also as a modeling language. 8
High-level Petri nets can combine graphical feedback with early simulation and might therefore be suitable as a modeling language in agile development. Especially of interest is the notion of test-driven modeling, which has been done before with other modeling languages, but has not been tried with Petri nets. A tool chain was presented with which test-driven development of Capa agents becomes possible. The agent’s internal components are reference nets which function as both implementation as well as their own model. The testing process was shown using a simple example.