Traceability Links Recovery has been a topic of interest for many years. However, Traceability Links Recovery in models in general, and BPMN models in particular, has not received enough attention yet. Through my work, I aim to ll this research gap by studying Traceability Links Recovery between requirements and BPMN models. So far, under the tutelage of directors Carlos Cetina and Oscar Pastor, I adapted Traceability Links Recovery code techniques to work over BPMN models. The produced approach was applied to two di erent case studies, an academic one and an industrial one. The outcomes of the research outperformed the state of the art baseline. Under the light of these novel ndings, opportunities for new research unfold.
Traceability Links Recovery (TLR) is de ned as the software engineering task
that deals with the identi cation and comprehension of dependencies and
relationships between software artifacts. It has been a subject of investigation
for many years within the software engineering community [
The three approaches were evaluated through the Camunda BPMN for Research case study (github.com/camunda/bpmn-for-research) and through a real-world industrial case study, provided by our industrial partner, CAF (Construcciones y Auxiliar de Ferrocarriles, www.caf.es/en), a worldwide provider of railway solutions. One of the adapted code techniques achieved the best results for all the measured performance indicators in both case studies, outperforming the other two techniques. The overall ndings of our paper suggested that adapting code techniques that provided good results in code was bene cial for TLR between requirements and BPMN models, since the outcomes outperformed those of a models-speci c baseline. Under the light of these ndings, a research question arises, unfolding opportunities for novel research: How can we further improve TLR in BPMN models?
The rest of the paper is structured as follows: Section 2 describes the Approach that obtained the best results and how to apply it to TLR between requirements and BPMN models. Section 3 details the baseline technique and the designed Evaluation. Section 4 presents the obtained Results. Section 5 formulates the Research Question that arises from our ongoing work. Section 6 discusses potential Future Work. Section 7 mentions the research Methodology in use. Finally, Section 8 reviews the works related to this one. 2
This section describes the Mutation Search technique, the technique designed
in [
The Mutation Search technique receives a query requirement and a BPMN model as input, generates a population of fragments, and ranks said fragments through Latent Semantic Indexing. From the ranking, the rst fragment is taken as the proposed solution. In order to generate the fragments population, algorithm 1 is followed. In the algorithm, an empty population and a seed fragment (chosen randomly from the input model) are created. Then, until the algorithm meets a stop condition (for instance, a certain number of iterations), the fragment is mutated and each new mutation is added to the population, avoiding the addition of repeated fragments.
In the algorithm, a mutation in a fragment can be caused by: (1) adding one new event, gateway, or task that is connected to an already present event, gateway, or task, (2) removing an element with only one connection, or (3) adding or removing a lane from the fragment. The performed mutation is chosen randomly on each iteration.
The top part of Fig. 1 shows this process, having the example input BPMN model on the left, and some example fragments on the right, generated through the usage of the algorithm. The generated fragments are represented through the text contained in all their elements. The text of both the input requirement and the generated fragments is then processed through general phrase
. Initialize the population . Create an initial seed fragment . While the stop condition is not met
. Mutate the fragment
. If the new fragment is not in the population
. Add the new mutation to the population
. Return the population
Algorithm 1 Mutation Search Algorithm
1: P []
2: F randomF ragment(inputM odel)
3: while !(StopCondition) do
4: F mutateF ragment(F )
5: if !(F 2 P ) then
6: P P + F
7: end if
8: end while
9: return P
styling techniques (lowercasing and tokenization), Parts-Of-Speech Tagging [
Finally, the requirement and the fragments are fed into Latent Semantic
Indexing, which ranks the fragments according to their similitude to the
requirement. Latent Semantic Indexing (LSI) [
To that extent, LSI produces a term-by-document co-ocurrence matrix. The bottom left part of Fig. 1 shows an example term-by-document co-occurrence matrix, with values associated to an example. Each row in the matrix (term) stands for each of the words that appear in the processed text of the requirement and the model elements. Each column in the matrix (document ) stands for each of the fragments (MF1 to MFn) generated through the algorithm. The nal column (query ), stands for the processed input requirement. Each cell in the matrix contains the frequency of each term in each document.
Vector representations of the documents and the query are obtained by
normalizing and decomposing the term-by-document co-occurrence matrix using a
matrix factorization technique called Singular Value Decomposition (SVD) [
The system will open the doors MODEL n a m iiino uH t b h In rooD leudoM Push doors button
Are the doors open?
Yes X
No Open the doors s Inhibition rdo Door yew Button K Open …
MF1 MF2 … 1 0 … 0 1 … 0 1 … 0 0 … … … … MF9 … 0 … 1 … 0 … 1 … … …
MFn Query 0 0 2 1 1 0 1 1 … … MF1 n o iiit b h n I
MF2
Push doors button
Are the doors open?
X Yes MF6
No
Open the doors
MF9 rooD leudoM Open the doors MMFF92 MFnMFN
MFM1F6
MFn Are the doors open? Push doors button X
Scores Model Element Ranking
MF9 = 0.97 MFn = 0.52
… MF6 = - 0.93 Documents
Query
Singular Value Decomposition
Spanoudakis et al. [
Requirements
BPM Model Approaches
Input
Linguistic Model Fragment
Mutation Search Precision, Recall, F-Measure, MCC
Measurements & Report
Approved Traceability
Oracle
A confusion matrix is a table that is often used to describe the performance of a classi cation model (in this case, the linguistic baseline and Mutation Search) on a set of test data (the solutions) for which the true values are known (from the oracle). The confusion matrix distinguishes between the predicted values and the real values, classifying them into four categories: (1) true positive; (2) false positive; (3) true negative; and (4) false negative. Then, some performance measurements are derived from the values in the confusion matrix. In particular, a report including four performance measurements (recall, precision, f-measure, and MCC) is created for each of the two case studies, both for the baseline and for Mutation Search. 3.3
In order to perform the evaluation of the approaches, we relied on two di erent case studies: (1) the Camunda BPMN for Research academic repository, and (2) a set of BPMN models provided by CAF, our industrial partner. In order to obtain the performance results of the approaches, we relied on the available correct solutions, provided in both case studies.
Camunda BPMN for Research: The Camunda BPMN for Research case study consists of four BPMN modeling exercises. Each exercise contains an associated textual description and the solution model for the provided description. In order to apply the approaches to the Camunda case study, a software engineer derived a set of requirements from the problem descriptions. Each exercise has an associated solution model for the provided description. The same software engineer who derived the requirements from the problem descriptions also generated a set of fragments from the solution model, mapping each fragment to a single requirement. Thus, we were provided with a set of requirements, the fragments that implement them, and the TLR mapping between both artifacts.
CAF: For our evaluation, CAF provided us with the requirements and BPMN models of ve railway solutions. They also provided us with their existing documentation on requirements to BPMN models traceability, where each requirement is also mapped to a single fragment. 4
From the results of our work, a Research Question arises: How can we further improve TLR in BPMN models? The following section will address this question, brie y mentioning some of the possible future works derived from a close inspection of the results. This section presents some ideas and opportunities for future work that arose from the presented Research Question: 1. (Accepted - CAiSE 2019) Tacit knowledge in the requirements may have a negative impact on semantic-based techniques. How can we minimize this impact? 2. (Currently under review - IS CAiSE 2018 special issue) BPMN models have some particularities that other models lack. Could we take in account these particularities in our techniques in order to lead them to enhanced results? 3. (Ongoing work) BMP models have less text that other models. Could we enrich the text of BPMN models to improve TLR techniques based on text search? 7 8
To perform this research, as well as our ongoing work, we have followed the
design science methodology guidelines presented in [
Related works focus on the impact and application of linguistic techniques to
TLR problem resolution at several levels of abstraction. Works like [
Other works target the application of LSI to TLR tasks. De Lucia et al. [