=Paper=
{{Paper
|id=Vol-2207/IWSM_Mensura_2018_paper_2
|storemode=property
|title=Analyzing the Performance of Two COSMIC Sizing Approximation Techniques Using FUR at the Use Case Level
|pdfUrl=https://ceur-ws.org/Vol-2207/IWSM_Mensura_2018_paper_2.pdf
|volume=Vol-2207
|authors=Francisco Valdès Souto
|dblpUrl=https://dblp.org/rec/conf/iwsm/Souto18
}}
==Analyzing the Performance of Two COSMIC Sizing Approximation Techniques Using FUR at the Use Case Level==
https://ceur-ws.org/Vol-2207/IWSM_Mensura_2018_paper_2.pdf
Analyzing the Performance of Two COSMIC Sizing Ap-
proximation Techniques Using FUR at the Use Case
Level
Francisco Valdés-Souto
National Autonomous University of Mexico
Science Faculty
CDMX, Mexico City, Mexico
fvaldes@ciencias.unam.mx
+52 55 56223899 ext 45735
ORCID: 0000-0001-6736-0666
Abstract. For accurate results, standards for the measurement of the functional
size of software require that the functionality to be measured be fully known.
However, when estimating in the early phases of software development where
there is a lack of detail, approximate sizing techniques must be used. An approx-
imation mechanism that has proven useful when there is no historical data is the
technique of approximation by EPCU, there are two EPCU contexts with the
range of the output variable other than 16.4 CFP and 44 CFP.
Previous studies have shown that when functional requirements are at a granu-
larity level of Functional Process, the context recommending being applied is that
the output variable has a cut-off at 16.4 CFP, this is done when comparing the
distribution of approximation results against the distribution of the REAL sizes.
This paper investigates the two EPCU contexts defined in the literature, seeking
to identify which technique appears to better represent the distribution of the
REAL sizes when the granularity level was Use Cases (UC), the ‘Equal Size
Bands’ (ESB) approximation and fuzzy logic-based approximation technique
(EPCU) were also compared to identify which technique appears to represent the
distribution of the REAL sizes better, when the granularity level was Use Cases
(UC).
From the results, it is not clear which approximation technique has the best per-
formance, however carrying out the non-parametric test, it is possible to confirm
statistically that the distribution of the EPCU44 approximation technique dis-
plays behavior similar to that of the distribution of the COSMIC REAL sizes.
Keywords. COSMIC ISO 19761; Approximate Sizing; Functional Size; EPCU
Model.
70
F. Valdés-Souto
1 Introduction
Functional Size Measurement (FSM) methods work best when the information to be
measured – the functional user requirements – is fully known. Santillo [1], for instance,
indicates that the “functional size of software to be developed can be measured pre-
cisely [only] after the functional specification stage: this stage is often completed rela-
tively late in the development process.” However, when estimating in the early phases
of software development projects, there is often a lack of detailed information, which
hinders the rigorous application of the measurement rules prescribed in international
standards [1, 2, 3].
As observed by Desharnais et al. [4], when software documentation is lacking, it is
not possible to apply all of the detailed measurement rules as specified in the interna-
tional standards for the measurement of the functional size of software. Thus, in such
early phases of the development cycle, to tackle this lack of detail and determine a
relevant range of candidate functional size, measurers must fall back on approximation
techniques for sizing requirements.
As Vogelezang points out [5], “a rapid size measurement will be acceptable if it can
be produced faster and still can deliver a reliable approximation of the detailed size
measurement.”
Most currently available approximation techniques for sizing the functional size of
software requiring calibration employ historical data for better results in local contexts,
such as the Equal Size Bands (ESB) approach described in [11]. However, collecting
such data may be both expensive and time-consuming [8], and approximation tech-
niques based on historical data are of little use without such data. This situation fre-
quently occurs in the software industry. Additionally, COSMIC size approximation
techniques were initially developed with a small sample of Functional Process (FP)/Use
Cases (UC).
To tackle this situation, a different approximation approach using fuzzy logic, re-
ferred to as the EPCU COSMIC size approximation technique was proposed by Valdés
et al. [9, 10, 11]. This approach does not require local calibration and is useful when
there are no historical data available. Additionally, it is less expensive than the calibra-
tion of the ESB approach or any other approximation approach that requires historical
data [8, 9, 10].
Research on the EPCU size approximation technique has focused on two granularity
levels [11, 12] of the Functional User Requirements (FUR) description: Functional Pro-
cess [7] and Use Case [12], with different EPCU context definitions, especially about
changing the domain of its output variable function.
In order to analyze which of both EPCU contexts utilized and previously docu-
mented [8, 9, 10] exhibits, a better performance for each granularity level of the FUR
description, in 2017 Valdés [13] investigated and compared using non-parametric test-
ing, which of the EPCU contexts (with upper size boundaries at 16.44 CFP1 as defined
in [9] and 44 CFP as defined in [10]) appears to better represent the distribution of the
REAL sizes, when the granularity level description was Functional Process.
1 In this paper when functional size unit is CFP, the version is v4.0.1.
71
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
This paper presents a case study with a more extensive set of Use Cases aiming to
identify which of the approximation techniques (the ESB technique and the EPCU tech-
nique using two distinct upper size boundaries) perform best, which means, statistically
demonstrating which values distribution from the approximation techniques is more
similar to REAL functional size distribution employing the standard COSMIC method,
when the functional requirements are at the granularity level of Use Cases, a situation
that presents very often in the industry.
It is known that there is no standard definition for Use Case; however, it has been
observed that frequently, Use Cases correspond to more than one Functional Process,
considering the results in [13], where the EPCU context with upper size boundaries at
16.4 CFP (EPCU16.4) appears to represent the distribution of the REAL sizes better,
and when the granularity level description was Functional Process, the hypothesis for
this work was the following:
H: The EPCU context with upper size cut-off at 44 CFP (EPCU44) better represents
the distribution of the REAL sizes, when the granularity level of the functional user
requirements description was Use Cases.
The structure of this paper is organized as follows. Section II presents related work.
Section III presents the experiment. Section IV presents the data including statistical
analysis, while Section V, the conclusions with suggestions for further work.
2 Related work on functional size approximation techniques
2.1 Approximation techniques based on averages
The IFPUG Function Point Analysis approximation technique for sizing was initially
proposed in 1992 by Bock [16]. In 1997, Meli [14] proposed two variants but did not
report on their performance.
In 2003, Desharnais et al. [4] analyzed two approximation techniques commonly
used in the industry: Function Points Simplified (FPS) [15], and Backfiring from lines
of code [16]. Using the detailed data from this study (e.g., 90 business information
projects from five organizations), the FPS technique, with average weights for each of
the five function types of the IFPUG Function Points method, exhibited better perfor-
mance (MMRE = 10.4%2 and PRED (0.15) = 76.2), while results from the Backfiring
approach were highly inconsistent.
In 2004, Conte et al. [3] extended the Early & Quick (E&Q) technique to the
COSMIC FSM method and indicated that further tests would be needed to make ad-
justments to the proposal, or to confirm it. This E&Q technique is based on (direct)
analogy and (derived) analysis. It is a human-based size approximation technique im-
pacted by the ability to “recognize” which components of the system belong to the
proposed classes [17].
Since 2007, in the COSMIC document “Related Topics” [18] that evolved in 2015
into the COSMIC Guideline for Early or Rapid COSMIC Functional Size Measurement
2 MMRE, PRED(0.15) calculations using the detailed data from [4]
72
F. Valdés-Souto
[6], two approximation techniques were based on averages where documented: the av-
erage Functional Process approach, and the average Use Case approach.
2.2 Approximation techniques based on size bands
In 2007, in a study of 50 projects, Vogelezang et al. [5] reported on a proposed size
approximation technique based on size bands using the quartile approach. The authors
also investigated the influence of distinct factors in approximate sizing and reported
that, within this sample, the sole factor that exerted a substantial influence on the size
of an average Functional Process in each of the quartiles was the number of Functional
Processes [5]. In their case study, a reference software system with a full set of stable
requirements and stated measured functional size was available.
In general, an approach to approximate the size of a scaling factor for FUR type(s)
of artifact(s) must be defined locally [18]. This requires, for instance, that an average
size of the artifacts to be measured be established locally.
This scaling factor represents the size that one can expect to be measured when FUR
are at a level of detail where an accurate measurement can be made because all neces-
sary details are available [5]. This solution requires historical data to produce an ade-
quate scaling factor. In 2011, Santillo [1] proposed the Early and Quick COSMIC sizing
approximation, based on earlier work [3] and the Analytic Hierarchy Process [19], a
technique, which provides a means for making choices among sizing alternatives.
In 2013, Almakadmeh [17] designed a framework to assign scaling factors for iden-
tifying the granularity level of documentation of the functional requirements. Two var-
iants of criteria for assessing granularity levels were defined: the first considered a
functional component of software, and the second, the elements of a UML use-case
model. To rank the levels of granularity identified, the scaling factors used in [5] were
selected. Next, scaling factor assignment was based on conducting an analogy-based
comparison with similar pieces of software in which the functional size of the software
pieces was accurately measured using the COSMIC measurement method.
In 2014, De Vito et al. [20] proposed a simplified measurement process
(Quick/Early) that addressed the need for a simplified and rapid COSMIC measurement
avoiding the use of scaling factors, where incorrect calibrations of scaling factors can
lead to inaccurate approximations. The Quick/Early approximation approach can be
applied on Use Case models to reduce measurement time. Quick/Early precision is di-
rectly proportional to the granularity level of the Use Case model analyzed. This means
that Use Cases require stable requirements that, however, do not occur too frequently
in the early stages. Nonetheless, the authors concluded that Quick/Early accuracy is
adequate.
2.3 Approximation techniques base on fuzzy logic
In 2012, Valdés et al. [9] proposed a COSMIC size approximation solution using a
fuzzy logic model referred to as the Estimation of Projects in a Context of Uncertainty
(EPCU) [2, 21, 22].
73
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
The advantages of the EPCU size approximation technique can be summarized as
follows [ 8, 9, 10]:
─ Does not require local calibration and is useful when there are no historical data
available.
─ Less expensive to calibrate than the ESB approach, which requires historical data.
─ Exhibits good behavior, even when individuals are not acquainted with the COSMIC
method.
─ Exhibits good behavior, even when requirements are not fully known.
─ Enables systematic replication of the information.
In these studies [2, 21, 22], two EPCU contexts were defined for a continuous range
of possible values with a “natural” upper boundary, or cut-off instead of size bands, and
a mixture of granularity levels (Functional Process and Use Case), simulating the early
phases of the software life cycle:
1. The first EPCU context, defined a cut-off at 16.4 CFP [8, 9] (EPCU16.4), based on
the ESB approach as defined by Vogelezang [5] (Small = 4.8 CFP, Medium =7.7
CFP, Large = 10.7 CFP, and Very Large = 16.4 CFP), and
2. The second context defined a cut-off at 44 CFP [11] (EPCU44), defined after ana-
lyzing the database used by Vogelezang [5], that contains two general analyses over
the functional process measured labeled Q-Size and Q-Number. Considering the Q-
Size where the total measured size is divided into quartiles and the average FP size
is calculated from each one (Q1=3.7 CFP, Q2=7.7 CFP, Q3=14.6 CFP and Q4=44.1
CFP)
For this new study, it is considered the integrated analysis, the concept of both is
described below.
EPCU approach research also focused on the definition of the EPCU context, select-
ing several samples from case studies, usually an industry or reference project with
fewer than 12 practitioners, focusing on analyzing the performance of the approxima-
tion technique in the early phases.
For instance, Valdés et al. [10] reported on a case study of a simulation of early
approximation using the EPCU model for an industry project for which only the names
of the Use Cases were made available to participants. This case study confirmed that
the EPCU size approximation approach does not require local calibration and is useful
when there are no historical data available. Besides, it proved less expensive than cali-
bration of the ESB approach, which requires historical data. In this case study, the out-
put variable was defined for a continuous range of possible values with an upper bound-
ary, or cut-off instead of size bands, at 16.4 CFP, as per the ESB approach defined by
Vogelezang et al. [5]. For a case study with a REAL industrial project, the EPCU size
approximation technique yielded better results than the ESB approach, while both tech-
niques led to lower sizes than the real functional size.
In 2015, Valdés et al. [11] proposed another version of their fuzzy logic size approx-
imation technique. It defined a continuous range of possible values for the output vari-
able with an upper Q4 (4th Quartile) cut-off of 44 CFP for a Functional Process using
74
F. Valdés-Souto
the dataset of Vogelezang et al. [5]. For the study of an industry project that considered
Use Case granularity level, the EPCU cut-off at 44 CFP [11] yielded better results on
comparison with the ESB approach and EPCU cut-off at 16.4 CFP [10]. The Functional
size was underestimated for Functional Process or Use Cases using the EPCU cut-off
at 16.4 CFP. On the other hand, results were above and below the REAL value for Use
Cases using the EPCU cut-off at 44 CFP. More realistic results were obtained using the
EPCU44.
Research on the EPCU size approximation technique has focused on two granularity
levels [11, 12] of the FUR description: Functional Process [7], and Use Case [12],
using two EPCU context definitions; however, it was not clear when to utilize each
EPCU context (EPCU16.4, EPCU44), in order to analyze which of the two has a better
performance for each granularity level of functional requirements. In 2017, Valdés [13]
investigated and compared using a non-parametric test, which of the EPCU contexts
appeared to represent the distribution of the REAL sizes better, when the granularity
level was Functional Process.
In the study [13], it was statistically demonstrated that distribution for approximation
values using EPCU16.4 was similar to REAL value distribution employing the standard
COSMIC method with 180 Functional Process.
There is no standard definition for Use Case, and it has been observed that frequently
that Use Cases involve more than one Functional Process, sounds logical that the EPCU
approximation technique with a cut-off of 44 CFP might be more useful if functional
requirements are at the granularity level of Use Cases, a situation that occurs very fre-
quently in the industry. However, based on the findings of [13], the valid conclusion
is that the EPCU44 approach is not as useful with the Functional Process level of gran-
ularity, as it leads to oversizing, and a similar assessment, but employing Use Cases, is
proposed as further work.
2.4 Smmary of COSMIC approximation techniques
The validity of the majority of approximation techniques is dependent on the represent-
ativeness of the samples with respect to the software being approximated. In other
words, the majority of approximation methods require local calibration, and this re-
quires local historical data. Even more COSMIC size approximation techniques were
initially developed with a small sample of data. However, as pointed out by Morgensht-
ern [8]: “Algorithmic models need historical data, and many organizations do not have
this information. Additionally, collecting such data may be both expensive and time-
consuming.” Approximation techniques based on historical data are of little use for
organizations without such data. Alternatives must, therefore, be developed for such
contexts of approximation.
The COSMIC Guideline for Early or Rapid COSMIC Functional Size Measurement
[6] integrates several techniques for the approximate sizing of new, ‘whole’ sets of re-
quirements. The approximation techniques described in [6] include approximation
techniques based on size bands or based on average.
75
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
The majority of the techniques presented in [6] are based on the existence of histor-
ical data to determine the scaling factor (average, or size bands) or another calibration,
and that there are stable requirements [11].
2.5 Impact of approximated size on the estimation of effort
In 2013, De Marco et al. [23] investigated to what extent some COSMIC-based approx-
imate sizing could be useful for project managers for early effort estimation for Web
applications. The authors reported an empirical analysis employing data from 25 Web
applications to assess whether two approximate sizes (number of COSMIC Functional
Processes (FP) or the Average Functional Process approach) could be exploited to ac-
quire accurate effort estimates. These authors concluded that COSMIC-based approxi-
mate sizing was a suitable approach for early effort estimates, while estimates obtained
with approximate sizes were worse than those achieved employing the size obtained
from the application of the standard COSMIC method.
3 Experiment with approximation techniques
This section describes the experiment carried out to evaluate the size approximation
techniques and identify which technique appears to represent the distribution of the
REAL sizes better, when the granularity level was Use Cases (UC).
3.1 Context and participants
As a part of a consultancy project whose objective was to implement the use of
COSMIC for a Government entity in Mexico carried out in 2016, with the objective of
generating formal estimation models, several projects were measured using the
COSMIC method.
The three main circumstances described in [6], in which only an approximate
COSMIC functional size may be possible were presented in the project:
─ When a size measurement is needed rapidly, and an approximate size measurement
is acceptable if it can be measured much faster than with the standard method. This
is known as ‘rapid sizing’;
─ Early in the life of a project before the actual requirements have been specified in
enough detail for precise size measurement. This is known as ‘early sizing’;
─ In general, when the quality of the documentation of the actual requirements is not
sufficiently good for precise size measurement.
Considering the information below, the functional size for the projects was gathered
using the approximation approaches as the first step and then, when the required detail
for the requirements was accomplished, the full standard was used to obtain the func-
tional size.
76
F. Valdés-Souto
To conduct a comparison with the previous study [13] focused on 180 Functional
Process, four projects were selected. These four projects integrated 293 Use Cases that
were approximated using ESB and EPCU techniques.
The people in the Government entity received 24 hours of training in COSMIC dur-
ing the consultancy project, including the EPCU approximation technique and that of
equal size bands. The information required for using the approximation techniques were
required from the technical people, specifically from the project leader for each project,
with a distinct project leader for each project.
It is important to mention that the techniques related to the Requirements Engineer-
ing used by the Government entity was not affected by the consultancy and was possible
to observe that sometimes the Use Cases include much functionality. Table 1 shows the
number of Use Cases by project.
Table 1. Use Cases by project considered in the case study
ID
Project # UC Assigned
1 43
2 96
3 55
4 99
Total 293
3.2 Participant instructions for functional size measurement and
approximation
Each project leader was asked to perform the following:
1. Identify, for each project, the set of Use Cases assigned to be developed.
2. Classify (using expert judgment) by size each of the Use Cases using the following
linguistic values: Small; Medium; Large, and Very Large3.
3. Classify (using expert judgment) the number of objects of interest for each of the
Use Cases using the following linguistic values: Few; Average, and Many.
4. Assign values (using expert judgment) in the range 0 - 5 ε R for the two previously
classified input variables (points 2 and 3, the Use Cases’ size, the number of objects
of interest related to the Use Cases) defined within the EPCU context, considering
the subjective classification relative to the functional size of the Use Cases (e.g., Step
2), and the subjective classification for the number of objects of interest in each Use
Case (e.g., Step 3).
3 The linguistic values were defined in concordance to the ESB Approach to enabled the compa-
rison.
77
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
5. Measure functional size using the COSMIC method and provide the size for each
Use Case.
3.3 Data collected by participants
Project leaders identified 293 Use Cases in four projects (Table 1), and the data pro-
vided by the project leaders were the following (see Appendix I for details):
─ A value assigned within the range of 0 - 5 ε R for the size of each Use Case.
─ A value assigned within the range of 0 - 5 ε R for the objects of interest for each Use
Case.
─ COSMIC size using the COSMIC method for each Use Case.
The linguistic classification of the Use Cases and the linguistic classification of the
objects of interest for each Use Cases (data from Steps 2 and 3) were not included in
the table in the Appendix since the input for the EPCU approximation approach were
the values assigned for each variable (data from Step 4).
3.4 Researcher steps
Using the linguistic classification (Small, Medium, Large, and Very Large) assigned
by the participants for the Use Cases the ESB technique was performed.
Using the values (between 0 and 5) assigned by the participants for the two input
variables of the fuzzy logic based EPCU approximation technique, CFP units were per-
formed by the researcher using the EPCU approximation technique with distinct EPCU
contexts (EPCU16.4 and EPCU44) defined in [8, 9] and [11].
The COSMIC size approximated with the data provided by the project leaders was
verified using the COSMIC measurement principles and rules by two consultants with
more than 7,000 CFP measurement experiences at the verification moment.
COSMIC functional size and approximate size for each Use Case are presented in
Appendix II where:
─ Column 1 presents the Project identifier. For confidential purposes, the Projects were
labeled sequentially, from “Proj 1” to “Proj 4.
─ Column 2 presents the Use Case identifier. For confidential purposes, the Use Cases
were labeled sequentially, from “UC 1” to “UC 293.
─ Column 3 presents the functional size obtained utilizing the standard COSMIC
method – in CFP units,
─ Column 4 presents the Equal Size Band approximation approach,
─ Column 5 presents the EPCU size approximation approach using an output variable
domain function from 2 - 16.4 CFP [8] [9], and
─ Column 6 presents the EPCU size approximation approach using an output variable
domain function from 2 - 44 CFP [10].
78
F. Valdés-Souto
4 Data Analysis
4.1 Quality Criteria
Three most frequently quoted quality criteria [24] were used to analyze the behavior of
the two approximation techniques :
─ Mean Magnitude of Relative Error (MMRE),
─ Standard Deviation of MRE (SDMRE), and
─ Prediction level, here PRED(25%) was selected.
The Median Magnitude of Relative Error (MdMRE) is also used. The primary ad-
vantage of the median over the mean is that the median is not sensitive to the outliers.
Table 2 presents the results for each of these quality criteria for each approximation
approach (top line) for the set of 293 Use Cases:
1. With an MMRE of 61.4%, the ESB presented the best results (in comparison to
MMRE = 65.7% with the EPCU16.4 technique and MMRE = 117.4% with
EPCU44).
2. With an SDMRE of 49.1%, ESB presents the best results, in comparison to SDMRE
of 62.2% for the EPCU16.4 technique and SDMRE = 156.1% for the EPCU44 tech-
nique.
3. Within a PRED (25%) at 20.8%, the EPCU with a cut-off at 44 CFP presents the
best results, in comparison to 18.8% with ESB and 17.1% with EPCU with the cut-
off at 16.4 CFP.
4. With a MdMRE of 56.9%, the EPCU16.4 technique presents the best results, in com-
parison to 59.5% with ESB and 63.3% with EPCU with a cut-off of 44 CFP. It is
possible to observe that the difference between the maximal and the minimal
MdMRE values are less than the other quality criteria.
Two quality criteria present the best results in the ESB approach (MMRE and
SDMRE); however, the prediction level presents the best results for the EPCU with the
cut-off at 44 CFP, and the MdMRE presents the best results for the EPCU16.4.
Table 2. Approximation technique performance for 293 Use Cases
ESB EPCU 16.4 EPCU 44
MMRE 61.4% 65.7% 117.4%
MdMRE 59.5% 56.9% 63.3%
SDMRE 49.1% 62.2% 156.1%
PRED(25%) 18.8% 17.1% 20.8%
From the quality criteria, it is not clear which approximation technique has the best
performance, this because the central tendency measurements are affected by outliers.
79
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
MMRE has been shown to be a biased estimator of central tendency of the residuals
of a prediction system because it is an asymmetric measure [25], [26], [27], [28]. Shep-
perd et al. [29] proposed the Mean Absolute Residual (MAR), which, unlike MMRE,
is not biased to compare the accuracy of a given estimation method P against the accu-
racy of a reference estimation method P0.
1
MAR = ∑𝑛𝑖=1 |𝑦𝑖 − 𝑦̂𝑖| (1)
𝑛
Based on the calculated MARP (the MAR of the proposed method) and MARP0 (the
MAR of a reference method), Shepperd et al. [29] propose to compute a Standardized
Accuracy measure (SA) for estimation method P.
𝑀𝐴𝑅𝑃
SA = 1 − (2)
𝑀𝐴𝑅𝑃0
Where values of SA close to 1 indicate that P outperforms P0, values close to zero
indicate that P’s accuracy is similar to P0’s accuracy, and the negative values indicate
that P is worse than P0. The authors [29] suggest to use a referenced model random
based considering the known (actual) values of previously measured projects, however,
Lavazza [30] observed that the comparison with random estimation is not very effective
in supporting the evidence that P is a good estimation model. Instead, proposed to use
a “Constant Model” (CM), where the estimate of the size of the ith project is given by
the average of the sizes of the other projects, then the calculation of the MARCM of
these estimates is realized, and then the compute of SA, comparing method P with a
method CM, generalizing that SA could be used to compare an estimation method P
against any other method P1 used as a reference method.
𝑀𝐴𝑅𝑃
SA = 1 − (3)
𝑀𝐴𝑅𝑃1
Table 3. Standardized Accuracy measure using the ESB as a reference (P1)
EPCU 16.4 EPCU 44
MAR
Calculated using (1), 16.7 17.6
The MAR for ESB = 17.7
SA
Calculated using (3) -0.96 -1.05
Considering ESB as P1
Table 3 presents the results related to the comparison between each EPCU context
(top line), considering the Standardized Accuracy measure approach proposed for
Lavazza [30], using the ESB approximation approach as CM as in (3). With a SA close
to zero (0.05 for EPCU 16,4 and 0.01 for EPCU 44), both EPCU context present similar
accuracy to the reference approximation approach (ESB). Considering the SA measure,
80
F. Valdés-Souto
the ESB present a better result, it is not clear which approximation technique has the
best performance.
4.2 Graphical Analysis
Fig. 1 A), graphically presents, for each of the 293 Use Cases, the REAL COSMIC size
(in blue) and the size approximated with the ESB technique (in orange).
Real v.s. ESB
400
350
300
250
200
150
100
50
0
0 50 100 150 200 250 300
Real ESB
Real v.s. ESB
100
80
60
40
20
0
0 50 100 150 200 250 300
Real ESB
Fig. 1. REAL COSMIC size vs. approximated size using the ESB technique – 293 Use Cases. A)
Vertical axis boundaries at 400 CFP. B) Vertical axis boundaries at 100 CFP.
81
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Note that with the ESB technique, the only four values possible for the approximated
size (in orange) are as follows: 4.8 CFP; 7.7 CFP; 10.7 CFP, and 16.4 CFP correspond-
ing to the four average size bands of Functional Process (Small, Average, Large, and
Very Large).
From the data (Appendix II, column 3), it is possible to conclude that 230 Use Cases
(78.5%) were underestimated; in consequence, overestimated 63 Uses Cases are
(21.5%). From these overestimated Use Cases, 139 are due to that the upper boundary,
or cut-off, was established at 16.4 CFP and the Use Cases had a functional size higher
than that of the cut-off.
Fig. 2 depict the graphical comparison with the EPCU16.4 technique. This technique
defines a continuous range of possible values between 2 CFP and an upper boundary
or cut-off at 16.4 CFP; consequently, at least 139 Use Cases were underestimated be-
cause of the upper boundary.
Looking at the data from (Appendix II, column 3), overestimated Uses Cases num-
bered 99 (33.8%), while underestimated Use Cases numbered194 (66.2%). It is possible
to observe that the number of Use Cases underestimated decrease in 36 Use Cases con-
sidering the ESB technique, and the Use Cases overestimated increase.
Fig. 3 presents the graphical comparison with the EPCU44 technique because this
approach has a cut-off at 44 CFP; naturally, fewer Use Cases were underestimated, 130
(44.4%), while overestimated Use Cases numbered 163 (55.6%), and for the EPCU44
technique, more Uses Cases were overestimated.
Intuitively from the previous figures, the EPCU44 better represents the distribution
of the REAL sizes; however, it is not easy to infer from Fig.1 to Fig. 3, because there
are several outliers. This confirms the reason regarding the big difference between the
maximal and the minimal values for MdMRE and MMRE from Table 2.
Considering the difference between MdMRE and MMRE, it is possible to assume
that the distribution is skewed and that the most representative value is the MdMRE,
because central tendency measurements were affected by the outliers.
In Fig. 4, the boxplots related to the REAL Value of functional size, and ESB,
EPCU16.4, and EPCU44 functional size approximation, are presented. This is a better
approach for analyzing the data without considering the outliers.
From Fig. 4, it might be easier to infer that EPCU44 better represent the distribution
of the REAL sizes, because both boxplots are very similar.
82
F. Valdés-Souto
Real v.s. EPCU16.44
400
350
300
250
200
150
100
50
0
0 50 100 150 200 250 300
Real EPCU16.4
Real v.s. EPCU16.44
100
80
60
40
20
0
0 50 100 150 200 250 300
Real EPCU16.4
Fig. 2. REAL COSMIC size vs. approximated size using the EPCU-16.4 technique – 293 Use
Cases. A) Vertical axis boundaries at 400 CFP. B) Vertical axis boundaries at 100 CFP.
83
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Real v.s. EPCU44
400
350
300
250
200
150
100
50
0
0 50 100 150 200 250 300
Real EPCU44
Real v.s. EPCU44
100
80
60
40
20
0
0 50 100 150 200 250 300
Real EPCU44
Fig. 3. REAL COSMIC size vs. approximated size using the EPCU-44 technique – 293 Use
Cases. A) Vertical axis boundaries at 400 CFP. B) Vertical axis boundaries at 100 CFP
4.3 Non-parametric test
Considering the quality criteria affected by the central tendency measurements, the ap-
proximation technique that provides better results was ESB. From the plots in Fig.s 1,
2, 3, and 4, the EPCU44 technique appears to better represent the distribution of the
REAL sizes; however, this needs to be confirmed by statistical analysis.
In non-parametric statistics, a well-known procedure for testing the differences
among more than two related samples is the Friedman test [24, 25] The objective of the
test is to determine whether it can be concluded, from a sample of results, that there is
a difference among treatment effects [32].
84
F. Valdés-Souto
Using the Friedman non-parametric test to analyze whether there is a difference
among the performances of distinct treatments across the same datasets for functional
size, that is, whether the data distributions are equal, a null hypothesis H0 was defined
as:
H0: There are NO meaningful differences in the distributions of REAL, ESB,
EPCU16.4, and EPCU44 datasets.
In consequence, the alternative hypothesis was defined as:
H1: At least one distribution (REAL, ESB, EPCU16.4, and EPCU44) is significantly
different. A significance level of ɑ (alpha (ɑ)) = 0.05 was assumed.
SPSS® version 22 software in the Spanish language was utilized to evaluate the
Friedman test for the four distinct treatments (REAL, ESB, EPCU16.4, and EPCU44),
and the results are summarized in Table 4. The full results from SPSS ® are presented
in Appendix III.
In Table 4, “N” represents the 293 Use Cases, “df” represents the degrees of freedom
(with four distinct treatments; the df is 3 (#treatments -1)). Here, the statistical signifi-
cance (“Asymp. Sig.” or p-value) is a very small number at E-101, thus below the re-
quired significance level of ɑ =0.05.
Therefore, the null hypothesis (e.g., H0: There are NO meaningful differences in the
distribution of REAL, ESB, EPCU 16.4, and EPCU 44) is rejected, and it is possible to
state that at least one treatment has a distinct distribution.
Fig. 4. Boxplots related to the REAL Value of functional size, and ESB, EPCU16.4, and EPCU44
functional size Approximation.
85
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Table 4. Friedman test results on the testing of the four distributions (REAL, ESB, EPCU16.4,
and EPCU44)
N 293
Chi-Square 468.936
df 3
Asymp. Sig. 2.5722E-101
In order to identify where the difference is, a post-hoc test is needed. In this instance,
a post-hoc test assesses the difference between treatments as follows:
─ REAL and ESB
─ REAL and EPCU16.4
─ REAL and EPCU44
─ ESB and EPCU16.4
─ ESB and EPCU44
─ EPCU-16.4 and EPCU44
Here, the post-hoc test compared two treatments at a time. The Wilcoxon [32] test
was executed using SPSS® software, and the Bonferroni correction [33] was consid-
ered; thus, the ɑ value (ɑ =0.05) was divided by 4 because four distinct treatments were
used. This means that the ɑ was reset at ɑ =0.0125.
Considering the latter, the null hypothesis H0 for the post-hoc test was:
H0: There are NO meaningful differences between the distributions for the two treat-
ments compared (see the previous list).
In consequence, the alternative hypothesis was defined as:
H1: The distribution for the two treatments compared is significantly different, as-
suming a significance level of ɑ = 0. 0125.
Table 5 presents the results of applying the Wilcoxon test for two treatments in
SPSS®. Column 1 indicates the comparison, and column 2, the significance for the
Wilcoxon test. The significance value was compared with ɑ = 0.0125 by accepting (>ɑ
= 0.0125) or rejecting (<ɑ = 0.0125) the null hypothesis; the results are presented in
column 3. The full results from SPSS® are presented in Appendix IV.
From Table 5, with a p-value of ɑ =0.0125, it is possible to confirm statistically that
only the distribution of the EPCU44 approximation technique (with a cut-off at 44 CFP)
displays a behavior similar to the distribution of the COSMIC REAL sizes (REAL
value), considering the granularity level of Use Cases, which graphically could be ob-
served in Fig. 4.
86
F. Valdés-Souto
Table 5. Wilcoxon post hoc test results
Comparison Asymp. Sig. Statistical Signif-
icance
(p-value)
>ɑ =0.0125
REAL and ESB 4.8089E-30 NO
REAL and 3.6339E-19 NO
EPCU16.4
REAL and 0.281 YES
EPCU44
ESB and 1.3091E-38 NO
EPCU16.4
ESB and 8.7473E-50 NO
EPCU44
EPCU16.4 and 8.2436E-50 NO
EPCU44
5 CONCLUSIONS
In this paper, using a large sample of 293 Use Cases from real projects, two approxi-
mation techniques were evaluated to identify which performs best with this dataset
larger, which is larger than previous sets mentioned in related works. This implies sta-
tistically demonstrating which value distribution from the approximation techniques is
more similar to REAL functional size distribution employing the standard COSMIC
method, when the functional requirements are at the granularity level of Use Cases, a
situation encountered very frequently in the industry.
From the previous work [13], the EPCU context appears to represent the distribution
of the REAL sizes better; when the granularity level was Functional Process, 180 Func-
tional Process were used.
From our findings related to quality criteria, it is not clear which approximation tech-
nique executes the best performance, this is because the central tendency measurements
are affected by outliers, and the sample has several outliers, as in reality occurs.
It is well known that there is no standard definition for Use Case, and this could be
a reason for the outliers. For instance, there are Use Cases with more than 100 or 300
CFP. The presence of outliers can be observed in Fig.s 1 - 4, even though, intuitively
from the previous figures, the EPCU44 better represents the distribution of the REAL
sizes. However, it is not easy to infer.
87
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
On carrying out the non-parametric test, it is possible to confirm statistically that
only the distribution of the EPCU44 approximation technique displays behavior similar
to that of the distribution of the COSMIC REAL sizes (REAL value), considering the
granularity level of Use Cases, accepted the following hypothesis:
H: The EPCU context with an upper size cut-off at 44 CFP (EPCU44) better repre-
sents the distribution of the REAL sizes, when the granularity level of the FUR descrip-
tion was Use Cases.
Considering the findings and the previous work, it is possible to define when the
granularity level of the FUR description was Use Cases, with our recommending the
EPCU44 approximation approach, while when the granularity level of the functional
user requirements description was Functional Process, the EPCU16.4 approximation
approach is recommended.
The research developed in this paper only includes two of the approximation tech-
niques mentioned in the Guideline for Early or Rapid COSMIC Functional Size Meas-
urement [6]; others should be investigated as well, using similar experiments.
Because the spread of the use of agile practices, a similar assessment to that of this
paper but employing User Histories as the granularity level of the functional user re-
quirements description should be conducted.
References
1. L. Santillo, Early and Quick COSMIC FFP Overview, in: A.A. Reiner
Dumke (Ed.), Cosm. Funct. Points Theory Adv. Pract., CRC Press,
Boca Raton, FL, USA, 2011: pp. 176–191.
2. F. Valdés-Souto, Design of a Fuzzy Logic Software Estimation Process,
École De Technologie Supérieure, Université Du Québec, 2011.
3. L. Conte, M., Iorio, T., Santillo, E&Q: An Early & Quick Approach to
Functional Size Measurement Methods, in: Istituto di Ricerca
Internazionale (Ed.), Softw. Meas. Eur. Forum SMEF 2004, Rome,
Italy, 2004: p. 416.
4. J.M. Desharnais, A. Abran, Approximation Techniques for Measuring
Function Points, Proc, in: 13th Inter. Work. Softw. Meas. (IWSM 2003),
Springer, Montreal, Canada, 2003: pp. 270–286.
5. F. Vogelezang, T. Prins, Approximate size measurement with the
COSMIC method: Factors of influence, in: Softw. Meas. Eur. Forum
SMEF 2007, Rome, Italy, 2007: pp. 167–178.
6. Common Software Measurement International Consortium
(COSMIC)., Guideline for Early or Rapid COSMIC Functional Size
Measurement, 2015.
7. Common Software Measurement International Consortium (COSMIC),
Measurement Manual v4.0.1, 2015.
8. O. Morgenshtern, T. Raz, D. Dvir, Factors Affecting Duration and
88
F. Valdés-Souto
Effort Estimation Errors in Software Development Projects, Inf. Softw.
Technol. 49 (2007) 827–837. doi:10.1016/j.infsof.2006.09.006.
9. F. Valdés-Souto, A. Abran, Case Study: COSMIC Approximate Sizing
Approach Without Using Historical Data, in: Jt. Conf. 22nd Int. Work.
Softw. Meas. 2012 Seventh Int. Conf. Softw. Process Prod. Meas.,
IEEE, Assisi, Italy, 2012: pp. 178–189. doi:10.1109/IWSM-
MENSURA.2012.34.
10. F. Valdés-Souto, A. Abran, COSMIC Approximate Sizing Using a
Fuzzy Logic Approach: A Quantitative Case Study with Industry Data,
in: F. Vogelezang, M. Daneva (Eds.), 2014 Jt. Conf. Int. Work. Softw.
Meas. Int. Conf. Softw. Process Prod. Meas., Conference Piblishing
Services (CPS), Rotterdam (Netherlands), 2014: pp. 282–292.
doi:10.1109/IWSM.Mensura.2014.44.
11. F. Valdés-Souto, A. Abran, Improving the COSMIC Approximate
Sizing Using the Fuzzy Logic EPCU Model, in: A. Kobylinski, B.
Czarnacka-Chrobot, J. Swierczek (Eds.), Jt. Conf. 25rd Int. Work.
Softw. Meas. 10th Int. Conf. Softw. Process Prod. Meas. - IWSM-
MENSURA 2015, Springer International Publishing, Cracow (Poland),
2015: pp. 192–208. doi:10.1007/978-3-540-71649-5.
12. M. Fowler, UML Distilled, A Brief Guide to the Standard Object
Modeling Language, Third Edit, Addison Wesley, n.d.
13. F. Valdés-Souto, Analyzing the performance of two COSMIC
approximation sizing techniques at the functional process level, Sci.
Comput. Program. 135 (2017) 105–121.
doi:10.1016/j.scico.2016.11.005.
14. R. Meli, Early Function Points: A New Estimation Method for Software
Projects, in: ESCOM 1997, Berlin, Germany, 1997: pp. 1–10.
15. D.B. Bock, Robert Klepper, FP-S: A Simplified Function Point
Counting Method, J. Syst. Softw. 18 (1992) 245–254.
16. C. Jones, Applied Software Measurement, Assuring Productivity and
Quality, 2nd ed., McGraw Hill, New York, NY, 1997.
17. K. Almakadmeh, Development of a Scaling Factors Framework to
Improve the Approximation of Software Functional Size with COSMIC
- ISO19761, École de Technologie Supérieure, Université Du Québec,
2013.
18. Common Software Measurement International Consortium
(COSMIC)., Advanced and Related Topics, 2007.
19. L. Santillo, Early FP Estimation and the Analytic Hierarchy Process, in:
ESCOM-SCOPE 2000, Munich, April, Munich, Germany, 2000: pp. 1–
20.
20. G. De Vito, F. Ferrucci, Approximate COSMIC Size: The Quick/Early
Method, in: Proc. - 40th Euromicro Conf. Ser. Softw. Eng. Adv. Appl.
SEAA 2014, IEEE, Verona, Italy, 2014: pp. 69–76.
89
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
doi:10.1109/SEAA.2014.30.
21. F. Valdés-Souto, A. Abran, Industry Case Studies of Estimation Models
Using Fuzzy Sets, in: Reiner Dumke (Ed.), Softw. Process Prod. Meas.
Int. Conf. IWSM-Mensura 2007, UIB-Universitat de les Illes Baleares,
Illes Baleares, Spain, 2007: pp. 87–101.
22. F. Valdés-Souto, A. Abran, Comparing the Estimation Performance of
the EPCU Model with the Expert Judgment Estimation Approach Using
Data from Industry, in: R. Lee (Ed.), Softw. Eng. Res. Manag. Appl.
2010, Springer-Verlag, Berlin, 2010: pp. 227–240.
23. L. De Marco, F. Ferrucci, C. Gravino, Approximate COSMIC Size to
Early Estimate Web Application Development Effort, in: 39th
Euromicro Conf. Ser. Softw. Eng. Adv. Appl. Approx., Conference
Publishing Services (CPS), Santander, Spain, 2013: pp. 349–356.
doi:10.1109/SEAA.2013.41.
24. M. Friedman, The Use of Ranks to Avoid the Assumption of Normality
Implicit in the Analysis of Variance, J. Am. Stat. Assoc. 32 (1937) 675–
701.
25. L. Lavazza, Accuracy Evaluation of Model-based COSMIC Functional
Size Estimation, in: ICSEA 2017 Twelfth Int. Conf. Softw. Eng. Adv.,
2017: pp. 67–72.
26. B.A. Kitchenham, L.M. Pickard, S.G. MacDonell, M.J. Shepperd, What
accuracy statistics really measure, in: IEE Proc. - Softw., 2001: pp. 81–
85. doi:10.1049/ip-rsn:20010506.
27. T. Foss, E. Stensrud, B. Kitchenham, I. Myrtveit, A simulation study of
the model evaluation criterion MMRE, Softw. Eng. IEEE Trans. 29
(2003) 985–995.
28. I. (Norwegian S. of M. Myrtveit, E. (Buskerud U.C. Stensrud, M.
(Bournemouth U. Shepperd, Reliability and validity in comparative
studies of software prediction models, IEEE Trans. Softw. Eng. 31
(2005) 380–391. doi:10.1109/TSE.2005.58.
29. M. Shepperd, S. MacDonell, Evaluating prediction systems in software
project estimation, Inf. Softw. Technol. 54 (2012) 820–827.
doi:10.1016/j.infsof.2011.12.008.
30. L. Lavazza, S. Morasca, On the Evaluation of Effort Estimation Models,
in: Proc. 21st Int. Conf. Eval. Assess. Softw. Eng. - EASE’17,
Karlskrona, Sweden, 2017: pp. 41–50. doi:10.1145/3084226.3084260.
31. J. Demšar, Statistical Comparisons of Classifiers over Multiple Data
Sets, J. Mach. Learn. Res. 7 (2006) 1–30.
32. S. García, A. Fernández, J. Luengo, F. Herrera, Advanced
nonparametric tests for multiple comparisons in the design of
experiments in computational intelligence and data mining:
Experimental analysis of power, Inf. Sci. (Ny). 180 (2010) 2044–2064.
doi:10.1016/j.ins.2009.12.010.
90
F. Valdés-Souto
33. O.J.D. Source, Multiple Comparisons Among Means, J. Am. Stat.
Assoc. 56 (1961) 52–64.
91
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Presence
(level,
Appendix I. Data Provided by Participants not num-
for Each Use Case Identified ber) of
objects
Table A1 shows the data provided by partici- of inter-
pants for each Functional Process identified in the Use Case size est re-
experiment. Project (value assign- lated to
UC ID
- Column 1 presents the Project identifier. For ID ment – range the Use
confidentially purposes, the Projects were la- from 0 - 5) Case
beled sequentially, from “Proj 1” to “Proj 4. (value
assign-
- Column 2 presents the Use Case identifier. For ment –
confidentially purposes, the Use Cases were la- range
beled sequentially, from “UC 1” to “UC 293. from 0 -
- Column 3 presents the functional size obtained 5)
utilizing the standard COSMIC method – in
CFP units, Proj 1 UC 9 3.5 3
- Column 4 presents the value assigned for the in- Proj 1 UC 10 3.5 3.5
put variable “Use Case size” for the EPCU ap-
proximation technique.
Proj 1 UC 11 3.5 2.95
- Column 5 presents the value assigned for the in-
put variable “Presence of objects of interest re- Proj 1 UC 12 3.5 3
lated to the Use Cases” for the EPCU approxi- Proj 1 UC 13 3.5 2.75
mation technique.
Proj 1 UC 14 4 2.5
Table A1: Data collected by participants (project lead-
ers) Proj 1 UC 15 3.5 3.65
Presence
Proj 1 UC 16 3 4
(level,
not num- Proj 1 UC 17 3.5 3
ber) of
objects Proj 1 UC 18 3 3
of inter-
Proj 1 UC 19 3 3
Use Case size est re-
Project (value assign- lated to Proj 1 UC 20 3.5 3.5
UC ID
ID ment – range the Use
from 0 - 5) Case Proj 1 UC 21 3.5 3.5
(value
assign- Proj 1 UC 22 3.5 3.5
ment –
range
Proj 1 UC 23 4 3.5
from 0 -
5)
Proj 1 UC 24 4 3.5
Proj 1 UC 1 3 2.5
Proj 1 UC 2 3 3 Proj 1 UC 25 3.5 3.5
Proj 1 UC 3 3 3
Proj 1 UC 26 4 3.5
Proj 1 UC 4 3 2.5
Proj 1 UC 5 3.5 3 Proj 1 UC 27 3.5 3.5
Proj 1 UC 6 3 3 Proj 1 UC 28 4 3
Proj 1 UC 7 3.5 3.5 Proj 1 UC 29 3.5 3
Proj 1 UC 8 3.5 3 Proj 1 UC 30 3.5 3
Proj 1 UC 31 3.5 3
92
F. Valdés-Souto
Presence Presence
(level, (level,
not num- not num-
ber) of ber) of
objects objects
of inter- of inter-
Use Case size est re- Use Case size est re-
Project (value assign- lated to Project (value assign- lated to
UC ID UC ID
ID ment – range the Use ID ment – range the Use
from 0 - 5) Case from 0 - 5) Case
(value (value
assign- assign-
ment – ment –
range range
from 0 - from 0 -
5) 5)
Proj 1 UC 32 4 3.5 Proj 2 UC 68 2.35 2.35
Proj 2 UC 69 2.55 2.1
Proj 1 UC 33 4 3
Proj 2 UC 70 2.55 2.1
Proj 1 UC 34 4 3 Proj 2 UC 71 2.6 2.6
Proj 2 UC 72 2.5 1.4
Proj 1 UC 35 4 3.5
Proj 2 UC 73 2.4 2.6
Proj 1 UC 36 4 3.5 Proj 2 UC 74 1.7 1.95
Proj 1 UC 37 4 4 Proj 2 UC 75 2.1 2.1
Proj 1 UC 38 3.85 3.5 Proj 2 UC 76 2.3 1.95
Proj 1 UC 39 4 3.5 Proj 2 UC 77 1.9 1.6
Proj 1 UC 40 4 3 Proj 2 UC 78 2.3 2.3
Proj 1 UC 41 4 4 Proj 2 UC 79 2.45 2.6
Proj 1 UC 42 4 4 Proj 2 UC 80 2.35 2.15
Proj 1 UC 43 4 3.5 Proj 2 UC 81 2.8 2.6
Proj 2 UC 44 2.4 1.8 Proj 2 UC 82 2.9 2.05
Proj 2 UC 45 2.5 2.05 Proj 2 UC 83 2.5 1.75
Proj 2 UC 46 2.85 2.4 Proj 2 UC 84 2.1 1.95
Proj 2 UC 47 2.3 2.2 Proj 2 UC 85 2.45 2.1
Proj 2 UC 48 3 2.6 Proj 2 UC 86 3.05 1.6
Proj 2 UC 49 2.1 1.85 Proj 2 UC 87 2.35 1.6
Proj 2 UC 50 2.55 2.55 Proj 2 UC 88 3 3
Proj 2 UC 51 2.65 2.45 Proj 2 UC 89 2.1 1.95
Proj 2 UC 52 2.35 2.8 Proj 2 UC 90 1.95 1.95
Proj 2 UC 53 1.65 1.8 Proj 2 UC 91 1.95 1.95
Proj 2 UC 54 2.8 2.8 Proj 2 UC 92 1.8 1.6
Proj 2 UC 55 2.85 2.55 Proj 2 UC 93 2 2.15
Proj 2 UC 56 2.3 2.1 Proj 2 UC 94 2.3 1.95
Proj 2 UC 57 2.6 2.6 Proj 2 UC 95 2.5 2.15
Proj 2 UC 58 1.95 1.95 Proj 2 UC 96 2.45 2.25
Proj 2 UC 59 2.3 2.1 Proj 2 UC 97 2.6 2.6
Proj 2 UC 60 2.65 2.1 Proj 2 UC 98 2.5 2.55
Proj 2 UC 61 2.1 2.05 Proj 2 UC 99 2.5 2.15
Proj 2 UC 62 3.1 2.65 Proj 2 UC 100 3.25 2.2
Proj 2 UC 63 2.5 2.1 Proj 2 UC 101 2.15 1.8
Proj 2 UC 64 2.6 2.25 Proj 2 UC 102 2.25 2.05
Proj 2 UC 65 2.5 2.5 Proj 2 UC 103 2.3 2.1
Proj 2 UC 66 2.35 2.3 Proj 2 UC 104 2.6 1.85
Proj 2 UC 67 2.35 2.1 Proj 2 UC 105 2.65 2.3
93
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Presence Presence
(level, (level,
not num- not num-
ber) of ber) of
objects objects
of inter- of inter-
Use Case size est re- Use Case size est re-
Project (value assign- lated to Project (value assign- lated to
UC ID UC ID
ID ment – range the Use ID ment – range the Use
from 0 - 5) Case from 0 - 5) Case
(value (value
assign- assign-
ment – ment –
range range
from 0 - from 0 -
5) 5)
Proj 2 UC 106 2.8 1.75 Proj 2 UC 137 2.35 2
Proj 2 UC 107 2.75 2.35 Proj 2 UC 138 3.25 1.75
Proj 2 UC 108 2.7 2.55 Proj 2 UC 139 2.6 2.05
Proj 2 UC 109 2.9 2.6 Proj 3 UC 140 3.5 3.75
Proj 2 UC 110 2.85 2.3 Proj 3 UC 141 3.5 3
Proj 2 UC 111 2.85 2.6 Proj 3 UC 142 3.5 2.5
Proj 2 UC 112 2.75 2.55 Proj 3 UC 143 3.25 3.6
Proj 2 UC 113 2.4 2.2 Proj 3 UC 144 3.3 3.4
Proj 2 UC 114 2.5 1.65 Proj 3 UC 145 2.15 3
Proj 2 UC 115 2.05 1.65 Proj 3 UC 146 3.6 3.9
Proj 2 UC 116 2.5 2.5 Proj 3 UC 147 3.5 3.65
Proj 2 UC 117 2.35 1.65 Proj 3 UC 148 2.75 3.05
Proj 2 UC 118 2.55 2.15 Proj 3 UC 149 3.25 3.35
Proj 2 UC 119 1.8 1.75 Proj 3 UC 150 3 4
Proj 2 UC 120 2.3 1.8 Proj 3 UC 151 3.15 3
Proj 2 UC 121 2.1 1.75 Proj 3 UC 152 2.55 3.8
Proj 2 UC 122 2.75 1.75 Proj 3 UC 153 2.6 3
Proj 2 UC 123 2.95 2.2 Proj 3 UC 154 2.55 2.5
Proj 2 UC 124 2.75 2.25 Proj 3 UC 155 3 3.75
Proj 2 UC 125 3.05 2.65 Proj 3 UC 156 2.25 2.75
Proj 3 UC 157 2.25 2.75
Proj 2 UC 126 2.65 1.75
Proj 3 UC 158 3 3
Proj 3 UC 159 2.5 3.25
Proj 2 UC 127 2.7 1.75
Proj 3 UC 160 1.5 3
Proj 2 UC 128 2.5 1.75 Proj 3 UC 161 2.7 2.9
Proj 2 UC 129 2.1 1.75 Proj 3 UC 162 2.45 2.5
Proj 2 UC 130 1.8 1.75 Proj 3 UC 163 2.65 3.25
Proj 2 UC 131 2.55 1.75 Proj 3 UC 164 1.8 2.75
Proj 2 UC 132 2.8 1.75 Proj 3 UC 165 2.65 3.5
Proj 2 UC 133 2.55 2 Proj 3 UC 166 2.85 3.75
Proj 2 UC 134 2.35 1.75 Proj 3 UC 167 1.4 2.75
Proj 2 UC 135 2.55 2.2 Proj 3 UC 168 3 3.5
Proj 2 UC 136 2.3 1.7
94
F. Valdés-Souto
Presence Presence
(level, (level,
not num- not num-
ber) of ber) of
objects objects
of inter- of inter-
Use Case size est re- Use Case size est re-
Project (value assign- lated to Project (value assign- lated to
UC ID UC ID
ID ment – range the Use ID ment – range the Use
from 0 - 5) Case from 0 - 5) Case
(value (value
assign- assign-
ment – ment –
range range
from 0 - from 0 -
5) 5)
Proj 3 UC 169 3 2.65 Proj 4 UC 207 2.8 2.6
Proj 3 UC 170 3 3.5 Proj 4 UC 208 3.4 2.8
Proj 3 UC 171 2.95 3.25 Proj 4 UC 209 3 2.8
Proj 3 UC 172 3.25 3.5 Proj 4 UC 210 3.2 2.2
Proj 3 UC 173 3.25 3.25 Proj 4 UC 211 3.3 2.5
Proj 3 UC 174 2.95 4.25 Proj 4 UC 212 2.7 2.8
Proj 3 UC 175 3.05 3.5 Proj 4 UC 213 3.2 2.3
Proj 3 UC 176 2.8 2.35 Proj 4 UC 214 3.2 3
Proj 3 UC 177 2.3 3 Proj 4 UC 215 2.5 2.8
Proj 3 UC 178 2.9 2.35 Proj 4 UC 216 2.8 2.4
Proj 3 UC 179 2.2 2.65 Proj 4 UC 217 2.1 2
Proj 3 UC 180 2.55 2.35 Proj 4 UC 218 3 2.6
Proj 3 UC 181 2.3 2.3 Proj 4 UC 219 3.1 3
Proj 3 UC 182 2.35 2.15 Proj 4 UC 220 2.8 3
Proj 3 UC 183 2.45 2.2 Proj 4 UC 221 3.2 3.1
Proj 3 UC 184 2.35 2.5 Proj 4 UC 222 1.9 2.3
Proj 3 UC 185 2.15 2.05 Proj 4 UC 223 2.6 3
Proj 3 UC 186 2.5 3.85 Proj 4 UC 224 3 2.5
Proj 3 UC 187 2.7 4 Proj 4 UC 225 3.3 2.4
Proj 3 UC 188 2.1 3 Proj 4 UC 226 3 3.1
Proj 3 UC 189 2.55 3.25 Proj 4 UC 227 2.5 2.5
Proj 3 UC 190 2.8 3 Proj 4 UC 228 3.5 2.5
Proj 3 UC 191 2.6 3.5 Proj 4 UC 229 2.5 2.5
Proj 3 UC 192 3.55 3.85 Proj 4 UC 230 3 3
Proj 3 UC 193 3.05 3.45 Proj 4 UC 231 3 2.5
Proj 3 UC 194 2.8 3.35 Proj 4 UC 232 3.2 2.5
Proj 4 UC 195 3.5 3.3 Proj 4 UC 233 3.3 3
Proj 4 UC 196 3.4 3.4 Proj 4 UC 234 3.2 3
Proj 4 UC 197 3 2.8 Proj 4 UC 235 3.4 3.1
Proj 4 UC 198 3 3.1 Proj 4 UC 236 3.2 2.5
Proj 4 UC 199 3.3 3.4 Proj 4 UC 237 2.5 3
Proj 4 UC 200 3.3 3.5 Proj 4 UC 238 3.9 3
Proj 4 UC 201 3.6 3 Proj 4 UC 239 3.5 2.9
Proj 4 UC 202 3.1 2.5
Proj 4 UC 240 3.2 3.4
Proj 4 UC 203 3.4 3
Proj 4 UC 204 3.3 3 Proj 4 UC 241 2.7 2.7
Proj 4 UC 205 3.3 3 Proj 4 UC 242 3.5 3.1
Proj 4 UC 206 2.6 2.5 Proj 4 UC 243 3 2.3
95
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Presence Presence
(level, (level,
not num- not num-
ber) of ber) of
objects objects
of inter- of inter-
Use Case size est re- Use Case size est re-
Project (value assign- lated to Project (value assign- lated to
UC ID UC ID
ID ment – range the Use ID ment – range the Use
from 0 - 5) Case from 0 - 5) Case
(value (value
assign- assign-
ment – ment –
range range
from 0 - from 0 -
5) 5)
Proj 4 UC 244 3 2.5 Proj 4 UC 275 2.7 2.7
Proj 4 UC 245 2.5 2.5 Proj 4 UC 276 2 2.5
Proj 4 UC 246 2.7 2.4 Proj 4 UC 277 3 2.6
Proj 4 UC 247 3.3 3 Proj 4 UC 278 3.1 2.6
Proj 4 UC 248 2.7 2 Proj 4 UC 279 3.2 1.8
Proj 4 UC 249 3.8 3.5 Proj 4 UC 280 3 3
Proj 4 UC 250 3.3 3.1 Proj 4 UC 281 3.2 2.8
Proj 4 UC 251 3.1 2.5
Proj 4 UC 252 2.8 2.8 Proj 4 UC 282 3 2.5
Proj 4 UC 253 3.6 3.5
Proj 4 UC 254 2.5 2.6 Proj 4 UC 283 2 2
Proj 4 UC 255 2.9 2.6
Proj 4 UC 256 3.3 2.5 Proj 4 UC 284 3.3 2.4
Proj 4 UC 257 2.6 2
Proj 4 UC 258 3.2 2.4 Proj 4 UC 285 3 2.9
Proj 4 UC 259 2.5 2.3 Proj 4 UC 286 3 2.3
Proj 4 UC 260 2.5 2.2 Proj 4 UC 287 2.5 1.8
Proj 4 UC 261 3 2.9 Proj 4 UC 288 3 2.5
Proj 4 UC 262 2.7 3 Proj 4 UC 289 2.5 2
Proj 4 UC 263 2.7 2.8 Proj 4 UC 290 3.3 2.6
Proj 4 UC 264 3.2 3 Proj 4 UC 291 3.9 2
Proj 4 UC 265 2.5 2.2 Proj 4 UC 292 2 2
Proj 4 UC 266 2.5 2.2
Proj 4 UC 267 3.8 3 Proj 4 UC 293 2.5 2.5
Proj 4 UC 268 3.4 2.8
Proj 4 UC 269 3.6 3
Proj 4 UC 270 3.6 3.5
Proj 4 UC 271 3.2 2.7
Proj 4 UC 272 2.4 2.9
Proj 4 UC 273 2.5 1.8
Proj 4 UC 274 2.7 2.7
96
F. Valdés-Souto
Appendix II. COSMIC
Functional Size and Pro-
UC RE EPCU1 EPC
Approximation ject ESB
ID AL 6.4 U44
ID
COSMIC functional size and approxi-
mation for each Functional Process are pre-
sented in Table A2 II where: 14.6
1 UC 4 22 7.7 9.84
- Column 1 presents the Project identifier. 0
For purposes of confidentiality , the Projects 13 26.7
1 UC 5 10.7 12.56
were labeled sequentially, from “Proj 1” to 2 8
“Proj 4, 27.5
1 UC 6 9 7.7 12.72
- Column 2 presents the Use Case identifier. 2
For purposes of confidentiality, the Use 1 UC 7 4 10.7 14.72
36.4
Cases were labeled sequentially, from “UC 9
1” to “UC 293, 34 26.7
1 UC 8 10.7 12.56
3 8
- Column 3 presents the functional size ob-
tained utilizing the standard COSMIC 26.7
1 UC 9 11 10.7 12.56
method – in CFP units, 8
UC 36.4
- Column 4 presents the Equal Size Bands ap- 1 8 10.7 14.72
10 9
proximation approach,
UC 26.7
- Column 5 presents the EPCU size approxi- 1 20 10.7 12.56
11 8
mation approach using an output variable UC 26.7
domain function from 2 - 16.4 CFP [9] [10], 1
12
8 10.7 12.56
8
and
UC 22.9
1 6 10.7 11.71
- Column 6 presents the EPCU size approxi- 13 6
mation approach using an output variable
domain function from 2 - 44 CFP [11]. 1 UC 14 154 10.7 9.84 14.60
1 UC 15 12 10.7 15.98 42.11
1 UC 16 9 7.7 16.40 44.00
1 UC 17 37 10.7 12.56 26.78
1 UC 18 75 7.7 12.72 27.52
1 UC 19 13 7.7 12.72 27.52
1 UC 20 9 10.7 14.72 36.49
Table A2: Functional size – Real and from 3
approximation techniques 1 UC 21 26 10.7 14.72 36.49
1 UC 22 132 10.7 14.72 36.49
Pro-
UC RE EPCU1 EPC 1 UC 23 14 10.7 15.09 38.12
ject ESB
ID AL 6.4 U44
ID
1 UC 24 9 10.7 15.09 38.12
1 UC 25 37 10.7 14.72 36.49
14.6
1 UC 1 9 7.7 9.84 1 UC 26 49 10.7 15.09 38.12
0
27.5
1 UC 2 9 7.7 12.72 1 UC 27 11 10.7 14.72 36.49
2
27.5 1 UC 28 9 10.7 12.46 26.36
1 UC 3 9 7.7 12.72
2
1 UC 29 22 10.7 12.56 26.78
97
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Pro- Pro-
UC RE EPCU1 EPC UC RE EPCU1 EPC
ject ESB ject ESB
ID AL 6.4 U44 ID AL 6.4 U44
ID ID
1 UC 30 16 10.7 12.56 26.78 2 UC 68 17 7.7 8.92 12.97
1 UC 31 8 10.7 12.56 26.78 2 UC 69 9 7.7 8.77 12.36
1 UC 32 8 10.7 15.09 38.12 2 UC 70 19 7.7 8.77 12.36
2 UC 71 10 7.7 10.56 17.81
1 UC 33 21 10.7 12.46 26.36
2 UC 72 12 7.7 6.95 8.53
1 UC 34 19 10.7 12.46 26.36 2 UC 73 19 7.7 10.08 16.26
2 UC 74 17 4.8 6.02 7.74
1 UC 35 32 10.7 15.09 38.12
2 UC 75 9 7.7 7.01 9.60
1 UC 36 5 10.7 15.09 38.12 2 UC 76 12 7.7 7.55 10.36
1 UC 37 46 10.7 16.40 44.00 2 UC 77 20 4.8 5.49 6.98
1 UC 38 17 10.7 15.09 38.12 2 UC 78 13 7.7 8.16 11.64
1 UC 39 11 10.7 15.09 38.12 2 UC 79 15 7.7 10.36 16.95
1 UC 40 15 10.7 12.46 26.36 2 UC 80 22 7.7 8.48 12.05
1 UC 41 15 10.7 16.40 44.00 2 UC 81 37 7.7 10.66 18.25
1 UC 42 16 10.7 16.40 44.00 2 UC 82 19 7.7 8.61 12.01
1 UC 43 10 10.7 15.09 38.12 2 UC 83 14 7.7 8.00 10.74
2 UC 44 4 7.7 7.53 10.05 2 UC 84 9 7.7 6.83 9.24
2 UC 45 27 7.7 8.79 12.39 2 UC 85 7 7.7 8.79 12.39
2 UC 46 31 7.7 9.41 13.70 2 UC 86 5 10.7 7.74 10.18
2 UC 47 39 7.7 7.97 11.24 2 UC 87 14 7.7 7.04 9.03
2 UC 48 37 7.7 10.73 18.59 2 UC 88 8 7.7 12.72 27.52
2 UC 49 8 7.7 6.65 8.86 2 UC 89 31 7.7 6.83 9.24
2 UC 50 6 7.7 10.56 17.81 2 UC 90 21 4.8 6.56 8.80
2 UC 51 20 7.7 9.84 14.60 2 UC 91 11 4.8 6.56 8.80
2 UC 52 47 7.7 11.05 20.60 2 UC 92 4 4.8 5.21 6.63
2 UC 53 20 4.8 5.50 7.09 2 UC 93 27 4.8 7.16 9.76
2 UC 54 12 7.7 11.80 23.38 2 UC 94 31 7.7 7.55 10.36
2 UC 55 23 7.7 10.70 18.45 2 UC 95 39 7.7 9.05 12.94
2 UC 56 32 7.7 7.76 10.81 2 UC 96 37 7.7 9.32 13.50
2 UC 57 32 7.7 10.56 17.81 2 UC 97 8 7.7 10.56 17.81
2 UC 58 11 4.8 6.56 8.80 2 UC 98 6 7.7 10.36 16.95
2 UC 59 35 7.7 7.76 10.81 2 UC 99 20 7.7 9.05 12.94
2 UC 60 59 7.7 8.73 12.26 UC
2 47 10.7 8.80 12.42
2 UC 61 23 7.7 7.01 9.60 100
2 UC 62 39 10.7 11.43 21.72 UC
2 20 7.7 6.76 8.93
2 UC 63 52 7.7 8.79 12.39 101
2 UC 64 9 7.7 9.27 13.39 UC
2 23 7.7 7.76 10.81
102
2 UC 65 9 7.7 9.84 14.60
UC
2 UC 66 6 7.7 8.71 12.53 2 32 7.7 7.76 10.81
103
2 UC 67 8 7.7 8.25 11.56
98
F. Valdés-Souto
Pro- Pro-
UC RE EPCU1 EPC UC RE EPCU1 EPC
ject ESB ject ESB
ID AL 6.4 U44 ID AL 6.4 U44
ID ID
UC UC
2 32 7.7 8.26 11.28 2 37 7.7 8.00 10.74
104 128
UC UC
2 35 7.7 9.15 13.16 2 9 7.7 6.47 8.48
105 129
UC UC
2 59 7.7 8.04 10.81 2 14 4.8 5.75 7.48
106 130
UC UC
2 23 7.7 9.43 13.74 2 7 7.7 8.01 10.75
107 131
UC UC
2 39 7.7 10.61 18.04 2 5 7.7 8.04 10.81
108 132
UC UC
2 52 7.7 10.70 18.45 2 8 7.7 8.52 11.82
109 133
UC UC
2 9 7.7 9.08 13.00 2 14 7.7 7.53 10.05
110 134
UC UC
2 9 7.7 10.70 18.45 2 31 7.7 9.02 12.88
111 135
UC UC
2 6 7.7 10.66 18.25 2 21 7.7 6.89 8.98
112 136
UC UC
2 8 7.7 8.48 12.05 2 11 7.7 8.01 11.06
113 137
UC UC
2 17 7.7 7.74 10.18 2 12 10.7 8.07 10.88
114 138
UC UC
2 9 7.7 6.28 8.09 2 15 7.7 8.77 12.36
115 139
UC UC
2 19 7.7 9.84 14.60 3 121 10.7 16.40 44.00
116 140
UC UC
2 10 7.7 7.29 9.54 3 8 10.7 12.56 26.78
117 141
UC UC
2 12 7.7 9.02 12.88 3 15 10.7 9.84 14.60
118 142
UC UC
2 19 4.8 5.75 7.48 3 5 10.7 15.17 38.48
119 143
UC UC
2 17 7.7 7.11 9.45 3 124 10.7 14.10 33.70
120 144
UC UC
2 9 7.7 6.47 8.48 3 13 7.7 11.27 22.46
121 145
UC UC
2 12 7.7 8.04 10.81 3 132 10.7 16.40 44.00
122 146
UC UC
2 20 7.7 8.79 12.39 3 153 10.7 15.98 42.11
123 147
UC UC
2 13 7.7 9.11 13.07 3 22 7.7 13.01 28.82
124 148
UC UC
2 15 10.7 11.43 21.72 3 22 10.7 14.10 33.70
125 149
UC UC
2 19 7.7 8.02 10.77 3 22 7.7 16.40 44.00
126 150
UC UC
2 22 7.7 8.02 10.77 3 6 10.7 12.74 27.61
127 151
99
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Pro- Pro-
UC RE EPCU1 EPC UC RE EPCU1 EPC
ject ESB ject ESB
ID AL 6.4 U44 ID AL 6.4 U44
ID ID
UC UC
3 103 7.7 16.40 44.00 3 23 7.7 16.40 44.00
152 174
UC UC
3 29 7.7 12.48 26.44 3 30 10.7 14.53 35.60
153 175
UC UC
3 17 7.7 9.84 14.60 3 39 7.7 9.43 13.74
154 176
UC UC
3 13 7.7 16.40 44.00 3 16 7.7 11.62 23.62
155 177
UC UC
3 20 7.7 10.73 19.66 3 19 7.7 9.41 13.70
156 178
UC UC
3 13 7.7 10.73 19.66 3 17 7.7 10.06 17.19
157 179
UC UC
3 15 7.7 12.72 27.52 3 5 7.7 9.48 13.85
158 180
UC UC
3 11 7.7 14.04 33.42 3 2 7.70 8.16 11.64
159 181
UC UC
3 15 4.8 9.31 16.00 3 5 7.70 8.48 12.05
160 182
UC UC
3 10 7.7 12.06 24.56 3 82 7.70 9.05 12.94
161 183
UC UC
3 54 7.7 9.84 14.60 3 62 7.70 9.58 14.05
162 184
UC UC
3 9 7.7 13.95 33.01 3 29 7.70 7.36 10.15
163 185
UC UC
3 40 4.8 9.64 17.05 3 40 7.70 16.40 44.00
164 186
UC UC
3 35 7.7 14.83 36.98 3 41 7.70 16.40 44.00
165 187
UC UC
3 23 7.7 16.40 44.00 3 21 7.70 10.96 21.42
166 188
UC UC
3 15 4.8 8.35 13.11 3 14 7.70 14.01 33.31
167 189
UC UC
3 47 7.7 14.55 35.72 3 36 7.70 12.59 26.93
168 190
UC UC
3 11 7.7 11.41 21.62 3 36 7.70 15.01 37.79
169 191
UC UC
3 26 7.7 14.55 35.72 3 12 10.70 16.40 44.00
170 192
UC UC
3 31 7.7 13.69 31.83 3 55 10.70 14.53 35.60
171 193
UC UC
3 15 10.7 14.58 35.82 3 18 7.70 14.25 34.39
172 194
UC UC
3 29 10.7 13.71 31.96 4 37 10.70 13.90 32.81
173 195
100
F. Valdés-Souto
Pro- Pro-
UC RE EPCU1 EPC UC RE EPCU1 EPC
ject ESB ject ESB
ID AL 6.4 U44 ID AL 6.4 U44
ID ID
UC UC
4 33 10.70 14.18 34.04 4 78 7.70 12.59 26.93
196 220
UC UC
4 9 7.70 11.94 24.01 4 35 10.70 13.05 28.97
197 221
UC UC
4 31 7.70 13.04 28.94 4 17 4.80 7.33 9.91
198 222
UC UC
4 32 10.70 14.10 33.70 4 83 7.70 12.48 26.44
199 223
UC UC
4 49 10.70 14.58 35.82 4 12 7.70 9.84 14.60
200 224
UC UC
4 64 10.70 12.50 26.53 4 5 10.70 9.40 13.68
201 225
UC UC
4 10 10.70 9.84 14.60 4 8 7.70 13.04 28.94
202 226
UC UC
4 18 10.70 12.62 27.08 4 6 7.70 9.84 14.60
203 227
UC UC
4 15 10.70 12.69 27.38 4 91 10.70 9.84 14.60
204 228
UC UC
4 31 10.70 12.69 27.38 4 26 7.70 9.84 14.60
205 229
UC UC
4 20 7.70 9.84 14.60 4 22 7.70 12.72 27.52
206 230
UC UC
4 13 7.70 10.66 18.25 4 158 7.70 9.84 14.60
207 231
UC UC
4 42 10.70 11.84 23.57 4 7 10.70 9.84 14.60
208 232
UC UC
4 64 7.70 11.94 24.01 4 21 10.70 12.69 27.38
209 233
UC UC
4 50 10.70 8.78 12.37 4 3 10.70 12.74 27.61
210 234
UC UC
4 24 10.70 9.84 14.60 4 15 10.70 13.02 28.85
211 235
UC UC
4 17 7.70 11.62 22.56 4 6 10.70 9.84 14.60
212 236
UC UC
4 7 10.70 9.05 12.94 4 4 7.70 12.46 26.36
213 237
UC UC
4 3 10.70 12.74 27.61 4 24 10.70 12.46 26.36
214 238
UC UC
4 3 7.70 11.41 21.66 4 5 10.70 12.12 24.81
215 239
UC UC
4 6 7.70 9.43 13.74 4 13 10.70 14.06 33.49
216 240
UC UC
4 7 7.70 6.83 9.24 4 11 7.70 11.21 20.75
217 241
UC UC
4 2 7.70 10.73 18.59 4 11 10.70 13.01 28.79
218 242
UC UC
4 23 10.70 12.75 27.66 4 4 7.70 9.06 12.95
219 243
101
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Pro- Pro-
UC RE EPCU1 EPC UC RE EPCU1 EPC
ject ESB ject ESB
ID AL 6.4 U44 ID AL 6.4 U44
ID ID
UC UC
4 39 7.70 9.84 14.60 4 4 7.70 9.05 12.94
244 266
UC UC
4 18 7.70 9.84 14.60 4 14 10.70 12.46 26.36
245 267
UC UC
4 26 7.70 9.46 13.79 4 13 10.70 11.84 23.57
246 268
UC UC
4 8 10.70 12.69 27.38 4 16 10.70 12.50 26.53
247 269
UC UC
4 8 7.70 8.50 11.77 4 14 10.70 14.93 37.42
248 270
UC UC
4 23 10.70 15.09 38.12 4 16 10.70 11.42 21.69
249 271
UC UC
4 4 10.70 13.03 28.91 4 15 7.70 11.53 22.76
250 272
UC UC
4 16 10.70 9.84 14.60 4 16 7.70 8.00 10.74
251 273
UC UC
4 32 7.70 11.80 23.38 4 6 7.70 11.21 20.75
252 274
UC UC
4 45 10.70 14.93 37.42 4 5 7.70 11.21 20.75
253 275
UC UC
4 7 7.70 10.36 16.95 4 6 4.80 8.53 11.84
254 276
UC UC
4 21 7.70 10.70 18.45 4 3 7.70 10.73 18.59
255 277
UC UC
4 27 10.70 9.84 14.60 4 3 10.70 10.74 18.65
256 278
UC UC
4 15 7.70 8.52 11.82 4 28 10.70 8.09 10.92
257 279
UC UC
4 14 10.70 9.39 13.65 4 6 7.70 12.72 27.52
258 280
UC UC
4 6 7.70 9.32 13.50 4 6 10.70 11.96 24.09
259 281
UC
4 14 7.70 9.05 12.94 UC
260 4 9 7.70 9.84 14.60
UC 282
4 3 7.70 12.37 25.94
261
UC
UC 4 3 4.80 6.56 8.80
4 7 7.70 12.53 26.64 283
262
UC UC
4 3 7.70 11.62 22.56 4 19 10.70 9.40 13.68
263 284
UC
4 15 10.70 12.74 27.61 UC
264 4 7 7.70 12.37 25.94
285
UC
4 2 7.70 9.05 12.94
265
102
F. Valdés-Souto
Pro-
UC RE EPCU1 EPC
ject ESB
ID AL 6.4 U44
ID
UC
4 6 7.70 9.06 12.95
286
UC
4 22 7.70 8.00 10.74
287
UC
4 3 7.70 9.84 14.60
288
UC
4 60 7.70 8.53 11.84
289
UC
4 16 10.70 10.72 18.53
290
UC
4 2 10.70 8.53 11.84
291
UC
4 108 4.80 6.56 8.80
292
UC
4 4 7.70 9.84 14.60
293
103
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Appendix III. Friedman Test Results from SPSS®
Descriptive Statistics
Std. Devia- Maxi-
N Mean tion Minimun mun
REAL 293 24.7406 31.84897 0.00 343.00
ESB 293 8.5556 1.68440 4.80 10.70
EPCU16 293 10.8944 2.76991 5.21 16.40
EPCU44 293 21.0759 10.40606 6.63 44.00
Ranks
Mean Rank
REAL 2.89
ESB 1.35
EPCU16 2.20
EPCU44 3.55
Tests Statisticsa
N 293
Chi-Square 468.936
df 3
Asymp. Sig. 2.57215388100136E-
101
104
F. Valdés-Souto
Appendix IV. Wilcoxon Test Results from SPSS®
ESB – REAL
N Mean Rank Sum of Ranks
ESB - REAL Negative Ranks 230a 165.49 38063.00
Positive Ranks 63b 79.49 5008.00
Ties 0c
Total 293
a. ESB < REAL
b. ESB > REAL
c. ESB = REAL
Test Statisticsa
ESB - REAL
Z -11.388b
Asymp.
4.8089036753386E-
Sig. (2-tai-
30
led)
a. Wilcoxon Test with sign
b. Based in negative ranks.
EPCU16 - REAL
Ranks
N Mean Rank Sum of Ranks
EPCU16 - REAL Negative Ranks 194 a
177.95 34523.00
Positive Ranks 99 b
86.34 8548.00
Ties 0c
Total 293
a. EPCU16 < REAL
105
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
b. EPCU16 > REAL
c. EPCU16 = REAL
Test Statisticsa
EPCU16 - REAL
Z -8.948b
Asymp. Sig.
3.6339E-19
(2-tailed)
a. Wilcoxon Test with sign
b. Based in positive ranks.
EPCU44 – REAL
Ranks
N Mean Rank Sum of Ranks
EPCU44 - REAL Negative Ranks a
130 153.62 19971.00
Positive Ranks b
163 141.72 23100.00
Ties 0 c
Total 293
a. EPCU44 < REAL
b. EPCU44 > REAL
c. EPCU44 = REAL
Test Statisticsa
EPCU44 - REAL
Z -1.078b
Asymp. Sig. (2-
.281
tailed)
a. Wilcoxon Test with sign
b. Based in positive ranks.
106
F. Valdés-Souto
EPCU44 – ESB
Ranks
N Mean Rank Sum of Ranks
EPCU44 - ESB Negative Ranks 1a
4.00 4.00
Positive Ranks 292b
147.49 43067.00
Ties 0 c
Total 293
a. EPCU44 < ESB
b. EPCU44 > ESB
c. EPCU44 = ESB
Tests Statisticsa
MRE_EPCU44 -
ESB
Z -14.835b
Asymp. Sig. (2-tailed) 8.7473E-50
a. Wilcoxon Test with sign
b. Based in positive ranks.
EPCU16 – ESB
Ranks
N Mean Rank Sum of Ranks
EPCU16 - ESB Negative Ranks 39a 68.58 2674.50
Positive Ranks 254b 159.04 40396.50
Ties 0 c
Total 293
a. EPCU16 < ESB
b. EPCU16 > ESB
c. EPCU16 = ESB
107
IWSM/Mensura’18, September 18–20, 2018, Beijing, China
Test Statisticsa
MRE_EPCU16 -
ESB
Z -12.995b
Asymp. Sig. (2-tai-
1.3091E-38
led)
a. Wilcoxon Test with sign
b. Based in positive ranks.
EPCU16 - EPCU44
Ranks
N Mean Rank Sum of Ranks
EPCU16 - Negative Ranks 0a 0.00 0.00
EPCU44
Positive Ranks 293b 147.00 43071.00
Ties 0c
Total 293
a. EPCU16 < EPCU44
b. EPCU16 > EPCU44
c. EPCU16 = EPCU44
Test Statisticsa
MRE_EPCU16 -
MRE_EPCU44
Z -14.839b
Asymp. Sig. (2-tai-
8.2436E-50
led)
108