The escalating demands on the development of software products require software organizations to produce mature software processes that are capable of providing the required levels of quality and productivity. The implementation of statistical process control (SPC) in performance process analysis uses data collected during the course of the project to analyze the behavior of organization processes, identifying actions that are needed for the stabilization and improvement of those processes. An essential element for the SPC application is the suitability of the measures being used. This paper presents the approach proposed in a doctorate thesis to support organizations obtain and maintain measurement repository suitable for SPC, as well as to perform measurements appropriate in this context. The approach is composed by an Instrument for Evaluating the Suitability of a Measurement Repository to SPC, a Software Process Measurement Ontology and a Body of Recommendations for Software Measurement.
The requirements of the industry of software products and services have increased the
organizations’ interest in process improvement. There are several frameworks that
support definition and institutionalization of programs of this nature, in which
measurement plays an important role, such as CMMI and ISO/IEC 15504. While the
process improvement evolves in an organization, the organizational maturity level has
a tendency to increase too. In high maturity1, process improvement based on
traditional measurement is not enough. It is necessary to carry out the statistical
process control (SPC) to get to know the behavior of the process, to determine its
1 High maturity is characterized by the highest levels of maturity models such as CMMI levels
4 and 5.
performance in previous executions and so predicting its performance in current and
future projects, checking if they will be able to achieve the established objectives.
Using SPC in this context is only possible for organizations that carry out
measurement appropriately. However, this is one of the biggest difficulties for
organizations that desire to achieve high maturity [
In this paper we present briefly the strategy proposed in the thesis and, as soon as the focus of discussion here is the SPMO, this is discussed in more details (Section 2). In section 3 we present the expected contributions and the proposal for evaluation of the results, specifically in the context of the SPMO.
Software organizations that want to perform Statistical Process Control (SPC) usually are in one of the following situations: (i) it achieved the initial maturity levels and it has measurement repository with data collected along fulfilled projects; or (ii) it is starting a process improvement program and wants to initially define a measurement repository and to perform measurement in a way that is appropriate to SPC. The strategy proposed in this thesis must contemplate both situations and, for that, it has three main components: (a) Instrument for Evaluating the Suitability of a Measurement Repository to SPC (IESMB): the goal of this component is to evaluate existing measurement repository and to determine its suitability to SPC, identifying corrective actions that can be done to make the measurement repository suitable to SPC, when necessary. (b) Software Process Measurement Ontology (SPMO): the goal of this component is to capture the conceptualization involved in the software measurement domain, including traditional and high maturity aspects. (c) Body of Recommendations for Software Measurement (BRSM): its goal is to supply guidelines on how to perform measurement suitable for SPC.
According to the proposed strategy, organizations that achieved initial maturity levels can use the IESMB in order to evaluate and adapt, when possible, their measurement repository to SPC. On the other hand, organizations that are starting a measurement program and look for high maturity can use BRSM and SPMO to define their measurement repository and measurement program in a suitable way for SPC. Besides, it is important to note that performing SPC requires frequent data collection for measures and sometimes the definition of new measures. Then, when new data are collected for measures and/or new measures are defined, the measurement repository suitability must be reevaluated (using IESMB) and new measures must be defined and collected appropriately (using BRSM and SPMO).
Considering related works, in context of measures evaluation for SPC, Tarhan and
Demirors [
Domain ontologies can be used to define a common vocabulary for sharing and
reusing of knowledge about some domain. As defended by Guarino [
The SPMO is based on terminologies used in the literature and on specific
requirements of software process measurement in high maturity organizations that
were identified during studies based on systematic reviews and in some practical
experiences. The building of the SPMO follows the process defined by SABiO
(Systematic Approach for Building Ontologies) [
In the development of the SPMO, the requirement specification involved the
definition of competency questions (CQs), such as the following ones: Which
indicators can be used to analyze an objective? What are the information needs
considered by an indicator? Based on the CQs, concepts were captured and grounded
on UFO, being represented in models UML, textual descriptions and first-order logic
axioms. The SPMO was integrated with a Software Process Ontology [
A Measurable Entity Type is a universal high order in UFO, meaning that its instances are universals (e.g., Organization, Process and Project). A measurable entity type indicates which measurable elements can be used to measure entities of this type. A Measurable Element is a property (quality universal in UFO) of a measurable entity type through which Measurable Entities of this type can be described. It can be directly (e.g., cost) or indirectly measurable (e.g., productivity). Measurable elements are quantified by measures. A Measure is a measurement instrument that is used to associate a value to a measurable element. When a measure quantifies a directly measurable element, it is said a Base Measure (e.g., number of defects). When a measure quantifies an indirectly measurable element it is said a Derived Measure (e. g., defects density). Organizations carried out measurement to attend their Objectives and/or Information Needs. An organization with the goal of “reducing operational costs in 10%” can, for example, take as information need “knowing costs with rework in projects”. Measures are used to attend information needs and when a measure is used to indicate the achievement of an objective, it plays the role2 of Indicator.
During the development of the SPMO, several restrictions were identified and, since the models did not capture several of them, we defined axioms to make them explicit. For instance, the following axiom holds: if an indicator ind considers an information need inf and ind quantifies the measurable element mel then the information need inf is quantified by the measurable element mel.
(∀ ind ∈ Indicator, inf ∈ Information Needs, mel ∈ Measurement Element)
(considers(ind,inf) ∧ quantifies(ind ,mel) → isQuantifiedBy(inf, mel))
2 According to UFO, it is a possible role that a substance sortal can play along its history. An
entity plays a role in a certain context, demarcated by its relations with other entities. Sortal
universals are entities that describe a standard of characteristics that can be carried out in a
number different from individuals and, besides, provide a principle of individualization,
persistence and identity.
Once that SPMO treats high maturity software measurement aspects and it is
grounded on a foundational ontology (characteristics that we did not found in others
proposals) we believe that the SPMO is a contribution itself. Besides, the works
carried out until now produced other contributions, like the reengineering of a
Software Enterprise Ontology at the light of UFO, for integration to the SPMO [
The main expected result related to the SPMO is that its conceptualization supports understanding and performing software process measurement in traditional and high maturity contexts. To evaluate this, as part of this thesis, SPMO is being used as basis for building BRSM (described in Section 2). Besides, the SPMO is being used together with the BRSM for building a measurement repository suitable for high maturity, which will be used at LENS (Software Engineering Laboratory) in COPPE/UFRJ. Both experiences will supply information about the usefulness of the SPMO. In the near future, it will also be possible to evaluate how SPMO supports interoperability between tools that will be developed in High Maturity Environment at LENS, context where this thesis is being developed.