=Paper=
{{Paper
|id=Vol-1293/paper14
|storemode=property
|title=Process Mining Extension to SCAMPI
|pdfUrl=https://ceur-ws.org/Vol-1293/paper14.pdf
|volume=Vol-1293
|dblpUrl=https://dblp.org/rec/conf/simpda/Valle14
}}
==Process Mining Extension to SCAMPI==
https://ceur-ws.org/Vol-1293/paper14.pdf
Process Mining Extension to SCAMPI
Arthur Valle1,2, Eduardo Rocha Loures1, Eduardo Portela1
1
Pontifical Catholic University of Parana, Industrial and Systems Engineering, Curitiba, Brazil
{arthur.maria, eduardo.loures, eduardo.portela}@pucpr.br
2
Wipro Technologies, Curitiba, Brazil
arthur.valle@wipro.com
Abstract. Existing process assessment methods, such as SCAMPI-Standard
CMMI Appraisal Method for Process Improvement, do not use contemporary
data collection and analysis techniques like processes mining, text mining or
data mining. On the contrary, they use traditional ones: questionnaires, docu-
ment review, interviews and demonstrations. Process mining is a technique that
can be used to aid process assessments, aiming to conduct them with greater
deepness and coverage, while keeping similar level of effort. The purpose of the
PhD work is to develop a framework (structure and content) to apply process
mining techniques in SCAMPI assessments.
1 Introduction
The present paper proposes the Process Mining Extension to SCAMPI, a frame-
work where process mining techniques are added to existing assessment techniques.
The paper is organized as follows. Section 2 presents the research question. Section 3
describes the background. Section 4 presents the significance of the work. Section 5
presents the research design and method. Section 6 provides the research stage.
2 Research Question
The research question can be stated as: “Comparatively with traditional SCAMPI
assessments, does the proposed method extension enable software process assess-
ments with more objectivity, accuracy, depth and coverage of aspects related to the
execution of processes, while maintaining similar levels of effort, cost and time?”.
3 Background
SCAMPI [1] is a method used to assess organizations that use CMMI as a refer-
ence for their operations (software development, service management, etc). The
fundamental idea behind the SCAMPI, as well as other similar assessments, is that the
conduction of an activity or process results in "footprints" called objective evidences,
adfa, p. 1, 2011.
© Springer-Verlag Berlin Heidelberg 2011
which are evaluated by experts to judge whether they satisfy best practices of a given
CMMI reference model [2].
According to Fig. 1, extracted from Process Mining Manifesto, Process Mining is
a set of techniques, tools, and methods to discover, monitor and improve real process-
es (i.e., not assumed processes) by extracting knowledge from event logs commonly
available in today's (information) systems [3].
Fig. 1. Process Mining Overview, reproduced from [3]
There are three main types of process mining techniques [3]: a) Process Discovery
(from an event log, a “as is” process model is identified); b) Conformance (or Com-
pliance) Checking (an existing process model is compared to an event log of the same
process); c) Enhancement (a process model is improved using information extracted
from a log).
In order to identify work that proposed the use of process mining in process as-
sessment, a systematic literature review was conducted. As a result of the application
of a defined criterion and procedure, in six renowned scientific databases, only 6 out
of 26 resulting papers were selected. Since none of them mentioned which process
assessment methods were used, an additional search on Google Scholar was conduct-
ed using the same terms, resulting in some relevant papers, as follows:
• PhD thesis of Samalik, entitled Process Mining Application in Software Pro-
cess Assessment [4]. The objective was to promote the use of process mining in soft-
ware process assessment and improvement. Her conclusion was that techniques for
collecting information derived from process mining can be applied to improve the
data collection on software process assessment. However, conclusion was reached by
qualitative judgment without objective criteria. In addition, process mining techniques
that should be used were not nominally listed.
• Master dissertation of Cruañas, entitled Process Mining Opportunities for
CMMI Assessments [5]. The objective was to investigate the literature concerning
support tools to find out if it is possible to use process mining to improve the assess-
ment of CMMI. His conclusion was that process mining can not only help improve
the current CMMI assessments, but can also be a useful tool to assist data collection.
However, conclusions were based on the generalization of processes mining tech-
niques and perspectives without using objective criteria. Moreover, no process mining
technique in particular was pointed out.
• A third paper found is [6] where different aspects of processes mining were
addressed, such as control perspective, information perspective and organizational
perspective. Some algorithms such as alpha algorithm, heuristics miner, genetic min-
er, social network miner, organizational miner and activity miner, which can be ap-
plied, were cited in the paper.
Although these papers demonstrate application of process mining in process as-
sessments, there is no formal guidance of how to conducting it, covering for instance,
how to capture business rules, how to compare models and logs, which process min-
ing algorithms to use, and when, etc.
4 Significance
According to [7], existing process assessment methods (such as SCAMPI) have
limitations: they are manual, time-consuming, inefficient, subjective and generally
require experienced appraisers. However, these days, detailed information about pro-
cesses is recorded in the form of event logs, transaction logs, databases, etc. In this
sense, in a process assessment is no longer justifiable that only a small set of process-
es are checked. Instead, the entire process and all its instances should be considered,
as long as this represents low costs, naturally. Additionally, in existing assessment
methods, the following techniques are used for gathering information about the run-
ning processes in an organization: questionnaires, document review, interviews and
demonstrations. It means that no contemporary data collection and analysis tech-
niques such as data mining, text mining or process mining is used.
Therefore, it is proposed the application of process mining techniques on the
SCAMPI. It means that event logs of software processes would be used to understand
past and current situation in a complete, economical, reliable and accurate manner,
thereby contributing to the collection and analysis of data, which are critical activities
in any software process assessment method.
The premise is that nowadays companies have been extremely efficient in collect-
ing, organizing, and storing a large amount of data obtained in their daily operations.
Most of these companies, however, do not use such data properly so as to transform
them into knowledge to be employed in assessment activities. The need of companies
to learn more about how their processes actually operate is a major driver behind the
development and increasing use of process-mining techniques.
The main objective of this work is to develop a framework for the application of
process mining techniques in SCAMPI-based assessments. This framework aims at
enable software process assessments with more objectivity, accuracy, depth and cov-
erage of aspects related to the execution of processes (such as duration and sequence
of activities, start and end dates and records of who were the real executors), while
maintaining similar levels of effort, cost and time.
5 Research design and methods
According to Fig 2, the proposed framework is a structure that serves as a guide
for applying process mining techniques in SCAMPI assessments. The intention is that
such guidance could be seen as an extension to SCAMPI method description. Analog
approaches already exist such as the SAFE extension to CMMI-DEV [8]. It means
that the Process Mining Extension to SCAMPI adds (or modifies) content to the cur-
rent SCAMPI method. Its content covers the application of process mining aspects in
SCAMPI method. Typically it comprises full or partial processes or activities, alt-
hough any element can be added or expanded, such as inputs, outputs, tools and tech-
niques.
Fig. 2. Process Mining Extension to SCAMPI
Process Mining Extension to SCAMPI is a document with the following chapters:
Executive Summary; Abstract; 1-Introduction; 1.1- Background and Acknowledge-
ments; 1.2-Purpose and Scope; 1.3-Relationships with CMMI and SCAMPI; 1.4-
Structure of the Process Mining Extension to SCAMPI; 1.5-Intended Audiences; 1.6-
Usage scenarios; 2-Content; Appendix A: References; Appendix B: Acronyms; Ap-
pendix C: Glossary; Appendix D: Contact.
In order to define which content would be added or modified in Process Mining
Extension to CMMI, some references were considered, beyond the SAFE extension to
CMMI-DEV. For instance, [9] have proposed process mining use cases – typical ap-
plications of process mining functionalities in practical situations – to be used for
detailed evaluation of process mining tools. Here, these use cases are used to identify
typical process mining situations that are pertinent and could be applied in SCAMPI
assessment. From the original list of 19 use cases, the following ones were taken in
consideration: from Discovery perspective, UC1-Structure of the process; from Con-
formance Checking perspective, UC6-The degree in which the rules are obeyed and
UC7-Compliance to the explicit model.
6 Research stage
The Process Mining Extension to SCAMPI is under development. Some chapters
of the document are more advanced than others, such as 1.4-Structure of the Process
Mining Extension to SCAMPI, 1.6-Usage scenarios and 2-Content.
In addition, the implementation of a running example to apply process mining
techniques in SCAMPI assessments was conducted. The approach seemed to be fea-
sible, as demonstrated using Disco and ProM process mining tools. Process mining
techniques are demanded in order to transform existing process assessment methods,
such as SCAMPI, into more productive and economically viable methods. Process
mining enables an easy comparison on how processes are performed in practice ver-
sus the way they are designed to operate.
As future work, it is intended to use the framework in real SCAMPI assessments
and to conduct statistical analysis and hypothesis thesis of performance parameters
such as effort, duration, coverage and quality of results to quantitatively evaluate the
benefits.
References
1. SCAMPI Upgrade Team.: Standard CMMI Appraisal Method for Process Improvement
(SCAMPI), Version 1.3a: Method Definition Document for SCAMPI A, B and C
(CMU/SEI-2013-HB-001). Pittsburgh, PA: Software Engineering Institute, Carnegie
Mellon University. http://cmmiinstitute.com/resource/standard-cmmi-appraisal-method-
process-improvement-scampi-b-c-version-1-3a-method-definition-document/
2. Chrissis, M. B., Konrad, M., & Shrum, S.: CMMI for Development: Guidelines for Pro-
cess Integration and Product Improvement. Addison-Wesley Professional (2011)
3. Van der Aalst, W. et al.: Process mining manifesto. In: Business process management
workshops p. 169-194. Springer Berlin Heidelberg. (2012)
4. Samalik, J.: Process mining application in software process assessment. Eindhoven: Tech-
nische Universiteit Eindhoven. ((Co-)promot.: prof.dr. R.J. Kusters, prof.dr.ir. P.W.P.J.
Grefen & dr.ir. J.J.M. Trienekens) (2012)
5. Cruañas, J. R.: Process Mining Opportunities for CMMI Assessments (2012)
6. Rubin, V. et al.: Process mining framework for software processes. In:Software Process
Dynamics and Agility. Springer Berlin Heidelberg. p. 169-181 (2007)
7. Rout, T. P., El Emam, K., Fusani, M., Goldenson, D., & Jung, H. W.: SPICE in retrospect:
Developing a standard for process assessment. Journal of Systems and Software, v. 80, n.
9, p. 1483-1493 (2007)
8. SAFE, V. 2: A Safety Extension to CMMI-DEV, V1. 2, Defence Materiel Organisation,
Australian Department of Defence, Software Engineering Institute, TECHNICAL NOTE
CMU. SEI-2007-TN-006http://www.sei.cmu.edu/pub/documents/07.reports/07tn006.pdf.
(2007)
9. Ailenei, I., Rozinat, A., Eckert, A. & van der Aalst , W.M.P.: Definition and Validation of
Process Mining Use Cases. In F. Daniel, K. Barkaoui, and S. Dustdar, editors, Business
Process Management Workshops, International Workshop on Business Process Intelli-
gence (BPI 2011), volume 99 of Lecture Notes in Business Information Processing, pages
75-86. Springer-Verlag, Berlin (2012)