Process Mining Extension to SCAMPI

Abstract

Existing process assessment methods, such as SCAMPI-Standard CMMI Appraisal Method for Process Improvement, do not use contemporary data collection and analysis techniques like processes mining, text mining or data mining. On the contrary, they use traditional ones: questionnaires, document review, interviews and demonstrations. Process mining is a technique that can be used to aid process assessments, aiming to conduct them with greater deepness and coverage, while keeping similar level of effort. The purpose of the PhD work is to develop a framework (structure and content) to apply process mining techniques in SCAMPI assessments.

Introduction

The present paper proposes the Process Mining Extension to SCAMPI, a framework where process mining techniques are added to existing assessment techniques. The paper is organized as follows. Section 2 presents the research question. Section 3 describes the background. Section 4 presents the significance of the work. Section 5 presents the research design and method. Section 6 provides the research stage.

Background

The research question can be stated as: “Comparatively with traditional SCAMPI assessments, does the proposed method extension enable software process assessments with more objectivity, accuracy, depth and coverage of aspects related to the execution of processes, while maintaining similar levels of effort, cost and time?”.

SCAMPI [ 1 ] is a method used to assess organizations that use CMMI as a reference for their operations (software development, service management, etc). The fundamental idea behind the SCAMPI, as well as other similar assessments, is that the conduction of an activity or process results in "footprints" called objective evidences, which are evaluated by experts to judge whether they satisfy best practices of a given CMMI reference model [ 2 ].

According to Fig. 1, extracted from Process Mining Manifesto, Process Mining is a set of techniques, tools, and methods to discover, monitor and improve real processes (i.e., not assumed processes) by extracting knowledge from event logs commonly available in today's (information) systems [ 3 ].

There are three main types of process mining techniques [ 3 ]: a) Process Discovery (from an event log, a “as is” process model is identified); b) Conformance (or Compliance) Checking (an existing process model is compared to an event log of the same process); c) Enhancement (a process model is improved using information extracted from a log).

In order to identify work that proposed the use of process mining in process assessment, a systematic literature review was conducted. As a result of the application of a defined criterion and procedure, in six renowned scientific databases, only 6 out of 26 resulting papers were selected. Since none of them mentioned which process assessment methods were used, an additional search on Google Scholar was conducted using the same terms, resulting in some relevant papers, as follows: • PhD thesis of Samalik, entitled Process Mining Application in Software Process Assessment [ 4 ]. The objective was to promote the use of process mining in software process assessment and improvement. Her conclusion was that techniques for collecting information derived from process mining can be applied to improve the data collection on software process assessment. However, conclusion was reached by qualitative judgment without objective criteria. In addition, process mining techniques that should be used were not nominally listed.

• Master dissertation of Cruañas, entitled Process Mining Opportunities for CMMI Assessments [ 5 ]. The objective was to investigate the literature concerning support tools to find out if it is possible to use process mining to improve the assessment of CMMI. His conclusion was that process mining can not only help improve the current CMMI assessments, but can also be a useful tool to assist data collection. However, conclusions were based on the generalization of processes mining techniques and perspectives without using objective criteria. Moreover, no process mining technique in particular was pointed out.

• A third paper found is [ 6 ] where different aspects of processes mining were addressed, such as control perspective, information perspective and organizational perspective. Some algorithms such as alpha algorithm, heuristics miner, genetic miner, social network miner, organizational miner and activity miner, which can be applied, were cited in the paper.

Although these papers demonstrate application of process mining in process assessments, there is no formal guidance of how to conducting it, covering for instance, how to capture business rules, how to compare models and logs, which process mining algorithms to use, and when, etc. 4

Significance

According to [ 7 ], existing process assessment methods (such as SCAMPI) have limitations: they are manual, time-consuming, inefficient, subjective and generally require experienced appraisers. However, these days, detailed information about processes is recorded in the form of event logs, transaction logs, databases, etc. In this sense, in a process assessment is no longer justifiable that only a small set of processes are checked. Instead, the entire process and all its instances should be considered, as long as this represents low costs, naturally. Additionally, in existing assessment methods, the following techniques are used for gathering information about the running processes in an organization: questionnaires, document review, interviews and demonstrations. It means that no contemporary data collection and analysis techniques such as data mining, text mining or process mining is used.

Therefore, it is proposed the application of process mining techniques on the SCAMPI. It means that event logs of software processes would be used to understand past and current situation in a complete, economical, reliable and accurate manner, thereby contributing to the collection and analysis of data, which are critical activities in any software process assessment method.

The premise is that nowadays companies have been extremely efficient in collecting, organizing, and storing a large amount of data obtained in their daily operations. Most of these companies, however, do not use such data properly so as to transform them into knowledge to be employed in assessment activities. The need of companies to learn more about how their processes actually operate is a major driver behind the development and increasing use of process-mining techniques.

The main objective of this work is to develop a framework for the application of process mining techniques in SCAMPI-based assessments. This framework aims at enable software process assessments with more objectivity, accuracy, depth and coverage of aspects related to the execution of processes (such as duration and sequence of activities, start and end dates and records of who were the real executors), while maintaining similar levels of effort, cost and time.

Research design and methods

According to Fig 2, the proposed framework is a structure that serves as a guide for applying process mining techniques in SCAMPI assessments. The intention is that such guidance could be seen as an extension to SCAMPI method description. Analog approaches already exist such as the SAFE extension to CMMI-DEV [ 8 ]. It means that the Process Mining Extension to SCAMPI adds (or modifies) content to the current SCAMPI method. Its content covers the application of process mining aspects in SCAMPI method. Typically it comprises full or partial processes or activities, although any element can be added or expanded, such as inputs, outputs, tools and techniques.

Process Mining Extension to SCAMPI is a document with the following chapters: Executive Summary; Abstract; 1-Introduction; 1.1- Background and Acknowledgements; 1.2-Purpose and Scope; 1.3-Relationships with CMMI and SCAMPI; 1.4Structure of the Process Mining Extension to SCAMPI; 1.5-Intended Audiences; 1.6Usage scenarios; 2-Content; Appendix A: References; Appendix B: Acronyms; Appendix C: Glossary; Appendix D: Contact.

In order to define which content would be added or modified in Process Mining Extension to CMMI, some references were considered, beyond the SAFE extension to CMMI-DEV. For instance, [ 9 ] have proposed process mining use cases – typical applications of process mining functionalities in practical situations – to be used for detailed evaluation of process mining tools. Here, these use cases are used to identify typical process mining situations that are pertinent and could be applied in SCAMPI assessment. From the original list of 19 use cases, the following ones were taken in consideration: from Discovery perspective, UC1-Structure of the process; from Conformance Checking perspective, UC6-The degree in which the rules are obeyed and UC7-Compliance to the explicit model.

Research stage

The Process Mining Extension to SCAMPI is under development. Some chapters of the document are more advanced than others, such as 1.4-Structure of the Process Mining Extension to SCAMPI, 1.6-Usage scenarios and 2-Content.

In addition, the implementation of a running example to apply process mining techniques in SCAMPI assessments was conducted. The approach seemed to be feasible, as demonstrated using Disco and ProM process mining tools. Process mining techniques are demanded in order to transform existing process assessment methods, such as SCAMPI, into more productive and economically viable methods. Process mining enables an easy comparison on how processes are performed in practice versus the way they are designed to operate.

As future work, it is intended to use the framework in real SCAMPI assessments and to conduct statistical analysis and hypothesis thesis of performance parameters such as effort, duration, coverage and quality of results to quantitatively evaluate the benefits.

References

1. SCAMPI Upgrade Team.: Standard CMMI Appraisal Method for Process Improvement (SCAMPI), Version 1 .3a: Method Definition Document for SCAMPI A, B and C (CMU /SEI-2013 -HB-001) . Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University. http://cmmiinstitute.com/resource/standard -cmmi-appraisal-methodprocess-improvement-scampi-b-c-version-1-3a-method-definition-document/

2. Chrissis , M. B. , Konrad , M. , & Shrum , S.: CMMI for Development: Guidelines for Process Integration and Product Improvement . Addison-Wesley Professional ( 2011 )

3. Van der Aalst , W. et al.: Process mining manifesto . In: Business process management workshops p. 169 - 194 . Springer Berlin Heidelberg. ( 2012 )

4. Samalik , J.: Process mining application in software process assessment . Eindhoven: Technische Universiteit Eindhoven . ((Co-)promot .: prof.dr. R.J. Kusters, prof.dr.ir. P.W.P.J. Grefen & dr.ir. J.J.M. Trienekens) ( 2012 )

5. Cruañas , J. R. : Process Mining Opportunities for CMMI Assessments ( 2012 )

6. Rubin , V. et al.: Process mining framework for software processes . In:Software Process Dynamics and Agility . Springer Berlin Heidelberg. p. 169 - 181 ( 2007 )

7. Rout , T. P. , El Emam , K. , Fusani , M. , Goldenson , D. , & Jung , H. W.: SPICE in retrospect: Developing a standard for process assessment . Journal of Systems and Software , v. 80 , n. 9, p. 1483 - 1493 ( 2007 )

8. SAFE , V. 2: A Safety Extension to CMMI-DEV , V1 . 2, Defence Materiel Organisation , Australian Department of Defence, Software Engineering Institute, TECHNICAL NOTE CMU. SEI-2007 -TN- 006http://www.sei.cmu.edu/pub/documents/07.reports/07tn006.pdf. ( 2007 )

9. Ailenei , I. , Rozinat , A. , Eckert , A . & van der Aalst , W.M.P. : Definition and Validation of Process Mining Use Cases . In F. Daniel, K. Barkaoui , and S. Dustdar, editors, Business Process Management Workshops, International Workshop on Business Process Intelligence (BPI 2011 ), volume 99 of Lecture Notes in Business Information Processing , pages 75 - 86 . Springer-Verlag, Berlin ( 2012 )