1 A Learning Support for the Definition of Process Flexibility O. Jaufman, M. Stupperich, K. Schneider, A. Dold, N. Kleiner Abstract —At present, software development in the automotive industry is characterized by frequent changes caused by new innovations, fast-growing system complexity, growing software portion in cars, changing business relationships. This dynamical environment demands for flexible software processes. In order to improve a software development process with respect to flexibility, it is necessary to characterize what kind of flexibility is required. Therefore, we defined a set of requirements for desired processes based on our process analysis in DaimlerChrys-ler’s engineering departments and analysis of related contributions proposed in the literature. Based on this requirement the existed processes can be analyzed to identify its improvement potential. The application of the requirements is illustrated in the context of a case study. Index Terms — Process flexibility, requirements, case study, software development —————————— u —————————— 1 INTRODUCTION N owadays, the automotive industry is confronted with a highly dynamic environment which is chara cterized by frequent changes due to innovations, business re- a way as to be measurable enabling a systematic evaluation of the process flexibility. Consequently, a suitable support for the definition and evaluation of the process flexibility is lationships or new product structures. One of the reasons called for. for this trend is the fact that software has come to play a In this paper, such a support is presented. It consists of more and more important role in the automotive industry an emergent knowledge base and a flexibility definition and will be the major source of innovation in the future. As process. The emergent knowledge base provides generic software is a relatively new science, the corresponding new knowledge related to the definition and evaluation of the effective and efficient processes for software development process flexibility. The flexibility definition process defines which consider the mechanical and electrical engineering the steps to be performed in order to define the process domains are needed. Especially important in such dynamic flexibility for a specific software development process. environments is flexibility: software engineers desire flex i- The benefit of our approach is support, first, through the ble software development processes in order to be able to definition and evaluation of the process flexibility and, sec- quickly react to the changes in the development environ- ond, through the identification of process weaknesses and ment. improvement potentials with respect to process flexibility. The problem is that the processes proposed for devel- This paper sets out both the emergent knowledge base opment of large, complex, and safety-critical software are and the flexibility definition process and introduces a case not flexible enough for our ever-changing environment. For study for validation. It therefore targets project managers, example, last year our prescriptive issue tracking process software engineers, and process quality assurance staff who was changed significantly three times. The reasons for the are interested in defining or evaluating process flexibility. changes were newly gained knowledge during the applica- The remainder of the paper is structured as follows. tion of the “old” process and the need for three depart- Chapter 2 discusses related work. Chapter 3 defines the ments to work together collaboratively. objectives of our research work. Chapter 4 presents our The research question arising from the problem is basic: approach, viz. the support for the definition and the evalua- What should a flexible software development process look tion of process flexibility. In Chapter 5 our approach is il- like? In order to define a flexible software development lustrated by means of a case study. Finally, Chapter 6 process, the requirements as to the process flexibility are summarizes the most important contributions of the paper first needed. These requirements should be defined in such and addresses future work. ———————————————— 2 RELATED WORK • O. Jaufman is with the DaimlerChrysler AG, Ulm, HPC U800, E-mail: There are two chief ways to define the quality of a proc- Olga.Jaufman@DaimlerChrysler.com. • M. Stupperich is with the DaimlerChrysler AG, Ulm, HPC U800, E-mail: ess. One is to employ a process assessment sta ndard such Michael.Stupperich@DaimlerChrysler.com. as SPICE [5][6]. Another way is the application of a define- • K. Schneider is with the University of Hanover, Hannover, E-mail: your-own-model approach such as the GQM measurement Kurt.Schneider@Inf.Uni-Hannover.de. method [9]. • Dold is with the DaimlerChrysler AG, Ulm, HPC U800, E-mail: A- xel.Dold @DaimlerChrysler.com. The process assessment standards [5], [6], [8] character- • N. Kleiner is with the University of Ulm, Ulm, E-mail: ize processes based on a reference model. This reference nikolaus.kleiner@informatik.uni-ulm.de. 2 QUATIC’2004 PROCEEDINGS model typically defines the process capability levels and We aim to achieve the second goal with our learning the criteria that have to be fulfilled for a considered level. support: our objective is to be able to effectively and effi- The process assessments are then performed on the basis of ciently define and evaluate process flexibility in a measur- the criteria. The advantage of the standards lies in the fact able way. “Effective” means that the measurement program that they define the necessary criteria to be fulfilled in order covers all the aspects that are important for process flexibil- to develop a product of the desired quality. The weak point ity from the project stakeholders’ point of view. “Efficient” of the standards considered is that they provide neither an means that the effort required for the definition of process explicit definition for process flexibility nor the criteria or flexibility should be as minimal as possible. We firmly be- metrics for the evaluation of process flexibility. lieve that reuse of the available knowledge and experience To define process flexibility in a measurable way, a would contribute to a more effective and efficient evalua- measurement approach can be applied. As different con- tion of process flexibility. texts have different understandings of the process flexibil- We aim to achieve the third goal to make it clear how the ity, a define-your-own-model approach should be applied knowledge provided in the emergent knowledge base can [2]. The representative for define-your-own approaches is be reused for the definition and evaluation of a concrete the GQM method. project. The GQM method is a goal-oriented measurement The benefit of our approach is a support for software en- method according to which a measurement goal should be gineers, first, through the definition of requirements placed refined into metrics via questions. The advantage of the on process flexibility and, second, through the evaluation of GQM approach is that it allows the context and the project process flexibility. stakeholders’ views of the process flexibility to be consid- ered. The key drawback of the method, however, is that it 4 LEARNING S UPPORT FOR THE EVALUATION OF calls for comprehensive knowledge and experience from measurement personnel. Unfortunately, a lot of companies PROCESS FLEXIBILITY do not have the experts with the necessary knowledge and experience. 4.1 The Informal Definition of Process Flexibility The missing knowledge and experience needed to define Process flexibility is defined as the ability of a process to be and evaluate process flexibility can be gained by learning easily and quickly adaptable to a new situation. A new situation from past knowledge and experience. Several approaches can be caused through an innovation, a new business con- for supporting the learning process have been proposed in nection, changes in product structure, or other factors. the area of artificial intelligence. For example, deriving a In order to make this key definition clear, the history of a process model based on logging of the activities of the peo- process, called issue tracking, is considered. The issue ple involved in the process [4] is one such method. These tracking process targets capturing of issues such as changes approaches address more the implementation aspect of in the product requirements, faults in the realization of the learning; yet before addressing the implementation aspect requirements or a decision to realize a specific feature ear- of process flexibility by learning, concrete requirements for lier than planned. Figure 1 illustrates the issue tracking process flexibility are needed. If process flexibility is to be process with respect to its four different phases. The proc- evaluated systematically, it needs to be measurable. There- ess is considered to be in another phase, if any process fore we aim at supporting a measurable definition and the changes are occurred. In the first phase, the issue tracking systematic evaluation of process flexibility by defining and process is described by two statuses “issue open” (i.e., a providing the following: new issue that needs to be remedied is identified) and “is- • The generic knowledge related to the process flexibil- sue closed” (i.e. the issue has been remedied). ity definition and evaluation. During the application of the issue tracking process • A process for the usage of the knowledge provided. (phase 1) for tracking the issues arising during the software development of an electrical device, for example, new ex- perience is gained. Based on this experience, the issue track- 3 T HE G OALS ing process is refined. The following steps are added: ana- The goals of our proposed concept are threefold: lyse an issue, confer with an expert, evaluate an issue, re- • To define the term “process flexibility” informally. fuse the correction of the issue, assign the issue to a person • To organize generic knowledge and experience related responsible for it, remedy the issue, and verify the correc- to the definition and evaluation of the process flexibil- tion. ity in the emergent knowledge base. During the application of the issue tracking process in • To define a process for the use of the emergent knowl- accordance with phase 2, the following modifications were edge base. found to be needed: We aim to achieve the first goal with our learning support, The assign step is modified: in this step not only the as- since the term “process flexibility” is not at all explicitly signment of an issue to a responsible person is to be per- defined in the process assessment standards considered; formed. Additionally, whether the issue is a software issue, however the industry needs such support for the definition a mechanical issue, or a hydraulic issue should be defined. and evaluation of process flexibility. JAUFMAN ET AL.: A LEARNING SUPPORT FOR THE DEFINITION OF PROCESS FLEXIBILITY 3 Fig. 1. Emergence of the Issue Tracking Process1 In order to support the identification, the causes for the process modification are analysed. As an input for the iden- The order of the activities is changed: the assign step is tification of change drivers, the Aalst et al. autonomy of the performed before the analyse and evaluate steps. The con- workflow change [1] is used. This autonomy is abstracted fer step is omitted. A new step is added - release: during to the autonomy of software development changes and this p the integration is performed. modified based on the experience gained in a case study. The process in the third phase shows the modifications. The modification encompasses the introduction of roles The emergence of the issue tracking process (phase 4) is who initiate the process modification. The case study is caused by a need for inter-departmental collaborative performed in the context of the issue tracking process. Fig- working. To achieve a consistent process, an additional ure 2 indicates that our autonomy of changes distinguishes process step, confer, is first added and a transition from the between the following change driving factors: market, assign* step to the close step is then omitted. software development legislation, new knowledge, techni- Referring to this example, process flexibility can be in- cal experience, errors made during software development, formally characterized by the ease to quickly and system failures. • identify the need for a process modification. These drivers are introduced by the roles: marketing de- • understand what kind of process modification is partment, client, customer, project manager, researcher, needed. cooperation partner, development team or the system plat- • perform a required process modification. form. Based on the knowledge taken from Figure 2, six issue Identification of the Need for a Process Modification classes are distinguished: Issue class 1: issues caused through availability of new knowledge, experience or laws (initiated by researchers or developers). Issue class 2: issues caused through a change in the internal organizational structure (initiated by the project manager). Issue class 3: issues caused through a change in the external organizational structure (initiated by cooperation partners or by the project manager). Issue class 4: issues caused through an incorrect implemen- tation of the requirements from the product concept catalog (initiated by a product developer). Issue class 5: issues caused through a change in product requirements (initiated by a customer/client or a marketing team). Issue class 6: issues caused through system failures (initi- ated by the system platform). The issue classes identified here are not to be seen as a silver bullet but as an emergent shopping list for the identi- fication of the relevant process-specific issue classes. Fig. 2. Drivers for a Process Modification “Emergent” means that this set should be updated in line with the newly acquired knowledge and the experience gained from its application. Both the autonomy of the change drivers and the issue classes are intended to first 1 The purpose and the content of the figure will be explained later. 4 QUATIC’2004 PROCEEDINGS help to quickly identify a need for a process modification provement of software development processes to achieve (e.g. by process monitoring with respect to relevant issue enhanced product quality. Nevertheless, improvement of classes). Second, the knowledge helps to define the issue software development processes with respect to flexibility classes that a given process should be especially flexible for. is a pivotal aspect, especially in such dynamic environ- ments as we are faced with today. Unfortunately, such Understanding the Type of the Process Modification standards do not define the criteria or metrics for the In order to understand what kind of process modifications evaluation of the process flexibility. are needed, the software development process at hand It is for this reason that the GQM method [9] is applied for should be understandable for people involved in the proc- this purpose. We decided to utilize the GQM method, as it ess. This means, for example, that everybody should be able is a widely used measurement method that allows a goal- to evaluate which activities have been carried out, which oriented identification of the desired criteria and metrics for ones are to be performed next, who should do which activi- process flexibility. To derive the desired criteria and met- ties, when the activities to be performed should be started, rics, the following activities are performed: and what kinds of resources are needed to perform the ac- The GQM goal is described with respect to its attributes tivities. (object: process, purpose: to define and to evaluate, focus: process flexibility, viewpoint: researcher, developer, man- Carrying out the Process Modification ager, context: software development). Looking back at our issue tracking example (Fig. 1), we Based on the knowledge presented in Section 4.1 and focus- note that the modification is performed through executing ing on the GQM goal, the process flexibility is described by one or several of the following modification activities : four high-level questions. Then different aspects of each of 1. Removing an existing process step. the four questions are described by sub-questions. The 2. Adding a new process step. modification activities, autonomy of the drivers, and the 3. Modifying an existing process step. issue classes presented in Section 4.1 are taken into account 4. Changing the order of the process steps. in doing so with the focus on the GQM goal. This “decom- 5. Removing a transition between two process steps. position” (i.e. refinement of different aspects of the high - 6. Adding a new transition. level question through sub-questions) is performed until Consequently, a flexible process is characterized by ease questions that are not concrete enough from our point of and speed of performance of modification activities. view are derived. The defined questions are the desired criteria. 4.2 Emergence Knowledge Base Finally, the metrics are assigned to the “concrete enough” questions. Metrics propose a scale for “answering” the In order to define and evaluate process flexibility system- questions. We proposed both qualitative and quantitative atically, it is important to understand four aspects: metrics. 1. The changes that can cause the process modifica- In this way, all the criteria and the metrics were defined. tion. The proposed set of criteria and metric is a generic, reus- 2. The changes to which the process should be espe- able set. The approach for the reusage of the set will be dis- cially sensitive. cussed later. 3. What the relevant stakeholders understand by a Organizing the Knowledge in the Knowledge Base flexible software development process (i.e. what are the criteria and metrics for the evaluation of the The knowledge and experience gained during the analy- sis is stored in the emergent base with respect to the scheme process flexibility). 4. The constraints that are to be fulfilled (e.g. quality depicted in Figure 4. The scheme is adapted from the de- fine-your-own-model, the so-called SQUID model [7]. The assurance, documentation, product preparation). SQUID model is employed, since it supports top-down de- The fourth aspect is essential as process flexibility is desired composition as defined by the GQM method. Figure 3 but may not negatively affect the product quality. There are shows that each criterion can either be decomposed in sub- process steps that have to be performed for this. The focus criteria that describe different aspects of the criterion in of the paper is, however, on the first three aspects. Conse- more detail or be made measurable through assignment of quently, the emergent knowledge base should provide the a measurement model. Each criterion is captured in the issue classes, the criteria and the metrics for the measurable emergent knowledge base as shown in Table 1. definition and evaluation of process flexibility, and the relevance of the criteria with respect to the issue classes. To achieve this knowledge, the criteria and metrics for the process flexibility are to be defined. Defining the Criteria and Metrics A set of criteria and metrics that might be relevant for the definition and evaluation of the process flexibility is needed. Before defining the desired set, existing process assessment standards [5], [6], [8] are considered. This is done because process assessment standards aim at the im- JAUFMAN ET AL.: A LEARNING SUPPORT FOR THE DEFINITION OF PROCESS FLEXIBILITY 5 very important, important, somewhat important, and not important at all. Before evaluating the issue classes, the pro- ject stakeholders should understand which effect the repre- sentatives of the issue classes have and how often the changes occur. In the third step, the appropriate criteria and metrics for the definition of the process flexibility are to be selected and identified. Which criteria are relevant for a given software development process is to be decided contingent on the importance of the issue classes. The relevant criteria and metrics are to be selected from the emergent base. The miss- ing criteria can be identified in a top-down manner by ap- plication of the GQM method. Fig. 3. Schema of the Emergent Knowledge Base In the fourth step, a process flexibility model is to be produced. This should be done by first deleting the criteria and metrics not relevant for a given software development TABLE 1 EXERPT FROM THE EMERGENT KNOWLEDGE BASE process from the decomposition tree. Second, the identified missing criteria and metrics have to be inserted into the decomposition tree with respect to the decomposition structure. Third, by specifying the criteria in the context of a given process. Finally, the relevance of missing criterion and metrics is to be evaluated on the same scale. The evaluation has to take the importance of the issue classes into account. The table indicates that each criterion in the emergent base is characterized by the reference to the father question. 5. APPLICATION EXAMPLE AND EXPERIENCE In order to clarify our approach and its benefits, a case 4.3 Using the Emergent Knowledge Base study in the context of the issue tracking process was per- The emergent knowledge base provides the issue classes, formed. criteria, and metrics for the definition and evaluation of 5.1 Application of Our Approach process flexibility. In order to keep the contents of the emergent base up to date, it should continually be evalu- The following sets out the four steps taken in defining ated and updated. Consequently, the corresponding proc- the flexibility of the process. esses are needed. The focus of the paper is, however, the process flexibility definition process. Step 1: Identify the process-specific issue classes. The objective of the process is to select and identify suit- As cited in the history of the si sue tracking process, the able criteria and measures for a given software develop- knowledge and experience we gain from the definition of ment process by reusing the knowledge from the emergent issue tracking process lets us identify the following drivers knowledge base. An overview of the process is set out in for a process modification: Fig. 4. 1. New experience, knowledge. 2. A change in the internal organizational structure. 3. A change in the external organizational structure. Consequently, when referring to the generic issue classes, the following issue classes are identified as issue tracking specific: Issue class 1: issues caused through availability of new knowledge or experience (initiated by developers). Fig. 4. The Flexibility Definition Process Issue class 2: issues caused through a change in the internal organizational structure (initiated by the project manager In the first process step, the relevant issue classes from the or developers). knowledge base are to be selected and the missing issue classes identified for a given specific software development Issue class 3: issues caused through a change in the external process are to be identified. organizational structure (initiated by cooperation partners The task of the second process step is to evaluate the is- or by the project manager). sue classes for which the process should be especially flex i- ble. The evaluation is to be performed by all relevant proc- Step 2: Evaluate the importance of the issue classes. ess stakeholders (e.g. software developers, process quality Based on the history of the issue tracking process, it is as- assurance staff, etc.). The importance of the issue classes sumed that the issues from the first issue class have the should be evaluated on the basis of the four-point scale: greatest impact and the highest likelihood of occurrence. 6 QUATIC’2004 PROCEEDINGS Therefore the first issue class is evaluated as very important. The effects of the second and third issue classes are as- sumed to be relatively small, because, for example, due to the wish to cooperate with each other the cooperation part- ners usually try to achieve a consistent process with as few process modifications as possible. The occurrence likeli- hood for the issues from the second and third issue classes is low, as a rule. Therefore the second and third issue classes are evaluated as somewhat relevant. Consequently, the issue tracking process should be espe- cially flexible with respect to the newly gained knowledge and experience. Step 3: Define the process-specific criteria and metrics First, the relevance of the criteria and metrics provided by the emergent knowledge base is evaluated based on the Fig. 5. An Excerpt from the Process Flexibility Model scale very relevant, relevant, somewhat relevant, and not relevant at all. Second, the missing criteria and metrics are identified by application of the GQM approach in a manner 5.2 Benefits of the Flexibility Model similar to that set out in Section 4.2 for the context issue The benefits of the process flexibility model are outlined in tracking process. The criteria and metrics provided in the the following. emergent knowledge base serve as a support. 1. We can evaluate the issue tracking process based on Step 4: Define a process specific flexibility model the criteria and metrics provided in the model. For ex- The process specific flexibility model is defined by ample, the issue tracking process (phase 3) presented • omitting criteria evaluated as non -relevant from the de- in Figure 1 can be evaluated as follows: composition tree. For example, the criterion “Does the • The process is complex, as it has nine activi- process have a high level of information hiding? (i.e., only ties and eight roles involved in the process; the absolutely necessary information produced during a but the process is part of the process hierar- process step should be passed on)” is removed. The crite- chy and the maturity of the process is quite rion is not relevant because, in the issue tracking process, high. the people involved in the process should be able to see • The process is understandable, as it has a the information of interest to them even if they do not clear responsibility concept; but it has many necessarily have anything to do with the information. dependencies (i.e. the synchronization and • adding missing criteria and metrics in the decomp osition coordination points) between process steps. tree with respect to its relationships to other tree criteria. • The organization intends to perform the ex- For example, the criterion “How easy it is to quickly iden- plicit process monitoring and has captureed tify the risk and attractiveness of the implementation of a the knowledge with respect to the attractive- considered issue?“ is identified as missing and is, there- ness and the risk of an implementation of the fore, added in the decomposition tree. issue. • specifying the criteria and metrics in the context of the 2. The improvement potentials of the issue tracking issue tracking process, for example, the specification of process are as follows: the criterion “How complex is the process?” (based on • It would be helpful if the variants of the issue our experience related to the issue tracking process): tracking process together with its context were • An issue tracking process is very complex if it has provided in the emergent knowledge base. The more than ten process steps, more than eight differ- ent roles are involved in the process, and it is not de- reuse of the process variants would help to quickly modify the process. fined hierarchically. • An issue tracking process is complex if it defines • An efficient re-reuse process would support a more than five process steps and has more than five quick and easy identification of the kind and different roles involved in the process. extent of the requisite process modification. • An issue tracking process is non -complex if it has • A tool-supported, regularly performed identifi- fewer than six process steps and has at most six dif- cation of the difference between the prescrip- ferent roles involved in the process, a clear responsi- tive issue tracking process and the activities bility concept, and few dependencies between proc- performed by the developers would help to ess steps. quickly identify the need for a process modifi- Thus, a flexibility model specific for the issue tracking cation. process described in Fig. 1 is provided. Fig. 5 portrays an 3. We achieve a definition of the requirements for the excerpt from the model. process flexibility. For example, in order to be flexible a process should JAUFMAN ET AL.: A LEARNING SUPPORT FOR THE DEFINITION OF PROCESS FLEXIBILITY 7 • define what artifacts need to be developed, flexibility is needed but may not have a negative impact on when the development of the artifacts should the final product quality. A further step is to analyze how be started, who should develop which artifact, much flexibility is permissible and whether the flexibility when an artifact should be finished, etc. alternatives proposed are sufficient. • be defined hierarchically in order to be well understandable and traceable for the people in- volved in the process. REFERENCES • explicitly define process monitoring. • explicitly define delta analysis between the pre- [1] Aalst, W. M. P., Jablonski, S. Dealing with Workflow scriptive and currently performed processes. Change: Identification of Issues and Solutions, Intern a- • systematically package and reuse the acquired tional Journal of Computer Systems, Science, and knowledge and experience related to process Engineering, 15(5), 2000: pp. 267 – 276. flexibility. [2] Boehm, B. and Turner, R., Balancing Agility and Discipline: A Guide for the Perplexed, Addison 6. C ONCLUSION Wesley, 2004. [3] Fenton, N. and Pfleger, S. Software Metrics – A A present trend in software development is a highly dy- Practical and Rigorous Approach. International namic environment. Such an environment is characterized Thomson Computer Press, 2 nd edition ed., 1996. by frequent changes caused by innovations, business rela- tionships, and other factors. In order to be able to quickly [4] Joachim Herbst, Niko Kleiner Workflow Mining: A react to a change, software engineers wish to have flexible Case Study from Automotive Industry, the 10th software development processes. And to be able to define European Concurrent Engineering Conference, Ply- flexible software development processes, the requirements mouth, UK, April 2003 placed on the process flexibility are needed. The require- ments should be defined in such a way that they are meas- [5] ISO/IEC TR 15504-2:1998(E), ISO Standard. Infor- urable in order to be able to evaluate whether a process is mation technology — Software process assessment sufficiently flexible for a given software development envi- — Part 2: A reference model for processes and proc- ronment. We have hence, in this paper, outlined our ap- ess capability, ISO/IEC, 1998. proach to support the research question “How can process flexibility be defined in a measurable way?”. [6] ISO/IEC TR 15504-5:1998(E) , ISO Standard Infor- For this purpose, we have first informally defined proc- mation Technology-Process Assessment-Part 5: An ess flexibility, second an emergent knowledge base has example process Assessment Model, ISO/IEC been developed, and, third, a process for the measurable JTC1/SC7, 2003. definition of the process flexibility has been derived. The informal definition of process flexibility was needed, as the [7] Kitchenham, B., Linkman, A. Pasquini, and Nanni, standards considered do not explicitly define the term. The V. The SQUID-Approach to Defining a Quality emergent knowledge base aims at providing the knowledge Model, Software Quality Journal, vol. 6, no. 3, pp. related to the definition of process flexibility in a measur- 211-233, 1997. able way. We believe that the application of the knowledge provided would help to more effectively and efficiently [8] ISO 15497, Road Vehicles – Development Guide- define the process flexibility for a concrete process. The lines for Vehicle based Software, the Motor Industry flexibility definition process allows us to define the issues Research Association, 2000. with respect to which the process should be especially flexible and, based on the evaluation of the issue's impor- [9] Rombach, R. Practical benefits of goal-oriented tance, to select and to identify the criteria and measures measurement. In N. Fenton and B. Littlewood, edi- relevant for the process flexibility of a software develop- tors, Software Reliability and Metrics. Elsevier ment process at hand. The knowledge stored in the experience base such as the flexibility definition process is validated and illustrated by means of a case study. The activities and the results gained in the case study are elaborated. We therefore feel that the work presented in the paper can be easily transferred into other business units and organizations and enable them to define the flexibility of their processes in a measurable way. Our next steps in context of this work are first to explic- itly define the constraints to be taken into account. This is essential as, in the automotive industry application domain,