In this case study, DEMO was applied in a real lean six sigma project. The study was conducted in the Merck organisation, a worldwide pharmaceutical company with a research lab and plant for the production of birth control pills in the Netherlands. The case study was conducted at this site and was intended to solve “order reliability” problems. While the general cause of low order reliability was known, it remained unclear why the organisation at Merck had difficulties in adapting to the market turbulence. The 'traditional' lean six sigma methodology had already been applied in three initiatives, which failed to restore reliability in order fulfilment.
The CTQ tree [1] from previous lean six sigma [2] initiatives was made available to us as a suggested starting point for a fourth initiative. In this CTQ tree, the strategic focal point (i.e. order reliability) was already specified by measurable variables. In this case, order reliability was specified by the delivery of the correct ‘amount’ on the agreed ‘delivery date,’. Also, the acceptable deviations for these quality variables existed when we started this initiative. Based on the information in the CTQ tree, we could define the lean six sigma concept of a ‘defect’ for this initiative. A defect is an order, for which the actual amount of product delivered and/or the actual delivery date fell outside the specified deviation. In a stable market, Merck achieves an ‘order reliability’ above ninety-five percent, which corresponds to 3.2 sigma. In the current turbulent market, Merck achieved a quality level that was no higher than seventy-two percent, which corresponds to 2.1 sigma. The sigma value adds extra valuable information on the level of defects. The sigma value expresses how tightly all the values of a quality variable are clustered around the mean. In the case of Merck, one can say the spread of the values of the quality variable in a stable market situation was more tightly clustered around the mean than in the unstable market. Or, in terms of lean six sigma, in the old situation the Merck company was more in control than in the present situation. 2
The task in this case was to solve “order reliability” problems. While the general cause of low order reliability was known, it remained unclear why the organisation at Merck had difficulties in adapting to the market turbulence. The ‘traditional' lean six sigma methodology had already been applied in three initiatives, which failed to restore reliability in order fulfilment. The task to solve the order reliability” problems was to be conducted by Enterprise Engineering. 3
We interviewed the lean six sigma project members from the previous initiatives to learn from the choices made in these initiatives. They reported difficulties in identifying appropriate cause and effect relationships. They referred to the difficulty of identifying a ‘stable’ set of process variables, which meant that they could not identify a limited set of the most significant and influential process variables. After statistical analysis, they said they were confronted with a large set of process variables that could not be reduced any further. The process of working towards a critical set of process variables was fruitless. To avoid this problem in the fourth iteration, we made a classification scheme containing ten ‘reason codes’ representing kinds of reason. This was used during the observation phase to classify the defect orders. With the help of the reason code system, we observed the order fulfilment organisation for three months. We noted each defect order (deviance of +/-10% in the order amount and/or +/- 30 days from the agreed delivery date), and we recorded their reason, which was classified using the reason code scheme. We also recorded, in a free format, what actually happened with the order. This supplementary information was used later to learn more about the details of a particular situation.
After observation, and with the help of statistical analysis, we determined the process variables (kinds of reason) that have had the most influence on the quality variables, based on their correlation strength (read Table 1). All entries in the Table should be read as a tuple of variables (e.g. <qv1, pv1.1>) representing an association between a quality variable (qv) and a process variable (pv). A reader may not directly recognise the relationship between the quality variables, process variables and the classification system. This is due to the extra step of ‘data preparation’ between the organisational diagnosis steps of ‘observing’ and ‘analysis.’ In our activity of ‘data preparation,’ we processed the ‘supplementary information’ that was also recorded for each registered defect. We took the reports of the observer and isolated a set of defects (e.g. orders with an incorrect amount of products, see qv1 in Table 1). From this set, we took the defects that were clustered to, or assigned to, a reason code (e.g. ‘rc1 Manufacturing’). We then studied the supplementary information and extracted from this information the process variables (e.g. within manufacturing, all defects related to planning errors). The process variable name reflects two things: (1) the reason code and (2) the extracted process variable from the supplementary information. For example, pv1 in Table has the name: rc1_ planning_error. Based on this ‘data preparation,’ we could reduce the number of process variables to those that were significant. Unlike the three previous lean six sigma initiatives, this initiative was not overwhelmed by a huge number of kinds of process variable. This procedure was perceived as a step forward in organisational analysis.
Variable
Occurrence
Value Range To expose the interactions and mechanisms that facilitate the detected associations (e.g. <v1, v1.1>), we sought support in organisational modelling, as per the triangulation design principles [3]. In this case - as a follow up on [4] - we developed a DEMO model [5]. DEMO proposes a clear way of working for creating a constructional Nr. qv1 pv1.1 pv1.2 qv2 pv2.1 pv2.2 pv2.3 pv2.4
Correct Amount Available rc1_planning_error rc7_artwork_change On Time rc1_release_delay rc1_production_delay rc7_shipping_doc_delay
25 % of rc7 rc7_approval delay 20 % of rc7 30 % of rc1 45 % of rc7 42 % of rc1 21 % of rc1
DEMO Concept [T11] [A04] [A05] [T11] [A06] [A03] [A08] [A08] -10% ... promised_amount + 10% -45% … prom_amount …0%
0% … rework … 30% -30 days - delivery_date + 30 days - 5 days - prom_release_date + 15days - 5 days - prom_prod_date + 25days 0 days - prom_ship_doc + 15days 0 days - prom_cust_approv - + 7 days model of the organisation under consideration at the ontological level. Guided by the Ψ-theory in DEMO, we identified the transactions that are the elements of the ‘order fulfilment’ organisation. We created an organisational construction diagram (OCD, see Fig. 2) and its corresponding transaction result table (TRT, see Table 2).
In this case the observations need to be mapped and plotted in the OCD and TRT to study the causal inference support of DEMO. The rationale behind the mapping in this case was to identify those transactions or actors who – in our eyes – control the values of the quality variables and process variables during run-time of organisation.
Both reasons (RC1 and RC7) can be attributed to different process variables (e.g. planning errors, artwork changes) and assigned to various transactions/actors (e.g. T11 or A02). This situation was for us an indication that, even with respect to a single observed reason (i.e. manufacturing delays), there may be diverse underlying causes. For example, a manufacturing delay can be caused by actor A03, A04 or A06. Knowledge about the practice gained in the observation phase is crucial to identify the exact constructional component. However, this is not enough. We need to map variables on actors using objective criteria. Otherwise the mapping would be arbitrary, unguided and not reproducible. We agreed on three mapping rules. The first states: “a variable is managed, one Actor is responsible for its values.” The second states: “a variable is a subject within a transaction: the initiator and executor are only successful when they agree on the variable’s value.” The third rule states: “a variable is mapped once.” This constrained mapping led to an augmented OCD and corresponding mapping table. This augmented OCD shows – in our eyes – all the entities and activities that are involved in the causal mechanism. It sets the stage for the third step in identifying a causal mechanism, namely to identify its organisation and operation. 3.2
Identifying the operation of the mechanism In the previous step, we combined two kinds of evidence: statistics and organisational modelling. The latter was guided by the Ψ-theory, its application resulted in an ontological model of the organisation. When applying the Ψ- theory, and, more specifically, the operation axiom [5], we learn that not all transactions follow a straight-forward sequence. Self-activating actors may be responsible for creating new ‘facts.’ For example, the self-activating actors (i.e. A02, A04 and A10) are responsible for determining a plan and according this plan initiating new requests in transactions with other actors to achieve the scheduled dates.
Especially in Merck’s situation, we find multiple self-activating actors. They are not driven directly by other transactions, but use the available information to determine something, such as an optimal delivery deadline. The information available to actors in the OCD is represented by interstriction links, see the dashed lines in the OCD (Fig. 2). If we take a closer look to these interstrictions to understand the operation of Merck, we learned (see Fig. 4) how actors rely heavily on information from the production banks. These production banks reflect the history of all production facts produced in the runtime of the enterprise. To understand the operation of the causal mechanism - using the information about the entities and activities that are involved in this mechanism - we asked: ‘are the self-initiating actors informed about the values of the process variables?’ This question was raised when an employee expressed doubts about the availability of such information and suggested that informed decision-making might be at risk. 4
To answer the question, we studied the planning methodologies of actors A02 and A04. We drew the planning methodologies in the augmented OCD (Fig. 4), we see that the self-activating actor ‘A02 production management completer’ establishes a one-year production plan. This production plan is based mainly on information from ‘CPB001 sales order forecast.’ Furthermore, we see that the self-activating actor ‘A04 packaging management completer’ creates a 12-week plan for packaging, based on information from ‘T01 sales order completion.’ In addition to the evaluation of the planning methodologies, we evaluated the information systems landscape (Fig. 5). From this assessment, it became apparent that three different information systems were in place. The first system supports the actors A01 and A08. The second system supports actors A02, A03, A04 and A05, and the third system supports actor A06.
Consequently, it can be concluded that actor A01 has no access to relevant information, which causes issues for order reliability, since A01 is restricted to the information available in his information system. More specifically, information concerning stock values, planning, and information concerning production delays are not available for A01. It is vital that this information should be available, to ensure that the delivery date and volume in T01 are feasible. This analysis is the final point of organisational diagnosis: a clear insight is provided into the constructional causes of the observed business performance issues. This case illustrates how DEMO provides support for the use of a constructional perspective in organisational diagnosis to gain such insight. Resolving the identified issues is a task for subsequent projects.
In this section, we reflect on how the organisational diagnosis was performed in the case study. This reflection is based on the fact that, in organisational diagnosis, a diagnostician attempts to explain the functioning (and dysfunctioning) of an organisation in causal terms. Such a causal explanation must be contrasted to any correlations identified. It should be noted that the initial phases of the case study focus merely on identifying correlations. The focus on correlations is useful to isolate and to demarcate the phenomenon to be diagnosed. However, correlations are not sufficient to support a causal description for the phenomenon to be explained. What is needed after the identification of the associational model is an understanding and identification of the organisational entities that should be changed to remedy a problematic phenomenon. The adoption of ‘causal mechanism’ as the conceptualisation for a cause helps with this task. In the case study at Merck, a DEMO model was used precisely for this purpose. In this section, we will reflect on the feasibility of using DEMO for detecting a causal mechanism in lean six sigma, with a focus on the steps of the identification of the associational model. In the scope of this reflection, we address two aspects that are related to feasible mechanism-based approaches [6]: ‘flexibility in explaining' and ‘validation in explaining.'
The first aspect that we address is the experienced flexibility in explaining the quality problem. We highlight three different aspects of flexibility we found in the case study, by asking three critical questions: 1. can we handle mechanisms in which different types of variables interact? 2. if the available data and background knowledge do not allow us to identify a causal mechanism responsible for the phenomenon, can we deliver some explanation on other grounds? and 3. does the diagnosis approach allow a reciprocal connection in the way of thinking between existing (general) theories on causal mechanisms and establishing a (specific) causal theory for the enterprise under investigation?
The first question addressed the flexibility of the presented approach. We detected no types of variable that could not be included in this diagnosing approach. For instance, one of Merck's first iterations of the lean six sigma on order reliability examined how environmental variables (e.g. temperature) affect delivery times and vice versa. In this case, the diagnostician observed the temperature in the production facility (a ‘physical' process variable). In the same study, the diagnostician also observed ‘day of the week' as a ‘social' process variable. These variables are of different kinds. The statistical method of lean six sigma allows us to characterise a mechanism by associations broad enough to include both ‘physical' and ‘non-physical' process variables. DEMO gives associations, even between variables of different kinds, a causal meaning. For example, measuring ‘temperature in production' is only ‘relevant' in the execution step of T03 for which A03 is responsible. The location of this physical variable in the OCD was in this case specified by the question ‘Who controls it?’ Furthermore, ‘day of the week' only matters for A04. The diagnostician concluded that this ‘global variable' is only ‘relevant' for A04, since interviews had shown that there are staffing problems on particular days. The location of this global variable in the OCD was in this case specified by its relevance.
In the case study, the researcher / diagnostician did not have the same background information as the employees of Merck. A DEMO analysis allows a diagnostician to raise critical questions about the construction of the organisation. These questions lead to a profound understanding of what is essentially going on in the situation described. For example, the dependence of the self-activating actors (i.e. A02, A04 and A10) on information from information banks cannot be inferred from statistical results. A DEMO analysis offers a diagnostician a way to deal with the lack of background information and draw conclusions. In this case we have shown (related to question ii) that background information, information from the DEMO model and statistical analysis results can lead to an explanation. As for question (iii), we notice that understanding the operation of a causal mechanism is close to ‘establishing a ‘theory.’ The approach applied in the case study shows a reciprocation between the theory of DEMO (the Ψ-theory) and establishing a ‘theory’ about the planning mechanisms in a production environment. On one hand, the generated theory – the causal mechanism being the misalignment between the planning philosophies of the selfactivating actors – is only applicable for Merck. On the other hand, the experience with the Ψ-theory helps us to adapt it for application in new lean six sigma initiatives.
The paragraphs above have explained the flexibility of interpreting associations in DEMO aspect models. This flexibility allows a diagnostician to adapt to the situational circumstances wherever different kinds of variables are observed. This flexibility raises questions about the validation, i.e. which methodological aspect in our approach offers the necessary validation when explaining?
We will use the distinction on the basis of three interrelated aspects: (i) statistical, (ii) epistemic, and (iii) ontological as we reflect on our experiences in the Merck case. The first aspect, statistical evaluation, is included in our triangulation approach. The DMA steps in lean six sigma offer the necessary support to guide the process of organisational analysis. The associational model is the result of organisational analyses. Its reliability can be increased by considering the relevant aspects when reading variation in a population or between populations. We conclude that nothing prevents a diagnostician re-evaluating the associational model by conducting new measurements, and that statistical evaluation exists in our approach. We also observe (ii) epistemic evaluation in our case study. Epistemic evaluation is validation on the basis of asking whether the associations correspond to employees’ background knowledge. In the case study, we showed the associational model to the employees, and they recognised the findings. However, we can suggest improvements to our procedure. One suggestion is to also show the involved employees the weaker associations and allow them to suggest variables that should be included in the final associational model. We cannot exclude mistakes in the statistical evaluation, and some variables may mask other variables that would be more significant than the selected variable (e.g. the variable day_of_the_week can hide the variable staffing_level). On the other side of triangulation, the DEMO analysis approach was subject to epistemic evaluation. In fact, epistemic evaluation is part of the DEMO analysis since background information from the involved employees is the material from which DEMO aspect models are build. On this side of the triangulation, the modelling is an epistemic evaluation.
Statistical evaluation (i), and epistemic evaluation (ii) existed in the case study in methodological terms, the DMA sequence and achieving coherence between background information from employees and the organisational model existed in both case studies. But in the case of ontological evaluation (iii), we see differences. DEMO is an ontological approach due its prescriptive approach to processing information about the organisation using the Ψ-theory and its axioms (Dietz 2006). It is claimed that correctly applying the Ψ- theory and its axioms will ensure that the organisational modeller achieves objectivity and only captures the essence of the organisation free from any implementation details (e.g. which information systems are used). Thus a DEMO model is not an interpretation of the modeller, it represents the construction of the organisation in its most essential form. We experienced this support in the Merck case. Ontological validation can be achieved when another DEMO expert is invited to process the captured background information. We are convinced that if the Ψ-theory and its axioms is applied on the same background information about the organisation, it will lead to the same DEMO aspect models.
If we consider the topic of ontological evaluation in step 3 of our approach, ‘the integration of models’ we found ontological homogeneity between the variables acting in mechanisms. We positioned each variable in an ontological context when mapping them into the DEMO aspect models. This ontological homogeneity between the concepts of DEMO and the variables is a combination of ontological validation with epistemic evaluation (asking which elementary actor is responsible for controlling its value). A serious weakness of this approach is that the reliability of this mapping depends on the procedure that is followed to obtain mapping information. However, in our experience it is only in rare cases that a mapping leads to a discussion. Constraining the mapping technique with the help of three mapping rules seems to contribute to a reproducible and relevant augmented organisational model.
Summarising our reflection we experienced the following. The approach of combining lean six sigma and DEMO was in this case not blocked by the kinds of variable involved or the characteristics of modelling concepts. The two evidence- gathering processes on the two sides of triangulation (the organisational analysis part of lean six sigma and the DEMO analysis) can be conducted in parallel and independently. We observed that validation processes occur on both sides of the triangulation and in every step of the suggested approach. The explanations in this diagnosis approach are subject to statistical, epistemic, and ontological evaluation. All three evaluations were present in this case, respectively from using the lean six sigma approach [2], from the DEMO approach (and its Ψ-theory) [5], and from the coherence between the types of evidence obtained (the associational and interaction models) [6].