According to Capability Maturity Model Integration for Development (CMMI-DEV), organizational process performance (OPP) area aims to establish and maintain a quantitative understanding of performance of selected processes. Many organizations have started their agile transformation to leverage the capabilities of their software development processes. In order to measure the effects of agile transformation, it is important to understand to what extent agile methods contribute to business objectives. Hoshin Matrix is a tool that provides a rationale for selecting processes identified for process performance analyses. Moreover, this rationale includes traceability from software development execution to business objectives. This study describes the implementation example of an adapted version of Hoshin Matrix at Huawei Turkey Research and Development Center which aims to establish its organizational quantitative quality and process performance objectives based on a set of business objectives. Our preliminary findings suggest that our OPP design may provide a guideline for software development organizations who are willing to adopt agile transformation and for those who would like to improve their software development processes.
1Huawei Türkiye Araştırma ve Geliştirme Merkezi, 2Bilgisayar Mühendisliği, Çankaya Universitesi, Ankara Türkiye l {zulal.akarsu, ozgun.onat.metin, yasemin.yigit.kuru}@huawei.com
2 myilmaz@cankaya.edu.tr Özet. Bütünleşik Yetenek Olgunluk Modeli (CMMI-DEV)’e göre, organizasyonel süreç performansı (OPP) alanı, standart prosedürler dizisinden seçilen süreçlerin performansının niceliksel bir anlayışını oluşturmayı ve sürdürmeyi amaçlamaktadır. Pek çok kuruluş yazılım geliştirme süreç yeteneklerini geliştirmek için çevik dönüşümlerini başlattılar. Çevik dönüşümün etkilerini ölçmek için, çevik yöntemlerin iş hedeflerine ne ölçüde katkıda bulunduğunu anlamak önemlidir. Hoshin Matrix, süreç performans analizleri için belirlenen süreçleri seçme yöntemi sağlayan bir araçtır. Bu seçim, yazılım geliştirme uygulamasından iş hedeflerine kadar izlenebilirliği sağlayacak şekilde gerçekleştirilir. Bu çalışma, Huawei Türkiye Araştırma ve Geliştirme Merkezi'ndeki Hoshin Matrix'in uyarlanmış bir versiyonunun uygulama örneğini tanıtmakta olup, organizasyonel niceliksel niteliğini ve iş hedeflerine dayalı süreç performans hedeflerini oluşturmayı amaçlamaktadır. Ön bulgularımız, OPP tasarımımızın çevik dönüşümü benimsemeye istekli olan ve yazılım geliştirme süreçlerini iyileştirmek isteyen kurumlar için bir rehber olacağını göstermektedir.
Anahtar Kelimerler: Yazılım Geliştirme, Çevik Dönüşüm, Organizasyonel Süreç Performansı, Hoshin, CMMI-DEV Seviye 5 1
A software development organization should strive a high productivity culture which
delivers the desired high performance. Organizational process performance (OPP)
derives to establish and maintain a quantitative understanding of the performance of
selected processes in an organization’s set of standard processes [
Lean approaches such as Hoshin, are gathering momentum for agile thinking and
methods. Hoshin Planning (also known as Hoshin Kanri or Policy Deployment) is an
inherited continuous improvement process that provides standardized tools for step by
step organizational planning for strategy [
The primary aim of this paper is to describe the implementation example of an adapted version of Hoshin Matrix which aims to be a guide for software development organizations, particularly who are willing to adopt ASD and ultimately also for who would like to improve their software development processes. The remaining part of the paper proceeds as follows: Section 2 gives definitions for key process indicators and metrics. In Section 3, Hoshin Process and Hoshin Matrix are explained and steps of using a Hoshin Matrix is given. Section 4 gives an example of implementation of adapted version of Hoshin Matrix in ASD. In Section 5, we discuss the preliminary findings and its effects. 2
Key Performance Indicators (KPIs) are a set of quantifiable measures that are used to
evaluate the success of an organization and/or of projects, programs, products and
other initiatives [
The Hoshin Kanri is a method which was developed by Akao [
The creation of the matrix can be done by organization leaders and/or any manager who is implementing it on a team level. The initial vision, goals, objectives, KPIs, and measures along with the processes used to realize them will be decided. The basic steps to set up a Hoshin Matrix are given as in the Figure 1. The steps are as in the following:
Figure. 1. The basic steps to construct a Hoshin Matrix In order to select metrics that are driven from the high level business objectives, a rating of influence is given by voting of the workshop participants in following order: 1. Give rating of influence to each process objective against each business objective, 2. Give rating of influence to each sub process objective (process y’s) against each process objective. The important process outcomes (Y) are a result of the drivers (x) (Y = f(x)), 3. Rated and eliminate controllable factors (process x’s). Give rating of influence to the selected set of process x metrics against process y's, 4. Select the process x's with highest score.
The following part gives the steps to construct a Hoshin Matrix in detail.
At the left quadrant of the matrix identified business objectives are listed. Business objectives are the long or medium term goals of an organization in order to achieve the mission and vision. In the annual strategic planning phase, senior leadership sets objectives for the organization by assessing what has been done in the past, latest yearly plan and results, the challenges of the industrial/external factors, current business environment and short to medium term plans. Each business objective are voted for their relative importance. This is done by discussions and voting among workshop participants that are business leaders. The relative scale of rating has been considered based on the guidelines on Table. 1.
Table. 1. The relative scale of rating for influence levels
Low Influence 2 Medium Influence 5 High Influence 7 Highest Influence 10
The next step is to identify key process objectives that will cause the organization to reach its strategic business targets. These process objectives are put in the top quadrant of the Hoshin Matrix. After the long-term goals are all set, list the most important process level objectives that you aim to achieve in a shorter time frame for example in one year. Provide rating of influence of process objective against each business objective. Voting members should consider what needs to be accomplished first to keep the processes on track. After the voting session each participant explains the reason of her vote and after a brainstorming session rating of influences are assigned. Subsequently, the sum product of the given rating of influence to the process objectives and business objectives needs to be calculated. If the selected product’s value is the highest or very close to the highest one, it is ranked as 10 (the highest influence). The influence reduces when the related process’s rating reduces.
The right quadrant will be filled with the process Y’s on our customized Hoshin Matrix. In the Six Sigma methodologies process improvements are based on scientific and structured problem-solving approach Y = f(x). This mathematical term is simply means that the process output measures (Ys) are a result of the drivers (x’s) within processes. Initially, we need to understand the process Y’s and how to measure them. These factors constitute the most important key process indicators (KPI) which are quantifiable measurements that reflect the critical success factors of an organization. The target for process Y’s may differ depending on the organization’s mission and products. On the Hoshin Matrix, the correlation between process objectives and process Y’s are also highlighted by assigning the rating of influence that is calculated by the explained method to the sub process objective against each process objective.
The lower quadrant is for selection of the process X’s metrics. Process X’s are the process inputs that are identified for arriving at process Y’s. In order to select the potential controllable factors (process X), sub processes of a process (e.g. development, testing) are listed and process x’s are prioritized and voted based on the following criteria and given rating scale on Table. 2.
Table. 2. Prioritization criteria for potential controllable factors for processes and the rating scale Criteria Rating Scale Impact on Business Objectives & 5 - High Impact Project Success 3 - Medium Impact
1- Low Impact Extent of Process Definition 5 - Well Established Process 3 - Process can easily be defined 1- Difficulty in Process Definition and implementation.
Extent of Data Availability 5 - Established Data Collection System 3 - Data Collection can easily be started 1- Difficulty in Data Collection/ It is not cost effective.
Extent of Process Definition 5 3 1
Impact on Business Objectives &
Project Success 1 5 3 1 1 3 45 27 9 3 5 125 75 25 5 Extent of Data Availability
Figure. 2. The multiplication matrix for controllable criteria prioritization rates
Historically, Huawei Turkey Research and Development Center (HTRDC) started its presence in Turkey with a strong Software Development & Testing under the framework of CMM. Subsequently over a period of years spreading its business capability and ownership several pioneering initiatives are implemented like CMMI. Within a few years, it has improved its software development process maturity and was certified with CMMI Level 3 and CMMI Level 5. It is also inevitable for HTRDC to get ASD’s technical and commercial benefits. As HTRDC is an organization that has maturity level 5 of CMMI-DEV, it aims to get productivity gains and strengthen the culture of value focus by applying ASD. In HTRDC, quality and operations department is responsible for conveying a workshop to construct the matrix for organization level and selection of most critical process metrics for organizational improvement projects, and for constructing process performance model (PPM).
The business objectives are updated annually for HTRDC considering the long term strategic plans. The organizational leaders formulate strategic business objectives every year for achieving the mission for what the organization is working towards. These are set on the left quadrant of the matrix are as in the following: 1- Budget Optimization: Deliver projects within Budget and Usage Rate should be greater than 85% 2- Customer Satisfaction (CSAT) should be greater than 85% 3- Achieve Zero Critical Quality Accident Relative Importance of the business objectives are voted by the workshop participants by considering their effects to process objectives. The voted annual business objectives are voted at HTRDC and the results are listed in the Table. 3.
Table. 3. Annual business objectives with decided relative importance
Budget Optimization: Deliver projects within Budget and Usage Rate should be greater than 85% Customer Satisfaction (CSAT) should be greater than 85% Achieve Zero Critical Quality Accident
Process objectives are selected in consideration of the goals to achieve in a shorter time frame. These are sometimes called top level improvement priorities. The process objectives aligned to business objectives on the table X2 are listed as in the following: 1- Deliver with HTRDC Quality Standard 2- Improve Engineering Capability to deliver within budget and on time Rating of influence of process objective against each business objective are voted by members by considering what needs to be targeted primarily to keep the processes on track. The calculated score of sum products of the given ratings and the relative importance set for the process objectives as explained on the Step 2 of the section 3.1. The relative importance is showed on the Figure. 3 which is showing the completed left and upper quadrants of the Hoshin Matrix.
e e cn iltva troa eR pm
I S E V I T C E J B O S S E N I S U B 7 10 10 7 10 7 7
The process output metrics (Ys) of HTRDC for ASD are listed on Table. 4. These are the quantifiable results of the drivers (x’s) within processes that reflect the critical success factors of an organization as explained on section 3.1 Step 3. Rating of influence are assigned to each sub process objective against each process objective by votes of workshop members. This rating score defines the relative importance of process outputs (Ys). The right quadrant is filled with the process Ys and the ratings which can be seen on Figure. 4. The two process Ys that are highlighted with blue are selected to be used at Process Performance Model (PPM) for prediction and process control.
Importance <<Provide rating of Sub Process Objective against each
10 10 10 10 7 7 7 7 8 7 7 7 10 10 7 7 E R O C S
Post Delivery Defect Density (System Integration Test+Customer Reported)
Base Code Testing Productivity
Base Code Quality Defect Density Sprint Level Base Code Quality Delivered Open DD
Sprint Level Delivered Open Defect Density
Number Of Trouble Tickets Rejected In Regression Test HTRDC Agile Software Development sub processes are analyzed and some of them are listed as in the Figure. 5. The determined threshold to select sub processes the are selected. The selected sub processes are namely; user story development, user story development review, user story development test, and user story system design verification (SDV) test. The relation of these sub processes are given on the right column of the Figure. 5.
Process Selection - Development Processes Process Selection Criteria - Quality &
Ontime Delivery
Extent of Extent of Process Data
Definition Availability The rating of influence of the process Xs against each Process Y’s are given on Figure. 6. Total score is calculated based on the given influence rating of process X on process Y and relative importance of process Ys. The process X Metrics are agreed to be selected by the workshop participants if the total score is 300 or above. This threshold defined by the organization.
No
Name of Sub Process ilrIssaeeeenoRDV litrfeeeeeecpndvDOD itseynD ttrfseaeecnowDDm tiseynD tiitrcdouv2yPEE litiraaedvokonoDW lileeaecdhupgpSS 10 10 10 10 10 8 8 The explained adapted version of Hoshin matrix is introduced to projects by the quality and operations department of HUAWEI Turkey R&D Center. HUAWEI Turkey R&D Center is a large software development company works in Turkey with around 500 engineers. With the CMMI Level 5 maturity lean methods are used in process management that are applied for controlling and monitoring the processes based on statistical data. Benefits of using Hoshin Matrix, in defining and mapping organizational goals has been observed at HTRDC. Since the business objectives are monitored and controlled by selected process x metrics that are mapped with the defined organizational goals, the risk to misinterpret the results and lose organizational alignment is decreased. This risk is manageable by the annual/half yearly evaluation of the process performances. This control mechanism has provided to be early in taking action and adjust the business goals and/or process improvements faster. Initial implementation of Hoshin Matrix was applied on traditional V-model Software Development Lifecycle at HTRDC. In the interest of improving business capability and productivity, agile transformation has been employed together with CMMI. As a result, Huawei Turkey made significant contributions as it has been managing its ASD processes and high product quality with lean management methods and CMMI Level 5, the highest level of maturity. Selected
Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Huawei Turkey Research and Development Center, Quality and Operations Department has been working on conduction of lean methods such as Hoshin Kanri. We would like to express our gratitude to whom gave their help and generous support during this research.