Project managers collect data on the performance of their projects in order to be able to report the status and to forecast future performance. The Standish Group recently surveyed 400 organizations and reported that only 32% of information technology projects were successful, with close to a quarter of the projects reported as failures (Levinson 2009a). Articles in CIO magazine point out that poor requirements and scope management contributes to these failures (Levinson 2009b, Levinson 2009c). Much has been written about how to manage scope, from improving business cases by establishing clear objectives, to ensuring requirements specify an acceptance criteria, to change management processes. The measuring of scope status has largely been ignored because of the difficulty of measuring requirements. This research looks at the fuzzy nature of the inputs to status reports for the scope constraint to answer two questions:
• Can fuzzy systems offer a tool that can capture the status of the scope of an individual activity in an IT project?
• Can the scope status for activities be aggregated into a meaningful project scope status?
It is anticipated that from the first question it might be possible to determine if there is a common or generally accepted understanding amongst project managers as to how to report status when the inputs are vague or imprecise for an activity. The Z-mouse tool proposed by Lotfi Zadeh, which is an extension of Zadeh's work on fuzzy set theory (Zadeh 1973), is a leading edge data collection mechanism that will be used as the data collection tool in this study.
Weill and Broadbent have stated that information technology (IT) is "very strongly project based" (Weill 1998). Understanding the status of a project is important to project managers, upper management, and executive sponsors of a project. There are many stakeholders outside of a project's organizational structure also interested in the status of a project (PMBOK Guide 2008). The overall project status many times is seen as an aggregation of the status of the three traditional project constraints: cost, schedule, and scope. Depending on the project, the fourth constraint of quality could also be present (PMBOK Guide 2008).
Further, the overall status of each specific constraint is an aggregation of the status of each activity status considering that constraint. For example, each activity is examined to judge how it is meeting the cost constraint; the project status related to cost is an amalgamation of all the individual activity cost-related statuses. The project status for schedule is similarly determined by first considering each how each activity is performing with respect to that constraint. Finally, the overall project status is established by combining the individual constraint statuses. Thus, for the entire project, the constraints are dimensions that are evaluated independently and then later aggregated.
As mentioned, the overall project status should be determined by aggregating the individual activity statuses. Instead, it is often reported as the opinion of the project manager, which is subjective. Recent postings on the LinkedIn internet site for professional project managers requesting help on project status met with a wide variety of rapid responses indicating the high interest level of practicing project management professionals.
Since executive managers tend to focus on problem areas, this translates to projects in trouble and as a result, there is a tendency to under report status. Snow and Keil investigated variance between the true status of a software project from the reported status and found that accuracy was a major problem. "The intangible nature of software makes it difficult to obtain accurate estimates of the proportion of work completed, which may promote misperceptions regarding project status" (Snow 2001).
Snow and Keil found that in addition to misperceptions in the status of a software project, project managers might also censor the status reports of poorly performing projects. They cited an example of a project that lost $125 million over 3 years, yet senior management did not have any insights into the problems. "The combined effects of project manager misperceptions (errors) and bias in reporting leads to what we call "distortion" in the project status information received by senior executives" (Snow 2001). Snow identified the need for better tools for understanding project status, and the necessity to automate the reporting of status to avoid project manager bias and reporting errors. With other research projects focused on schedule and cost constraints, this study investigates scope.
Projects by definition are unique, and "because of the unique nature of projects, there may be uncertainties... The project team must be able to assess the situation and balance the demands in order to deliver a successful project" (PMBOK 2008). Assessments are the feedback during the execution of a project so that the project can be guided to a successful completion. As projects move forward, project managers are constantly gathering data on the status, converting that data into useful information to be reported, and then acting upon the information. Often the data is vague, or needs interpretation. An example of vague data is that it is difficult to determine to what extent the scope is being met. To label project scope as 67.35% met is recognized as impractical precision. The imprecision in the data is the subject of this study, and rather than using traditional methods to attempt to quantify scope, computational intelligence offers new tools and techniques for capturing vagueness.
Computational intelligence tools can "identify semantically ambiguous concepts and convert them to fuzzy sets" (Cox, 1999) which can then be resolved into solutions that can be handled by project managers.
With over 300,000 members, the Project Management Institute (PMI) is recognized worldwide as an authority on project processes. Their Project Management Book of Knowledge (PMBOK) does not spell out the format of status reports, nor does it tell project managers specifically how to write a status report. Instead the PMBOK identifies processes, defines inputs, tools and techniques, and the data flows that tie the processes together (PMBOK 2008). The PMBOK, as stated in section 1.1, is an assembly of good practices that has the consensus and general agreement of project management professionals. The PMBOK "is a guide rather than a methodology. One can use different methodologies and tools to implement the framework" (PMBOK 2008). This gives practitioners the flexibility to choose techniques that work for their given situation.
The Project Management Institute's PMBOK identifies the performance reporting process as part of their Monitoring and Controlling process group (PMBOK 2008). The PMBOK lists three outputs from the performance reporting process: 1) Performance reports, 2) Organizational process assets updates, and 3) change requests. The purpose of the reports is to act as feedback into the processes that "track, review, and regulate the progress and performance of the project; identify any areas in which changes to the plan are required; and initiate the corresponding changes" (PMBOK 2008). To this extent, data is converted into actionable information guiding the project to completion.
This process of reporting performance is crucial to initiating corrective actions and preventive actions, and becomes part of the organization's lessons learned historical database.
When reporting the status of projects, Dow and Taylor have found that project dashboards are often used by senior managers (Dow 2008). Dashboards are a graphical summary of the status of a project, many having a drill down capability. The purpose is to give a quick, high level overview of a project to upper management whose role is to prioritize, review, and make funding decisions (Benson 2004). Dow and Taylor state that two constraints of project management, cost and schedule, are evaluated independently and summarized. It is interesting to note that they make no mention of scope. They also found that to assist with quick problem identification sometimes a stoplight report is produced where each area is assigned a color to represent the status of that constraint. Typically the stoplight colors of red, yellow, and green are used to represent the status of each constraint (Dow 2008). These constraint statuses will be aggregated into a cumulative status for the project (Barnes 2009). Green-Yellow-Red traffic light status reporting is widely used because of its simplicity, and the quickness with which people can identify if there is a problem that needs addressing. This traffic light technique is in common use many projects, and especially popular in status reports to stakeholders who might have little time or inclination to understand the project details. Performance reports are essential inputs necessary to monitor and control a project (PMBOK 2008), but the dashboards get the attention of the executives.
It would seem that numerical inputs into reports and dashboards should yield an objective status for reporting purposes. The ideal ought to be that for a given activity, the fixed numerical data goes in and a Green, Yellow, or Red project status comes out. The next stage in the process would be that the individual activities are then mechanically aggregated into an overall project status. The reality is that there are many factors that influence the decision to label a project status with a particular status value for a singular activity, and that the aggregation of those statuses for multiple activities of the critical constraints is open to interpretation as well.
When the project status is not a clear green or red, Barnes and Hammell found that "ambiguity is present in the scenario where the expert had to decide that the status of a project is Yellow" (Barnes 2009). Looking at the case of rating just one of the activities in a project, it is simple for status green. Most managers would look at a truly green activity and agree that the status is okay, or green. Beyond green status, it becomes questionable. Barnes has shown that yellow status can be misinterpreted or communicated as green.
The problem is much worse when the project is in serious trouble. Snow and Keil found that IT project status of red is frequently misreported (Snow 2002). Projects that are failing need the most attention from the executive management team; yet, without the knowledge that the status is red, the proper level of actions are not taken to bring a red project into compliance which often leads to financial disasters. The magnitude of project failures is alarming. For example, barely ten years ago The San Francisco Chronicle reported that the state of California wasted over $1 billion on failed computer automation projects (Lucas 1999).
The second stage in reporting status, aggregation of the constraints into an overall project status, has been studied by a number of authors. But there are complexities that make the automatic summarization difficult. For example, a project that is ahead of schedule might also be significantly over cost at that point in time. What is the true status of that project? Just looking at the raw data might yield a green status on schedule, but a red status for cost. However, the costs might reflect that fact that the project is ahead of schedule, so it might be the case that the project will finish ahead of schedule, and ultimately within cost constraints. Ahead of schedule, and meeting cost constraints when completed would seem to mean the project is "green", in spite of a "red" cost. This implies that making status a simple mechanical output of numeric inputs can produce status errors. Human intervention is required to interpret the data into meaningful information.
The cost and schedule constraints of project management have numerical quantities that can be measured. The numbers have an element of objectivity which can be used in forecasts. Econometric methods such as regression analysis and autoregressive moving averages, or time series methods such as linear prediction, trend estimation, and moving averages have been used by practitioners of project management (PMBOK 2008). Currencies are tracked and reported using time series methods such as earned value (PMBOK 2008). Calendar dates and/or labor hours can be tracked for the time constraint. Depending on the project, quality might also be measurable and reportable.
Scope, however, is much more difficult to measure, and at the same time is the critical element from which the time and cost are derived. Richardson and Butler stated that "the concept of project scope is a foundation idea. It establishes the base for much of the subsequent management activities" (Richardson 2006). At a high level overview of the project management processes defined by the PMI, scope is derived from the project charter and requirements, the scope baseline then feeds into the Work Breakdown Structure (WBS). The WBS is the input to time management, which was an output of the scope definition being decomposed into activities. "Activities provide a basis for estimating, scheduling, executing, and monitoring and controlling the project work" (PMBOK 2008).
In a similar manner, scope and the WBS feed into cost estimates and cost management. This means that if the scope is wrong, the time and cost estimates will be wrong, or if the scope changes then time and cost can be severely impacted. Time and cost estimates are calculated by activity, but the list of activities comes from the scope definitions and WBS that were completed early in the life cycle of a project (Gido 2009). This implies that scope and requirements errors early in a project can carry over into constraints that are perceived to be more objective, such as cost.
Schwalbe states that managing scope is especially difficulty on IT projects. Scope can be relatively undefined at the beginning, can grow out of control due to creep, and suffer from an inability to verify (Schwalbe 2010). Textbooks on project management will point to cases such as the bankruptcy of FoxMeyer Drug in 1996 due to an IT project that had scope problems (James 1997 andScott 1996) Weill and Broadbent have stated that projects are late sometimes due to specification changes, or new business needs that occur during the project (Weill 1998). This event, called scope creep, impacts the other areas that management tracks for status reporting. The criteria that are more readily measured by objective criteria (time, cost, and resources) are directly impacted by scope creep (PMBOK 2008).
The uncontrolled changes of scope creep add costs of which a customer might not approve, delay schedules, and reroute critical resources.
The IT industry is full of examples of scope creep. A Google search of the term "project scope creep" produced over 4 million hits. A quick review of just a fraction of these web sites demonstrates a common assumption: that a project manager knows exactly and precisely the scope, and that the problem is that the scope changes or grows. This is a questionable assumption.
Fleming and Koppelman, major advocates of the deterministic Earned Value model, admit that "earned value accurately measures project performance, but must assume that scope definition is adequate" (Fleming 2010). Many sites are devoted to advice about managing scope through a change control process, a respected technique, but this assumes that the scope is well defined, and that the changes are recognized. In reporting project status the ascertaining and reporting of scope status is critical, and yet lacks a clear and measureable standard. Stakeholders and executives have difficulty making decisions based on vague, subjective, and imprecise inputs. To put it simply, scope is fuzzy. Scope and the corresponding set of requirements are a collection of words describing an end product, and whether or not the deliverable meets the requirements can be open to interpretation.
Computational intelligence (CI), implemented in a variety of soft computing techniques, has allowed the automation of the handling of vague and imprecise data. Computational intelligence offers a revolutionary set of tools capable of responding to fuzzy, inaccurate inputs. This research envisions that these tools and techniques can be effectively applied to project status assessment. This study concentrates on Information Technology (IT) projects, in particular the scope constraint, because of the inherent lack of a measure for scope.
The IEEE Computational Intelligence Society defines CI as a number of core technologies, among them fuzzy systems, neural networks, evolutionary programming, and genetic algorithms (IEEE 2011).
These technologies build intelligent systems to help with complex problems in which the information and data are vague, approximate, and uncertain. For this research computational intelligence will focus on fuzzy logic as applied to project status. In order to put a reasonable boundary around the subject, only project scope status will be evaluated.
Lotfi Zadeh proposed the concept of fuzzy variables that are linguistic in the 1960's. For project cost these linguistic variables might be (costs = {over, on cost, under}) (Li 2006). Fuzzy systems can replicate human decision making by handling vague data, to the point of coping with noisy and/or missing data (Yen 1999) Example: "hot", not 85 degrees Nonlinear and changeable Analog (ambiguous), not digital (yes/no) Zimmermann expanded upon Zadeh's description of fuzziness as that of possibility, with the idea of a possibility distribution. Zimmermann's example is that a fuzzy set F~ = { (1,1), (2,1), (3,0.8) } has a possibility distribution such that 0.8 is the possibility that X is 3 (Zimmermann 1996). The possibility distribution thus allows for something to be both "true" and "fairly false" at the same time. This concept is the basic question that will be asked of the experienced project managers in this research: is it possible that the measurement of scope is inherently fuzzy, and therefore does it make more sense to use tools and techniques that can capture the fuzziness associated with scope status.
Some authors have suggested that in spite of objective and measureable numbers in cost and time constraints, there can be fuzziness in the interpretation of those numbers. Li, Moselhi, and Alkas proposed a forecasting method for cost and schedule constraints using Fuzzy Logic to compensate for the variability found on construction projects. They looked at four different, generalized methods to forecast project status (Li 2006). The first were stochastic methods that assumed each unit of work has a mean and standard deviation, but according to Li, et al, these methods are weakened by variability in costs per reporting period. The second methods were deterministic, such as earned value. The third method that they looked at was social judgment theory based, using human judgment in lieu of mathematic methods. The last method was their proposed use of fuzzy logic for project forecasting and status (Li 2008).
Other researchers have applied computational intelligence tools to project management for schedule and time control. Jin-Hsien Wang and Jongyun Hao proposed a Fuzzy Linguistic PERT (Program Evaluation & Review Technique) to replace stochastic methods that use means and standard deviations. They assert that too much data may be needed to obtain the random variable distribution, so fuzzy methods are more applicable (Wand 2007). Wang and Hao expanded PERT/CPM (Critical Path Method) by storing each activity duration as a fuzzy set. Klakegg, et al, have already analyzed what should be measured in projects, and at the same time they acknowledge that warning signs of problems are "often unclear and imprecise" (Klakegg 2010). They describe what, but not how to measure the subjective constraints. While other researchers have proposed how fuzzy set theory can be integrated into project management across the time and schedule constraints, this work focuses on the scope constraint. Additionally, we will use a CI tool to capture scope status directly from experts in a more realistic, human friendly form. Once the status is captured, it can then be aggregated into an overall scope status for a project. Without an objective criterion such as currency spent or elapsed time, scope is difficult to measure. Fuzzy systems allow the capturing of this subjective data, and then the aggregation using recognized fuzzy set mathematics.
This study proposes to use computational intelligence (CI) tools, in particular alternative tools like fuzzy logic, to understand the status of a project's scope. This use of CI is in contrast to the more conventional bivalent logic, which Zadeh described as working well with exact numbers, intervals, and probabilities. Rather than the hard, crisp nature of bivalent logic, these CI alternatives have been sometimes labeled "soft computing."
Zadeh stated that "it is a common practice to ignore imprecision, treating what is imprecise as if it were precise" (Zadeh 2009). The computing power available in the 21 st century allows for the implementation of the concepts that Zadeh called computing with words (Zadeh 2009). Given the imprecise nature of project scope due to the linguistic nature of requirements, it makes more sense to use fuzzy intervals and fuzzy sets to capture the essence of the status of scope.
In his acceptance speech when receiving the Ben Franklin award at Villanova University in 2009, Zadeh provided an analogy for fuzzy logic:
In bivalent logic, the writing/drawing instrument is a ballpoint pen. In fuzzy logic, the writing/drawing instrument is a spray pen-a miniature spray canwith an adjustable, precisely specified spray pattern (Zadeh 2009). Zadeh has stated that a valid application of fuzzy logic is in the handling of imperfect information. At Villanova Zadeh went on to say that "imperfect information is defined as information which in one or more respects is imprecise, uncertain, vague, incomplete, unreliable, partially true or partially possible" (Zadeh 2009). This leads to the core concept that membership in a fuzzy set is a matter of degree. For project managers, when looking at the status of a given line item's scope using fuzzy logic, that status is allowed to be a matter of degree. In practical terms this means that the scope of an item can be of status mostly yellow, and at the same time that same scope item can be of status a little red.
This study uses a fuzzy data collection tool proposed by Zadeh, colloquially referred to as the Z-mouse. This tool is a spray paint web gadget that implements Zadeh's spray paint analogy. Jose Barranquero and Sergio Guadarrama created the Z-mouse to gather fuzzy opinions, or perceptions as they call it, from users (Barranquero 2010).
In their work, they give users an English language word and ask the participant to rate that word on a scale using the Z-mouse.
This study builds upon their prototype by evaluating the fitness of their Z-mouse concepts when applied to project management. Project managers are asked to rate the scope for a WBS activity on a scale that is words, not numbers. It is anticipated that the non-numeric scale will be quickly recognized and easy to use by experienced project managers. Figure 1 illustrates the Z-mouse web gadget using a non-numeric, linguistic scale.
Figure 1. Linguistic scale for project scope Barranquero and Guadarrama go on to state that the Z-mouse can be easily learned by non-expert end users (Barranquero 2010). This could lead to a design where the individuals doing the work of a WBS activity would input their opinions on the scope status, which would be passed on to the project manager and stakeholders. The scope status would be seen as a measurement that is analogous to cost and schedule measurements. Since errors in scope lead to errors in cost and schedule, the awareness of scope problems should contribute to early corrective actions, increasing project success.
In contrast to fuzzy systems, social scientists have used psychometric scales extensively in survey research. In Likert scales the survey participants are asked to select one number from a variety of choices. Many times these choices are an ordered scale, forcing the user to select one and only one value (Trochim 2006). The evaluator of a Likert scale survey can take advantage of the bipolar nature of this scheme, and apply conventional statistical tests, such as variance from a mean. Likert and other systems such as Thurstone scaling have strict rules.
One drawback from using Likert is that it cannot handle that people will perceive a given choice as falling into two categories simultaneously. Those models view this human tendency as a paradox, or a violation of the rules. A fuzzy system allows that project managers might perceive the status as mostly yellow, with some modest amount of red.
Having both statuses at the same time for one activity is an acceptable possibility in fuzzy systems. Another drawback to scaling systems is that a statistically valid number of participants are required in order to validate the data. In project management there might only be two or three participants working on a WBS activity, a number not amenable to conventional crisp probability.
This study gives participants a description of an activity and asks them to evaluate that activity for the status of the scope. Figure 2 gives an example that is illustrative of the types of questions in this survey.
Figure 2. Sample project activity Figure 3 is an example of a potential response using the Zmouse. The individual inputting the status would select one of the four shades of grey from the pallet, and then paint the status bar where they think appropriate. If they want to indicate a lesser importance, then they would select a lighter shade of grey. Figure 3 would be an example of a project manager deciding that scope constraint was mostly yellow, yet leaning towards red. It should be pointed out that these spray paint data points are converted to numeric values, and then evaluated using the strict mathematical rules of fuzzy sets.
The next step is to aggregate the individual scope statuses for each activity into an overall status for a project. Since the origins are the fuzzy words (green-yellow-red), the proposed aggregation method would be an implementation of Zadeh's computing with words. Zimmermann offers three common methods to aggregate the individual inputs: COA (center of area), COS (center of sums), and MOM (mean of maxima) (Zimmermann 1996). This study will use COS to aggregate the fuzzy sets into the crisp value that will be reported at the overall status. Klir states that COS is the most common method to find a value that represents the overall conclusion. The COS calculation is based on recognized and accepted mathematics for fuzzy sets, which can be found in textbooks by authors such as Klir (Klir 1997). The COS solution finds the geometric centroid for the aggregated first moments, and then translates the solution value into a status.
Professional project managers have objective data for the time and cost constraints on their activities. With the introduction of an input tool to capture the status of scope, the measuring and reporting of subjective opinions of scope status can be done. The next step would be that each and every WBS activity would have a scope status that could be aggregated into an overall project scope status. Based on the gathering of this scope data it is expected that this would become the third constraint in a fuzzy system such as the one proposed by Li, Moselhi, and Alkas that only addresses cost and schedule. Since IT projects are unique and, thus have vague and imprecise scope requirements, it is believed that two questions will be answered in the positive by this study: 1) fuzzy logic can provide a tool for measuring scope of individual activities, and 2) the fuzzy scope can be aggregated into a meaningful project status.