The business process revolution has had two impacts on project management: the recognition of a process perspective (such as the 39 appearing in the PMBOK), and the acknowledgement that these processes reflect project management knowledge (such as the nine knowledge areas in the same publication). These two levels have been extended, through an architecture (HCRN - hierarchical case retrieval network), to include and interlink decisionmaking tasks encountered by project managers. Experiments indicate an adequate degree of success in being able to transform a decision situation into a knowledge focus comprising relevant cases from different case bases, and the interactions between them. The knowledge focus provides a basis for experience management of decisionmaking within project management processes.
organizing a project only provides a generic framework for knowledge capture and reuse, but does not detail the intra- and inter-process decisionmaking knowledge that would actually be associated with a specific project. Nevertheless, as these guidelines are familiar to and recognized by practitioners, they can serve as an ontology for developing a knowledge representation for experience management in the field of project management. This is demonstrated in the COPER (Components for Project Management Experience Repositories) reference model [BN01]; and, as elaborated below, in the experiential model described in this article.
Within the business process concept a specific set of processes can be differentiated: knowledge management processes. Several schemes have been proposed for these processes, either generalized or related to CBR as the central tool for their execution. Probst et al [PRR99, quoted in Mi01] propose six general stages – identification, acquisition, development (creation), transfer, use and preservation – extended by Minor [Mi01] to include targets (knowledge-based abilities to be fostered) and evaluation. Aamodt and Plaza [AP94] describe a “4-RE” scheme for CBR – retrieve, reuse, revise and retain – extended by Iglezakis and Reinartz [IR02] to include review and restore (a “6-RE” scheme).
We may divide these knowledge processes into three groupings (a “3-A” scheme): knowledge acquisition (retain, identify, acquire, develop, organize, preserve) knowledge application (retrieve, reuse, transfer, use, targets) knowledge quality assurance (revise, review, restore, evaluate) Within the framework of our research we then define “experience management” as “a special form of knowledge management which deals with task-based [and experiential] knowledge” [Mi01] that is both gained and applied when carrying out business-related tasks. Our specific tasks are those concerned with knowledge application in decisionmaking within project management; and the specific experience is that obtained and employed by project managers when making such decisions (our “target”). This aspect of experience is succinctly expressed by Watson [Wa99]: “… that knowledge is not so much a capacity for specific action, but the capacity to use information, learning and experience resulting in an ability to interpret information and ascertain what information is necessary in decisionmaking”. Some method is needed to help project-oriented organizations support their decisionmaking capability on the basis of experience regarding previous decisions – a project-specific (or episodic) experience management paradigm, building on and augmenting the generic framework [Pm00], that derives from examples of decisions taken in the past. We thus seek to extend the knowledge-oriented focus of the PMBOK, whilst providing the project manager with a practical approach to applying past experience when making new judgments. For this purpose, CBR seems to be ideal (see next section), as a heterogeneous case base provides a highly flexible format for storing the variegated knowledge associated with the nine project management knowledge areas [Pm00], and the disparate types of decisions taken by different project managers. Thus it can be expected that several types of cases will be involved in a decision; and that these cases must be linked in order to emphasize their joint relationship to the decision made. This consideration introduces a new concept: a path linking associated cases in the case base, and a technique – path-based reasoning (PBR) – to store and traverse these paths in order to retrieve multi-attribute decisions. Our intention is to describe an architecture, based on CBR and PBR, for knowledge management of project management processes at the detailed level, concentrating on decisionmaking tasks.
We adopt CBR as the knowledge representation and the foundation for experiential knowledge management for project management processes for the following reasons: • It provides many benefits over other AI-based approaches [Re99, Gr98, Aa98]: - A case base becomes useful with the first case - A case base captures knowledge easily (no need to discover complex interrelationships between cases for the same issue [see following section 4]) - Case bases are understandable (logical and easy to follow) - CBR augments human capabilities (comprehensive case storage and tracking) • The nature of decisionmaking in project management is suited to an examplebased paradigm, as different project managers have different management styles; and they would prefer to view previous situations and decisions rather than face a set of prescribed rules or models. • Likewise, project management knowledge derives from a wide range of knowledge sources, and is often idiosyncratic. Any preprocessing or processing is likely to eliminate important aspects of decisions taken. In CBR the knowledge is not preprocessed [Be99, XR99, Le98], and therefore bias is minimized. • The case format is flexible and may be modified over time without impacting the methodology. • CBR is amenable to being incorporated into a experience management process. • The CBR cycle is similar to experience reuse in project management [BN01]. • Organizational learning is similar to the CBR cycle and so can be supported by
CBR technology [BN01]. • Brandt and Nick [BN01] describe a reference model for project management experiences and their reuse based on CBR. The model components include business processes, problems and solutions, and guidelines for specific projectrelated actions. The experience-based architecture and its various levels are illustrated in Figure 1. It is an hierarchical heterogeneous case-based structure, which we term HCRN (Hierarchical Case-Based Retrieval Network), based on ideas originally described by Lenz [LE99]. The architecture and case and path bases are detailed in [KK02]; we provide a brief overview here. (a) • • (b) (c) • • • •
Project management knowledge area – management of scope, time, cost, quality, human resources, communications, integration, risk and procurement (knowledge area level) Project management process – thirty-nine project management processes (process level within a specific knowledge area)
Issue – a mapping of a specific decisionmaking task onto one or more project management processes (issue level, mapped onto the emphasized processes) Entity (attribute) –an atomic knowledge item in the issue domain such that an issue is defined by a (unique) set of entities and their domain of values (entity level) Case – represents an actual decision-making situation from the past, comprising the same entities as its associated issue, but having assigned values for most or all entities (case level)
Path –a trajectory between several issues and/or cases indicating that they have been considered jointly relevant to a decision situation in the past (path level)
Aamodt and Plaza [AP94, Figure 2] detail the “retrieve” and “reuse” stages as follows: (a) Retrieve - identify features - collect descriptors - interpret problem - infer descriptors - search (case base) - follow direct indexes - initially match - calculate similarity - explain similarity - select case(s) - use selection criteria - elaborate explanations (b) Reuse - copy solution/method - adapt solution/method - modify solution/method We propose the following flow for experiential knowledge application processes, based on case-based and path-based reasoning: (a) Capture (define and transform the decisionmaking problem or task) 1. Express the decisionmaking situation in textual form 2. Decompose the situation into one or more issues 3. Decompose the situation into one or more values for the issue entity(ies) to create a “case(s) by example” (b) Retrieve (cases related to an issue for all issues selected) 4. Search the case base, using the “case by example” 5. Calculate similarity (to the “case by example”) 6. Select relevant cases as knowledge focus components (see (d) below) 7. Juxtapose cases from different issues as implying possible knowledge area or “business process” interactions (c) Traverse (paths related to the cases selected) 8. Search the path base for paths incorporating the cases selected 9. Select relevant paths 10. Traverse the relevant paths – explaining the connection between the cases comprising the path arcs (see Figure 1 – path level) (d) Focus (concentrate retrieved knowledge to support decisionmaking) 11. Correlate selected cases and selected paths (knowledge focus) 12. Explain and interpret case-based knowledge focus 13. Explain and interpret path-based knowledge focus 14. Integrate knowledge focus and its relation to the decision to be made (e) Decide 15. Make a decision based on the knowledge focus
30 students from the Faculty of Industrial Engineering and Management at the Technion, familiar with project management principles, participated in the experiment. They were provided with a lexicon of the issues, entities and values defining the HCRN base, a set of cases for each issue, and several paths between cases. We concentrate here on three experiential transfer aspects of the experiment: • capture – decompose and transform a given scenario (steps 1 through 3) • traverse – explain causes and effects (step 10 and Figure 1) • focus – explain and interpret path-based knowledge focus (step 13) For the capture and traverse parts of the experiment (steps 1 through 3 and step 10), all 30 students carried out the steps in the same way. For the focusing step 13 the group was divided into two sub-groups. The first worked with a tool consisting of a computerized database (cases and paths) and a flow diagram of the process described above (section 5). The second worked with the database but without the guidance of the flow diagram.
management
Knowledge area Risk management planning Risk identification Qualitative risk analysis Quantitative risk analysis Risk response planning Risk monitoring and control Activity definition Activity sequencing Activity duration estimating Schedule development Schedule control
- E
Organizational planning Staff acquisition Team development Specialization System Experience Salary Analyzing Human Resource Mgt.
Beginner 10-15K
Case C16 (+,+,+,+)
C16
Risk/problem category Risk/problem Reason/trigger category Reason/trigger Corrective action category Corrective action Team Lack of experience Customer Change of requirements Time Project delay
Case R14 (a1) Step 1: scenario “Select the best candidate – beginner, experienced or expert – for analyzing a human resource management system, concentrating on the trade-off between the risks concerned with inexperience and project resources (salary and time)”. (a2) Steps 2 and 3: decomposition into issues and values
(compare Figure 1, issue and entity levels) (C)andidates: specialization = analysis; system = human resources (HRM) (E)stimation of times and costs: specialization = analysis; system = HRM (R)isk: problem category = team; problem = lack of experience (b) Step 10: explain causes and effects (compare Figure 1, path level) “Selection of a beginner with a given salary for HRM system encoding (C16: ++++) led to (a) recording of a case reflecting initial duration (1.5 - 2) and cost (30-40K) estimates (E18: ++++--); and (b) anticipating a problem of lack of experience triggered by a possible change in requirements (R14: ++++--). Foreseeing this problem led to a revision of task duration and cost (E18: ++++++); this, in turn, caused an expectation of project delay (R14: ++++++)”. (c) Step 13: explain and interpret path-based knowledge focus Path P2: A change in requirements that led to an increase in product complexity (R15) resulted in replacing an analyst in human resource analysis by an experienced worker (C2). Thus there is a connection between product complexity and the experience required.
Path P3: as the result of his predecessor (an expert in financial analysis) leaving because of dissatisfaction with his salary (C6), a different employee was allocated to a task (R13). Thus there is a risk of an employee leaving when he considers his salary to be too low.
Path P18: After the activity duration and cost were estimated (E2), a beginner in human resource systems analysis was considered suitable for the task (C1) in terms of salary and likely performance. Thus the initial budget for the corresponding hammock activity influences the choice of employee in terms of his salary.
In section 5 above we have outlined a process flow for the application of experiential knowledge, encompassing five phases. Each phase requires an ability to interpret knowledge and to transfer experience. In section 6 we have described experiments to study these skills for three of the phases: capture – the aptitude for formulating a decision situation in terms of issues; traverse – the aptitude for understanding interactions implied by a path; and focus – the aptitude for relating paths to the multiattribute decision situation. We wish to be able to measure these aptitudes, in terms of the success of the process flow in supporting the development of an effective knowledge focus.
We thus propose the following metrics for the ability to decompose a decisionmaking situation, to traverse a path, and to explain the relevance of the path-based focus: 1. Compliance metric – the ability to express the decisionmaking situation (10 points). Identification of issues and values: 3 points for issues C and E; 4 points for issue R. 2. Accordance metric – the ability to interpret nodes and arcs of a path (10 points).
Explanation of cases and arcs: 1 point each (E18 and R14 are referenced twice). 3. Convergence metric – the ability to explain and interpret the path-based knowledge focus (3 points). Explanation and interpretation of paths: 1 point each. The scores for the three metrics are summarized in Table 1.
The main reasons for student misconceptions of the experiential requirements, in descending order of occurrence, were: 1. Compliance: redundant search (non-directed entity values); over-constrained search (too many values); incomplete search (not all issues identified); misinterpretation of retrieved cases 2. Accordance: arc ignored; node (case) misinterpreted in terms of currently assigned or unassigned values (‘+’ or ‘- convention in Figure 1) 3. Convergence: knowledge focus ignored (path P3 and/or path P1)
Using the Mann-Whitney comparison test, the difference between the convergence achievement scores for the two sub-groups (2.5 and 2.1 respectively) has been found to be statistically significant (p < 0.01). We have described an experience-oriented knowledge application process for the decisionmaking task in project management. Experience is accumulated in decisions taken in the past, regarding job candidates, activity resource allocations and risks associated with project-related problems. The experiments outlined in this paper have examined three aspects of experience transfer: • The ability to translate a decision problem into its underlying issues in order to relate it to an HCRN case base • The ability to track and understand interactions between issues reflecting the multi-attribute characteristic of decisionmaking • The ability to distil a knowledge focus from retrieved cases and interactions as a basis for coming to a decision The student subjects showed a success level of about 70% for decomposing the scenario; more experience is required in formulating effective “cases by example”. Success rose to about 80% for understanding the meaning of an interaction or “path”; more attention needs to be paid to path trajectories and the creation of values along a path as the decision evolves.
Convergence to the interaction knowledge focus is seen to be far more effective when the process flow (section 5) was used by the first sub-group to guide the various phases of the experiment. This observation emphasizes the importance of the process aspect of a experience-based architecture. The lack of a “perfect” score indicates that relationships should not be regarded as “irrelevant” until the focus has been established.
These results indicate that CBR and PBR (path-based reasoning) can support project management experience acquisition and application, if project managers are given a correct understanding of these mechanisms. Further capture and traverse experiments are being carried out, incorporating “push” mechanisms regarding compliance and accordance, to ensure as far as possible that all components of retrieved cases or paths will be considered. Refinement of the metrics and scoring method used for experience transfer is also being studied.