=Paper=
{{Paper
|id=Vol-67/paper-6
|storemode=property
|title=Experience Management within Project Management Processes
|pdfUrl=https://ceur-ws.org/Vol-67/gwem2003-KK.pdf
|volume=Vol-67
}}
==Experience Management within Project Management Processes==
https://ceur-ws.org/Vol-67/gwem2003-KK.pdf
Experience Management within Project Management
Processes
Maya Kaner and Reuven Karni
Faculty of Industrial Engineering and Management
Technion – Israel Institute of Technology, Technion City
Haifa 32000, Israel
kmaya@techunix.technion.ac.il, rkarni@ie.technion.ac.il
Abstract: The business process revolution has had two impacts on project
management: the recognition of a process perspective (such as the 39
appearing in the PMBOK), and the acknowledgement that these processes
reflect project management knowledge (such as the nine knowledge areas in
the same publication). These two levels have been extended, through an
architecture (HCRN – hierarchical case retrieval network), to include and
interlink decisionmaking tasks encountered by project managers. Experiments
indicate an adequate degree of success in being able to transform a decision
situation into a knowledge focus comprising relevant cases from different case
bases, and the interactions between them. The knowledge focus provides a
basis for experience management of decisionmaking within project
management processes.
1 Project management and business processes
The “business process revolution” has introduced a paradigm shift in management –
the process view of the firm – which has swept through the corporate landscape
[HS99]. This perspective has been adopted in project management [Pm00] as
emphasized by Brandt and Nick [BN01]: “There is general agreement today that
increasing individualization of business performance and business processes is the
reason for the assimilation of routine-oriented business processes to the classic project
model”. The business process concept separates enterprise (or organizational)
processes into two categories: technological processes concerned with specifying and
creating the enterprise product or service; and business processes concerned with
administering, directing and managing other enterprise activities. “When a number of
tasks accumulate to constitute the execution of some substantial organizational (or
business) requirement, they are commonly referred to as business processes” [Fa01].
In project management this distinction is reflected in classical product-oriented tools
for supporting technology and deliverable research and development, as opposed to
management-oriented tools as reflected in the 39 “business” processes incorporated
into the Guide to the Project Management Body of Knowledge (PMBOK) [Pm00],
also accepted as an IEEE Standard 1490-1998. The Guide also reflects a further
management paradigm – knowledge management – as reflected in the classification of
the 39 processes into nine knowledge areas [Pm00]. However, this two-level
hierarchical approach to knowledge areas and project management processes for
organizing a project only provides a generic framework for knowledge capture and
reuse, but does not detail the intra- and inter-process decisionmaking knowledge that
would actually be associated with a specific project. Nevertheless, as these guidelines
are familiar to and recognized by practitioners, they can serve as an ontology for
developing a knowledge representation for experience management in the field of
project management. This is demonstrated in the COPER (Components for Project
Management Experience Repositories) reference model [BN01]; and, as elaborated
below, in the experiential model described in this article.
2 Project management and experience management
Within the business process concept a specific set of processes can be differentiated:
knowledge management processes. Several schemes have been proposed for these
processes, either generalized or related to CBR as the central tool for their execution.
Probst et al [PRR99, quoted in Mi01] propose six general stages – identification,
acquisition, development (creation), transfer, use and preservation – extended by
Minor [Mi01] to include targets (knowledge-based abilities to be fostered) and
evaluation. Aamodt and Plaza [AP94] describe a “4-RE” scheme for CBR – retrieve,
reuse, revise and retain – extended by Iglezakis and Reinartz [IR02] to include review
and restore (a “6-RE” scheme).
We may divide these knowledge processes into three groupings (a “3-A” scheme):
knowledge acquisition (retain, identify, acquire, develop, organize, preserve)
knowledge application (retrieve, reuse, transfer, use, targets)
knowledge quality assurance (revise, review, restore, evaluate)
Within the framework of our research we then define “experience management” as “a
special form of knowledge management which deals with task-based [and
experiential] knowledge” [Mi01] that is both gained and applied when carrying out
business-related tasks. Our specific tasks are those concerned with knowledge
application in decisionmaking within project management; and the specific
experience is that obtained and employed by project managers when making such
decisions (our “target”). This aspect of experience is succinctly expressed by Watson
[Wa99]: “… that knowledge is not so much a capacity for specific action, but the
capacity to use information, learning and experience resulting in an ability to interpret
information and ascertain what information is necessary in decisionmaking”.
Some method is needed to help project-oriented organizations support their
decisionmaking capability on the basis of experience regarding previous decisions – a
project-specific (or episodic) experience management paradigm, building on and
augmenting the generic framework [Pm00], that derives from examples of decisions
taken in the past. We thus seek to extend the knowledge-oriented focus of the
PMBOK, whilst providing the project manager with a practical approach to applying
past experience when making new judgments. For this purpose, CBR seems to be
ideal (see next section), as a heterogeneous case base provides a highly flexible
format for storing the variegated knowledge associated with the nine project
management knowledge areas [Pm00], and the disparate types of decisions taken by
different project managers. Thus it can be expected that several types of cases will be
involved in a decision; and that these cases must be linked in order to emphasize their
joint relationship to the decision made. This consideration introduces a new concept: a
path linking associated cases in the case base, and a technique – path-based reasoning
(PBR) – to store and traverse these paths in order to retrieve multi-attribute decisions.
Our intention is to describe an architecture, based on CBR and PBR, for knowledge
management of project management processes at the detailed level, concentrating on
decisionmaking tasks.
3 Motivation for a CBR-based approach
We adopt CBR as the knowledge representation and the foundation for experiential
knowledge management for project management processes for the following reasons:
• It provides many benefits over other AI-based approaches [Re99, Gr98, Aa98]:
- A case base becomes useful with the first case
- A case base captures knowledge easily (no need to discover complex
interrelationships between cases for the same issue [see following section 4])
- Case bases are understandable (logical and easy to follow)
- CBR augments human capabilities (comprehensive case storage and tracking)
• The nature of decisionmaking in project management is suited to an example-
based paradigm, as different project managers have different management styles;
and they would prefer to view previous situations and decisions rather than face a
set of prescribed rules or models.
• Likewise, project management knowledge derives from a wide range of
knowledge sources, and is often idiosyncratic. Any preprocessing or processing is
likely to eliminate important aspects of decisions taken. In CBR the knowledge is
not preprocessed [Be99, XR99, Le98], and therefore bias is minimized.
• The case format is flexible and may be modified over time without impacting the
methodology.
• CBR is amenable to being incorporated into a experience management process.
• The CBR cycle is similar to experience reuse in project management [BN01].
• Organizational learning is similar to the CBR cycle and so can be supported by
CBR technology [BN01].
• Brandt and Nick [BN01] describe a reference model for project management
experiences and their reuse based on CBR. The model components include
business processes, problems and solutions, and guidelines for specific project-
related actions.
4 Architecture - structure
The experience-based architecture and its various levels are illustrated in Figure 1. It
is an hierarchical heterogeneous case-based structure, which we term HCRN
(Hierarchical Case-Based Retrieval Network), based on ideas originally described by
Lenz [LE99]. The architecture and case and path bases are detailed in [KK02]; we
provide a brief overview here.
(a) Project management ontology levels (PMBOK)
• Project management knowledge area – management of scope, time, cost,
quality, human resources, communications, integration, risk and procurement
(knowledge area level)
• Project management process – thirty-nine project management processes
(process level within a specific knowledge area)
(b) Case base levels
• Issue – a mapping of a specific decisionmaking task onto one or more project
management processes (issue level, mapped onto the emphasized processes)
• Entity (attribute) –an atomic knowledge item in the issue domain such that an
issue is defined by a (unique) set of entities and their domain of values (entity
level)
• Case – represents an actual decision-making situation from the past,
comprising the same entities as its associated issue, but having assigned values
for most or all entities (case level)
(c) Path base level
• Path –a trajectory between several issues and/or cases indicating that they have
been considered jointly relevant to a decision situation in the past (path level)
5 Architecture – knowledge application process
Aamodt and Plaza [AP94, Figure 2] detail the “retrieve” and “reuse” stages as
follows:
(a) Retrieve (b) Reuse
- identify features - initially match - copy solution/method
- collect descriptors - calculate similarity - adapt solution/method
- interpret problem - explain similarity - modify solution/method
- infer descriptors - select case(s)
- search (case base) - use selection criteria
- follow direct indexes - elaborate explanations
We propose the following flow for experiential knowledge application processes,
based on case-based and path-based reasoning:
(a) Capture (define and transform the decisionmaking problem or task)
1. Express the decisionmaking situation in textual form
2. Decompose the situation into one or more issues
3. Decompose the situation into one or more values for the issue entity(ies) to
create a “case(s) by example”
(b) Retrieve (cases related to an issue for all issues selected)
4. Search the case base, using the “case by example”
5. Calculate similarity (to the “case by example”)
6. Select relevant cases as knowledge focus components (see (d) below)
7. Juxtapose cases from different issues as implying possible knowledge area or
“business process” interactions
(c) Traverse (paths related to the cases selected)
8. Search the path base for paths incorporating the cases selected
9. Select relevant paths
10. Traverse the relevant paths – explaining the connection between the cases
comprising the path arcs (see Figure 1 – path level)
(d) Focus (concentrate retrieved knowledge to support decisionmaking)
11. Correlate selected cases and selected paths (knowledge focus)
12. Explain and interpret case-based knowledge focus
13. Explain and interpret path-based knowledge focus
14. Integrate knowledge focus and its relation to the decision to be made
(e) Decide
15. Make a decision based on the knowledge focus
6 Experimentation
30 students from the Faculty of Industrial Engineering and Management at the
Technion, familiar with project management principles, participated in the
experiment. They were provided with a lexicon of the issues, entities and values
defining the HCRN base, a set of cases for each issue, and several paths between
cases. We concentrate here on three experiential transfer aspects of the experiment:
• capture – decompose and transform a given scenario (steps 1 through 3)
• traverse – explain causes and effects (step 10 and Figure 1)
• focus – explain and interpret path-based knowledge focus (step 13)
For the capture and traverse parts of the experiment (steps 1 through 3 and step 10),
all 30 students carried out the steps in the same way. For the focusing step 13 the
group was divided into two sub-groups. The first worked with a tool consisting of a
computerized database (cases and paths) and a flow diagram of the process described
above (section 5). The second worked with the database but without the guidance of
the flow diagram.
Human Risk Time
resources management management
Knowledge area
Organizational Risk management planning Activity definition
planning Risk identification Activity sequencing
Staff acquisition Qualitative risk analysis Activity duration
Team development Quantitative risk analysis estimating
Risk response planning Schedule development
Risk monitoring and control Schedule control
Process level
Selecting a candidate Selecting a corrective Duration (cost) estimating
for a task - C action - R -E
Issue level
Specialization Risk/problem category Specialization
System Risk/problem System
Experience Reason/trigger category Duration
Salary Reason/trigger Cost
Corrective action category Updated duration
Corrective action Updated cost
Entity level
Analyzing Team Analyzing
Human Resource Mgt. Lack of experience Human resource management
Beginner Customer 1.5-2
10-15K Change of requirements 30-40K
Time 3-3.5
Project delay 50-60K
Case C16 Case R14 Case E18
Case (specific values)
(+,+,+,+, -,-)
R14 E18 (+,+,+,+,+,+)
(+,+,+,+)
(+,+,+,+,+,+)
C16 → the connection between actuating node and actuated
node; +/- existence/nonexistence of value when the case
E18 (node) is recorded
(+,+,+,+, -,-) Path level
Figure 1: Illustration of the proposed architecture structure
(a1) Step 1: scenario
“Select the best candidate – beginner, experienced or expert – for analyzing a
human resource management system, concentrating on the trade-off between
the risks concerned with inexperience and project resources (salary and time)”.
(a2) Steps 2 and 3: decomposition into issues and values
(compare Figure 1, issue and entity levels)
(C)andidates: specialization = analysis; system = human resources (HRM)
(E)stimation of times and costs: specialization = analysis; system = HRM
(R)isk: problem category = team; problem = lack of experience
(b) Step 10: explain causes and effects (compare Figure 1, path level)
“Selection of a beginner with a given salary for HRM system encoding (C16:
++++) led to (a) recording of a case reflecting initial duration (1.5 - 2) and cost
(30-40K) estimates (E18: ++++--); and (b) anticipating a problem of lack of
experience triggered by a possible change in requirements (R14: ++++--).
Foreseeing this problem led to a revision of task duration and cost (E18:
++++++); this, in turn, caused an expectation of project delay (R14:
++++++)”.
(c) Step 13: explain and interpret path-based knowledge focus
Path P2: A change in requirements that led to an increase in product
complexity (R15) resulted in replacing an analyst in human resource analysis
by an experienced worker (C2). Thus there is a connection between product
complexity and the experience required.
Path P3: as the result of his predecessor (an expert in financial analysis)
leaving because of dissatisfaction with his salary (C6), a different employee
was allocated to a task (R13). Thus there is a risk of an employee leaving when
he considers his salary to be too low.
Path P18: After the activity duration and cost were estimated (E2), a beginner
in human resource systems analysis was considered suitable for the task (C1)
in terms of salary and likely performance. Thus the initial budget for the
corresponding hammock activity influences the choice of employee in terms of
his salary.
7 Metrics
In section 5 above we have outlined a process flow for the application of experiential
knowledge, encompassing five phases. Each phase requires an ability to interpret
knowledge and to transfer experience. In section 6 we have described experiments to
study these skills for three of the phases: capture – the aptitude for formulating a
decision situation in terms of issues; traverse – the aptitude for understanding
interactions implied by a path; and focus – the aptitude for relating paths to the multi-
attribute decision situation. We wish to be able to measure these aptitudes, in terms of
the success of the process flow in supporting the development of an effective
knowledge focus.
We thus propose the following metrics for the ability to decompose a decisionmaking
situation, to traverse a path, and to explain the relevance of the path-based focus:
1. Compliance metric – the ability to express the decisionmaking situation (10
points). Identification of issues and values: 3 points for issues C and E; 4 points
for issue R.
2. Accordance metric – the ability to interpret nodes and arcs of a path (10 points).
Explanation of cases and arcs: 1 point each (E18 and R14 are referenced twice).
3. Convergence metric – the ability to explain and interpret the path-based
knowledge focus (3 points). Explanation and interpretation of paths: 1 point each.
The scores for the three metrics are summarized in Table 1.
The main reasons for student misconceptions of the experiential requirements, in
descending order of occurrence, were:
1. Compliance: redundant search (non-directed entity values); over-constrained
search (too many values); incomplete search (not all issues identified);
misinterpretation of retrieved cases
2. Accordance: arc ignored; node (case) misinterpreted in terms of currently
assigned or unassigned values (‘+’ or ‘- convention in Figure 1)
3. Convergence: knowledge focus ignored (path P3 and/or path P1)
Score Compliance Accordance Convergence
0 0 0 0 (1) 0 (2)
1 0 0 0 3
2 0 0 8 8
3 1 0 7 4
4 1 0 -- --
5 1 1 -- --
6 10 2 -- --
7 11 6 -- --
8 3 10 -- --
9 3 7 -- --
10 0 4 -- --
Sample 30 30 15 15
Average 6.7 8.1 2.5 2.1
Standard deviation 1.3 1.3 0.5 0.7
Variance 1.7 1.6 0.3 0.5
(1) (2)
With the use of the process flow diagram Without the use of the process flow diagram
Table 1: knowledge focus performance
Using the Mann-Whitney comparison test, the difference between the convergence
achievement scores for the two sub-groups (2.5 and 2.1 respectively) has been found
to be statistically significant (p < 0.01).
8 Discussion and summary
We have described an experience-oriented knowledge application process for the
decisionmaking task in project management. Experience is accumulated in decisions
taken in the past, regarding job candidates, activity resource allocations and risks
associated with project-related problems. The experiments outlined in this paper have
examined three aspects of experience transfer:
• The ability to translate a decision problem into its underlying issues in order to
relate it to an HCRN case base
• The ability to track and understand interactions between issues reflecting the
multi-attribute characteristic of decisionmaking
• The ability to distil a knowledge focus from retrieved cases and interactions as
a basis for coming to a decision
The student subjects showed a success level of about 70% for decomposing the
scenario; more experience is required in formulating effective “cases by example”.
Success rose to about 80% for understanding the meaning of an interaction or “path”;
more attention needs to be paid to path trajectories and the creation of values along a
path as the decision evolves.
Convergence to the interaction knowledge focus is seen to be far more effective when
the process flow (section 5) was used by the first sub-group to guide the various
phases of the experiment. This observation emphasizes the importance of the process
aspect of a experience-based architecture. The lack of a “perfect” score indicates that
relationships should not be regarded as “irrelevant” until the focus has been
established.
These results indicate that CBR and PBR (path-based reasoning) can support project
management experience acquisition and application, if project managers are given a
correct understanding of these mechanisms. Further capture and traverse experiments
are being carried out, incorporating “push” mechanisms regarding compliance and
accordance, to ensure as far as possible that all components of retrieved cases or paths
will be considered. Refinement of the metrics and scoring method used for experience
transfer is also being studied.
Bibliography
[Aa98] Aarts, R.J.: A CBR Architecture for Project Knowledge Management. In: (Smyth, B.;
Cunningham, P. Eds.): EWCBR-98, Lecture Notes in Artificial Intelligence, Vol. 1488,
Springer Verlag , Berlin Heidelberg New York, 1998; pp. 414-425.
[Be99] Beckman, T.: A Methodology for Knowledge Management. In: (Liebowitz, J. Ed.):
Knowledge Management Handbook, CRC Press, Boca Raton, 1999.
[BN01] Brandt, M.; Nick, M.: Computer-Supported Reuse of Project Management Experience
with an Experience Base. In (Althoff, K-D., Feldmann, R.L., Mueller, W. Eds.): LSO 2001,
Lecture Notes in Computer Science, Vol. 2176, Springer Verlag, Berlin, 2001; pp. 178-189.
[Fa01] Fahey, L.; Srivastava, R.; Sharon, J.S.; Smith, D.E.: Linking E-business and Operating
Processes: The Role of Knowledge Management. IBM Systems Journal 40, 2001; pp. 889-907.
[Gr98] Grupe, F.H., Urwiler, R., Ramarapu, N.K., Owrang, M.: The Application of Case-Based
Reasoning to the Software Development Process. Information and Software Technology 40,
1998; pp. 493-499.
[HS99] Hammer, M.; Stanton, S: How Process Enterprises Really Work. Harvard Business
Review, November December 1999; pp. 108-118
[IR02] Iglezakis, I.; Reinartz, T.: Relations between Customer Requirements, Performance
Measures and General Case Properties for Case Base Maintenance. In (Craw, S., Preece, A.
Eds.): Advances in Case-Based Reasoning, 6th European ECCBR2002 Conference, Lecture
Notes in Artificial Intelligence; Vol. 2416, Springer Verlag, Berlin, 2002, pp. 159-173.
[KK02] Karni, R.; Kaner, M.: Case-Based Knowledge Focusing for Training in
Decisionmaking within Project Management Processes. In (Gonzalez-Calero, P.A. Ed.):
Workshop Proceedings on Case-Based Reasoning for Education and Training, 6th European
ECCBR2002 Conference, Robert Gordon University, Aberdeen, 2002, pp. 25-40.
[Le98] Lenz, M., Bartsch-Sporl, B., Burkhard, H-D., and Wess, S. (Eds.): Case-Based
Reasoning Technology: from Foundations to Applications. Lecture notes in artificial
intelligence; Vol. 1400, Springer Verlag Berlin Heidelberg New York, 1998.
[Le99] Lenz, M.: Case Retrieval Nets as a Model for Building Flexible Information Systems.
Ph.D. Dissertation, Faculty of Mathematics and Natural Sciences, Humboldt University, Berlin,
Germany, 1999.
[Mi01] Minor, M.: Experience Management – Case Based Reasoning for Knowledge
Management. In (Czaja, L. Ed.): Proceedings of the CSuP-2001 Workshop, Warsaw, 2001; pp.
150-159.
[Pm00] Project Management Institute: A Guide to the Project Management Body of
Knowledge. PMI Standards Committee, Newton Square, 2000.
[PRR99] Probst, G.; Raub, S.; Romhardt, K.: Wissen Managen: Wie Unternehmen ihre
Wertvollste Ressource Optimal Nutzen, Gabler, Wiesbaden, 1999.
[Re99] Reuber, R.: Management Experience and Management Expertise. Decision Support
Systems 21, 1999; pp. 51-60.
[Wa99] Watson, JT: Data Management: Databases and Organizations. John Wiley, New York,
1999.
[XR99] Xia, Q.; Rao, M.: Dynamic Case-Based Reasoning for Process Operation Support
Systems, Engineering Applications of Artificial Intelligence 12, 1999; pp. 343-361.