This paper reports results of an empirical examination of the facilitating role of experimentation and socialisation in enhancing individual knowledge and performance in decision making. A laboratory experiment was conducted using 28 graduate students as voluntary subjects. Performance of actual subjects was compared with that of their nominal naive and optimal counterparts. Results indicate that both opportunities for independent experimentation and socialisation among subjects significantly facilitated individual knowledge enhancement and led to improved decision performance. Subjects encouraged to interact with others tended to make better quality decisions than those who individually experimented with the decision task. Both performed better than notional naive subjects who applied random walk decision strategy. However, the results indicate room for further improvement. Subjects failed to reach performance of notional optimal counterparts who used linear decision strategy. The results also suggest the need for a holistic approach to managing knowledge by combining and integrating various initiatives to create even higher levels of knowledge and performance.
Knowledge management literature indicates a widespread agreement among researchers and practitioners alike that knowledge is the only sure source of lasting competitive advantage or even economic survival for organisations operating in a new-age economy [Dev99,Dru93,Ste97].
With this comes a growing recognition of the need to determine ways to better cultivate, nurture and exploit knowledge in modern organisations at different levels and in different contexts. Yet, there is little understanding of the nature of the knowledge-creating organisation and how it should be managed.
Western theorists generally view organisations as information processing machines. They show central preoccupation with hard and quantifiable "explicit" knowledge embedded in organisational repositories as the only useful kind of knowledge [Bax99,Non98]. Eastern theorists, on the other hand, focus more on "tacit" knowledge that people derive from their own experience and through sharing [Non95,Non98]. The notable success of Japanese companies suggests that any company that wants to compete on knowledge should learn from their examples, and master techniques for creating and sharing tacit knowledge. Given that between 40% [Aao98] and 90% [Hew99] of the needed knowledge in organisations is tacit, it is not surprising that there is currently a sense of urgency felt among knowledge management researchers to better understand how to tap the wealth of knowledge in people's heads. Some authors suggest that new knowledge always begins with the individual, and that making personal knowledge available to others is the central activity of the knowledge-creating company [Non95]. The spiral knowledge model assumes that the process of sharing will result in the organisational amplification and exponential growth of working knowledge. Others propose that in order to build a learning organisation the first step should be to foster the environment conducive to individual learning, that is allow experimentation to gain experience, and second, to open up boundaries and stimulate exchange of ideas [Gar98]. However, given the current infancy of the knowledge management research, there is little empirical evidence regarding the ways in which tacit knowledge is actually acquired and shared, and the impact it has on performance.
The main purpose of this study is to address the issue by empirically investigating the potential of two specific knowledge management initiatives (experimentation and socialisation) to facilitate individual knowledge enhancement and application in a judgemental decision making task context. Decision making is an important and knowledge intensive activity. Business decisions in organisational environments can be made individually or in groups. The literature suggests that the majority of The copyright of this paper belongs to the paper's authors. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage.
Basel, Switzerland, 30-31 Oct. 2000, (U. Reimer, ed.) http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-34/ M. Handzic 12-2 important decisions are made individually, but after significant social interaction [Hea95]. Therefore, it is important to address both individual and social factors that influence knowledge and performance of organisational decision makers.
Decision making has been studied from normative and descriptive perspectives. Normative (classical) decision theory is the collection of axiomatic models of utility and probability that describe the optimal decision making under uncertainty [Bea93]. In its normative role, this theory describes decisions of an ideal economic man who would behave in accordance with the principle of benefit maximisation. In its prescriptive role, the way that the economic man would behave is assumed to be a uniquely appropriate rational way.
Empirical research has repeatedly demonstrated that decision makers do not conform consistently to the logic of normative theories [Tve74]. Instead, individuals use heuristics or general rules of thumb to arrive at their judgements. This leads to predictable biases or deviations from normatively derived answers. Some of the best known individual biases and errors include representativeness, availability, and anchoring and adjustment. Studies have shown that reliance on representativeness leads people to ignore base rate information which, in turn, leads to inaccurate predictions. The availability heuristic can lead to critical biases in judgement of probabilities and frequency estimates. The effects of anchoring show that people adjust insufficiently from anchor values regardless of the topic. People also have difficulties in assessing correlations among variables [Plo93].
Because people are social by nature, their judgements and decisions are subject to social influences. Much of the earlier research into group interactions is devoted to a groupthink phenomenon. According to Janis [Jan82] members of the cohesive long-term groups strive for unanimity and do not realistically appraise alternative courses of action. This results in unfavourable outcomes. In general, past research indicates that most individuallevel biases and errors tend to operate with equal force in groups [Plo93].
While there is little doubt that people violate the principles of normative theory, these violations do not mean that people are irrational, or that the way people make decisions is unreasonable. Simon [Sim90] suggested that processing capabilities of the decision maker interact with the complexity of the environment to produce bounded rationality. The awareness of the calculated rationality has led to several models that emphasise the cognitive costs and benefits of various strategies people might use in constructing preferences and beliefs. The Beach and Mitchell [Bea78] model for the selection of decision strategies represents a direct extension of the bounded rationality concept. Selection of which strategy to use in a particular decision problem is contingent upon the demands of the task, environment and the characteristics of a person. Most empirical studies show that people adapt their behaviour to changes in task and context in ways that seem reasonable given a concern for both accuracy and effort [Pay88,Cre90]. These studies also show that people adapt well enough to satisfice, but do not in general optimise.
In summary, most earlier research in the psychology of human judgement and decision making is devoted to individual and group biases and errors. Studies of flawed reasoning are six times more often cited in journal articles than studies of successful reasoning [Plo93]. However, adaptive models of human behaviour offer a more optimistic view. Beach and Mitchell [Bea78] have identified a number of specific factors that lead to decision behaviour in the direction suggested by the normative models. Thus, greater uncertainty and significance, lower complexity and constraints of the decision task, as well as higher decision maker's knowledge, motivation and ability to learn are likely to encourage the choice of a more analytical strategy and lead to better performance.
Failures in judgement and decision making described by "pessimists" and adaptive nature of decision making emphasised by "optimists" are quite informative for knowledge management research. They provide a basis for its interventionist approach to knowledge processes, as they identify points of concern and suggest adaptive directions. It is argued here that a generic knowledge management model [Aao98] with four major enablers (technology, culture, leadership, measurement) that facilitate knowledge processes can provide an appropriate theoretical framework for studying knowledge management in judgement and decision making. The central task here is to identify those initiatives and practices that would reduce the complexity and uncertainty of the decision task, minimise environmental constraints, and facilitate development of relevant decision makers' knowledge and skills to maximise decision performance.
The focus of the current study is on two knowledge management initiatives, experimentation and socialisation, both aimed at fostering working knowledge of individual decision makers. These two initiatives have been suggested as central activities of a knowledge creating company [Gar98,Non95]. Organisational culture encouraging experimentation is assumed to foster individual learning, while socialisation is believed to enable amplification and exponential growth of working knowledge. The objective of this study is to empirically test these two assumptions. More specifically, the study will attempt to answer the following research questions: (i) whether and how an opportunity for individual experimentation and social interaction among decision makers affect their working knowledge, and (ii) what impact these two initiatives have on their subsequent decision performance.
The experimental task for the current study was a simulated production planning activity in which subjects made decisions regarding daily production of fresh icecream. The participants assumed the role of Production Manager for an imaginary dairy firm that sold ice-cream from its outlet at Bondi Beach in Sydney, Australia. The company incurred equally costly losses if production was set too low (due to loss of market to the competition) or too high (by spoilage of unsold product). The participants' goal was to minimise the costs incurred by incorrect decisions. During the experiment, participants were asked at the end of each day to set production quotas for ice-cream to be sold the following day. Subjects were required to make thirty production decisions over a period of thirty consecutive days. Before commencing the task, participants had an opportunity to make five trial decisions for practice purposes only.
From pre-experimental discussions with actual store owners at Bondi Beach, three factors emerged as important in determining local product demand: the ambient air temperature, the amount of sunshine and the number of visitors at the beach. This important contextual information was provided to the participants in addition to past product demand to aid their decision making. Subjects were free to use the available information as much as they wished to, by making explicit request to the computerised information system. All contextual factors were artificially generated to provide similarly moderate predictive power in estimating future sales. This was achieved by setting correlation coefficients between contextual and predicted variables to r=0.80.
Subjects performed the task under different working conditions. Half of the subjects were allowed to experiment with their information to learn causal relationships among contextual and demand variables. They were expected to apply that knowledge in their final individual production decisions. The other half were encouraged to share their personal knowledge with others. In particular, participants from this group were placed in teams of two and instructed to discuss their ideas and opinions before making final decisions. However, they were not required to reach a consensual decision.
At the beginning of the experiment, task descriptions were provided to inform participants about the task scenario and requirements. The given text differed with respect to the working conditions prescribed. Throughout the experiment instructions and feedback were provided to each participant to analyse earlier performance and to adjust future strategies.
A laboratory experiment with random assignment to treatment groups was used in the current investigation because it allows drawing of stronger causal inferences due to high controllability. The only independent variable was knowledge management initiative (experimentation vs. socialisation). It was manipulated by providing the subjects with an opportunity to either individually experiment and learn the task by trial and error, or by encouraging them to socially interact with each other and share ideas and opinions while handling the task.
The dependent variable was decision performance. It was operationalised by symmetric absolute percentage error (SAPE), chosen because it controls for scale. SAPE is a popular accuracy measure suggested by the forecasting literature [Mak93]. In this study, it was obtained by dividing the absolute difference between estimated and actually demanded units of product by an average of the two values and multiplying by 100%. In addition, the corresponding errors of nominal naive and nominal optimal decision makers were calculated. These are error scores that would have been obtained by completely ignorant and ideally knowledgeable people who produced their decisions using naive (random walk) and optimal (linear) strategies respectively.
The subjects were 28 graduate students enrolled in the Master of Commerce course at The University of New South Wales, Sydney. They participated in the study on a voluntary basis and received no monetary incentives. Generally, graduate students are considered appropriate subjects for this type of research [Ash80,Rem96,Whi96]. Individuals were assigned randomly to one of the two treatment groups by picking up a disk with an appropriate version of the instrument. Then, they were directed to two designated microcomputer laboratories where they were briefed about the purpose of the study, read case descriptions and performed the task. The session lasted about one hour.
The collected data were analysed using a series of statistical T-tests to compare decision performance (SAPE) among four experimental groups of subjects (notional naives, independent experimenters, social interactives and notional optimals). Results are presented graphically in Figure 1.
Results of the analyses performed indicate significant improvement in decision performance due to individual experimentation. Independent experimenters tended to make significantly smaller decision errors (SAPE) than their notional naive counterparts (17.80% vs. 20.67%, p<0.05). There was a real drop in error by 14%. Results also indicate significant further improvement in decision performance due to socialisation. Social interactives tended to make significantly smaller decision errors than independent experimenters (11.42% vs. 17.80%, p<0.05). This meant a further drop in error by 31%, to a total of 45%.
Furthermore, the results of the analyses indicate that participants failed to achieve optimal performance. Both independent experimenters and social interactives tended to make significantly larger decision errors than their notional optimal counterparts (17.80% or 11.42% vs.
12-4 6.25%, p<0.05). These participants were able to acquire and apply 20% and 64% respectively of the knowledge possessed by an ideally knowledgeable decision maker on the same task.
In summary, the main findings of the present study indicate that knowledge management initiatives aimed at providing opportunities for experimentation and socialisation were quite useful in enhancing individual decision makers' working knowledge and performance in a judgemental decision making task. In addition, socialisation was relatively more valuable than experimentation. However, performance gains were less than possible given an expert knowledge of the task.
With respect to experimentation, the study demonstrated that it had a significant positive impact on individual knowledge and performance. Independent experimenters were found to make significantly smaller decision errors that notional naive decision makers. In real terms their error scores dropped by 14%. This finding suggests that subjects allowed time and opportunity to experiment on their own were able to acquire enough relevant knowledge of the decision problem and solving strategies to improve performance. As a result they significantly reduced decision errors.
This finding provides a more optimistic view of human ability to learn multivariate probabilistic judgement tasks through experience than suggested by earlier research [for review see Bre80]. It is possible that graphical presentation of historic data used in this study enabled subjects to easier identify the existence and direction of relationships among various task variables. Such proposition is consistent with the earlier finding by Lawrence et al. [Law85] that graphical presentation form enhances the accuracy of novice decision makers. It is also possible that the subjects in this study were given sufficient time for experimentation that enabled them to appropriately adjust their strategies through task repetition and from feedback. Klayman [Kla88] also reported that people could learn reasonably well cue discovery over a larger number of trials.
The study also demonstrated a highly beneficial effect of socialisation on individual decision making. Social interactives were found to make significantly more accurate decisions than independent experimenters. The incremental drop in their error scores was 31%. This finding suggests that subjects encouraged to interact with others were able to enhance knowledge through exchange of ideas and opinions and consequently improve performance. Participants might have brought their personal analysis and know-how to the task, acquired information about their partner's ideas and arguments and considered both in making final decisions.
The beneficial effect of socialisation evident in this study is consistent with the theoretical propositions of the knowledge management literature [Gar98,Non95]. It also agrees with frequent anecdotal evidence from the real world organisations [Hew99]. However, such findings are contrary to a large number of earlier studies on group decision making that emphasise negative aspects of decision making in groups such as groupthink [Jan82,Plo93]. One potential reason for the discrepancy may be due to the nature of the group decision process used in this study. The study encouraged participants to interact, but did not require them to reach consensual decisions. Thus, it might have avoided a potential negative effect of groupthink. In addition, subjects in this study were provided with continuous performance feedback that might have enabled them to evaluate their own ideas against those of their partners and adjust future strategies accordingly. Earlier empirical research indicated beneficial effect of objective feedback on performance [Kop86]. In short, generating and sharing personal knowledge coupled with the opportunity to test its contribution to performance might have enhanced knowledge and resulted in greater accuracy.
An important additional finding of the study is a substantially larger positive effect of socialisation than experimentation on decision making. Social interactives tended to make more than twice as many accurate decisions as independent experimenters. This finding appears to agree with the proposition of the spiral knowledge model [Non95] suggesting that sharing personal knowledge results in amplification and exponential growth of working knowledge. Some reports indicate that a number of large western companies (eg. British Airways) have already realised this and have built an appropriate infrastructure (eg. a coffee village) to facilitate social interaction and knowledge sharing among its employees [Hew99].
With respect to overall performance, the study revealed serious deviations from optimal performance irrespective of the knowledge management initiative implemented. Both independent experimenters and social interactives were found to make significantly larger decision errors than they could have. Further analysis revealed that, on average, they acquired and applied 20% and 64% of the knowledge assets of an expert decision maker through experimentation and socialisation respectively. One potential explanation for the failure to achieve optimal performance may be the difficulty of learning optimal functional forms among given variables. According to Klayman [Kla88] people can learn reasonably well the existence and direction of a cuecriterion relationship, but have difficulties in learning its shape. As a result they tend to perform sub-optimally. Alternatively, the failure could be attributed to the moderately predictive power of contextual factors. Hoch and Schkade [Hoc96] have proved theoretically that pattern matching strategy can not produce good results in environments that are not highly predictive. Finally, the participants in the study were not expert decision makers, but novices. Garvin [Gar98] suggests that to become an expert each individual must pass through a number of stages of knowledge. In this context experimentation and socialisation might have helped push participants from lower to higher stages. However, examples from literature suggest that for maximum effectiveness other initiatives including transfer of knowledge through education and training programs are essential.
While the current study provides a number of interesting findings, some caution is necessary regarding their generalisability due to a number of limiting factors. One of the limitations refers to the use of a laboratory experiment that may compromise external validity. Another limitation relates to artificial generation of information that may not reflect the true nature of real business. The participants chosen for the study were students and not real life decision makers. The fact that they were mature graduates may mitigate the potential differences. No monetary incentives were offered to the participants for their effort in the study. Consequently, they could find the study tiring and unimportant and would not try as hard as possible. Most decisions in real business settings have significant consequences.
Although limited, the findings of the current study may have some important implications for organisational knowledge management strategies. Firstly, they provide information about two valuable knowledge management initiatives (experimentation and socialisation) that facilitate knowledge creation and improve performance of organisational knowledge workers. Secondly, they point to the need to consider relative importance and limitations of these initiatives in planning knowledge management strategies to prevent unrealistic expectations. Finally, they suggest the need for additional initiatives (eg. knowledge transfer through education and training) to enable even higher levels of knowledge and performance. According to Davenport and Prusak [Dev97] only by taking a holistic approach it is possible to realise the full potential of knowledge ecology.
Future research is necessary to empirically investigate the potential of other various individual or combined and integrated knowledge management initiatives to further enhance knowledge and enable optimal performance. One possible direction for future research is to explore the potential contribution of instruction (eg. coaching and mentoring) to enhancing individual tacit know-how. Future research may also examine the role of explicit analytical and procedural knowledge embedded in organisational repositories. Technology may play an important role in capturing and distributing organisational knowledge, as well as in promoting human interaction and knowledge sharing. Therefore, future research may examine the potential of various information and telecommunication technologies to enable and facilitate knowledge processes. Finally, future research may look at how different knowledge management initiatives interact to create potential synergy effects. These suggested directions are neither prescriptive nor exhaustive, but represent only a small selection of issues that are currently of interest to the author.
The main objective of this study was to investigate whether and how the opportunity for individual experimentation and socialisation with others may affect decision makers' working knowledge and performance in a specific judgemental decision task. The findings of the study indicate that both experimentation and socialisation were beneficial in enhancing decision makers' working knowledge and subsequent decision performance. The opportunity to experiment led to enhanced decision accuracy compared to naive strategy, while encouraged social interaction led to further significant improvement in accuracy over and above that achieved through independent experimentation. However, optimal performance was not achieved. These findings indicate room for improvement. They also suggest that non-expert professional knowledge workers in judgement and decision making tasks could benefit from other additional knowledge management initiatives. Therefore, further research is necessary to systematically address various combined and integrated initiatives in different tasks and contexts and among different knowledge workers if a better understanding of the field is to be achieved.