AWRE’2002 199 The Human and Organizational Impacts of Technology Uptake and Use Christine Wood and Terry Webb DSTO Fern Hill, Dept of Defence, CANBERRA ACT 2601 christine.wood@dsto.defence.gov.au terry.webb@dsto.defence.go.au Abstract Any particular IT system can be specified in many different ways, reflecting the processes and practices of the organization and organizational culture. It is important to understand the interconnections between IT and the organization and to understand the implications of impacts of different systems on the organization. This paper explores some of the human and organizational needs and requirements of an IT system in relation to: • requirements issues where although the complete system and/or individual components of the system work as intended by the developers users are dissatisfied and • implementation issues where the complete system and/or individual components of the system does not work as was intended by the developers (e.g. crashes or it produces erroneous data). A user perception survey was conducted in a large government department after the introduction of a computerized maintenance management system. Results indicated that users were largely dissatisfied with the system. Keywords: requirements, usability, perceived ease of use, perceived usefulness, technology acceptance model, organizational effectiveness, training, technology uptake. Introduction A number of factors contribute to overall user satisfaction and uptake of new IT systems within organizations. These can be broadly classified into two categories; requirements issues which includes usability, perceived ease of use, perceived usefulness, communication, information about the system and training. The other is implementation issues which require that a system work as it was intended and is reliable. This includes a lack system crashes, access to the system, data integrity, system speed, and that the system does not produce any errors. Users and organizations develop expectations regarding how a system will perform before the system is implemented. Both formal and informal methods are employed by organizations and within organizations to build expectations. Prasad and Prasad (1994) described a study of the computerization of a health administration system in a Health Care Organization. They found that potential users had discussed and anticipated the arrival of the new system and had categorized the implementation of the system as a professional and professionalising process. This helped create a climate of acceptance where personal concerns were reduced. Although management AWRE’2002 200 did little to involve employees in the implementation process, or to understand the social roles and networks as a result of the computerization process, commitment to using the new system was high. We report the results of a user perception survey of the implementation of a Computer Maintenance Management system in a large government department. In this case the organization used a variety of promotional materials (ranging from printed matter to information contained in an intranet site). Potential users were also invited to comment on the design of the system prior to and during its introduction. An extensive training program was also implemented. All of these factors contributed to users having a high expectation of the new system. The foundation of our research derives from three areas of research; the Technology Acceptance Model also known as TAM (Davis 1986, 1989, Davis Bagozzi & Warsaw 1989, Davis & Venkatesh 1996), Organizational Effectiveness (Creed, Stout & Roberts 1993; LaPorte & Consolini 1991, Damadoran & Olphert 2000) and Usability (Shackel, 1986, Eason, 1988). The TAM (see Figure 1) was developed by Davis (1986) in an effort to predict, explain and increase user acceptance of technology. It is based on the Theory of Reasoned Action (Fishbein & Ajzen, 1975) and investigates the ability to predict peoples’ acceptance of technology including computer acceptance and to predict intentions from attitudes, perceived usefulness, perceived ease of use and other related variables. External variables affect perceived ease of use and perceived usefulness, which in turn influence behavioural intention which contributes directly to actual usage. In this model external variables provide a link between internal beliefs, attitudes and intentions and may include system design features, training, user involvement in design and information about the system. Perceived Usefulness Behavioural Intention Actual System Usage External Variables to Use Perceived Ease of Use Figure 1. The Technology Acceptance Model (Davis, et al, 1989). Perceived ease of use has a direct effect on perceived usefulness and is defined as “the user’s perception of the extent to which using a particular system will be free from effort” (Davis & Venkatesh 1996, p20). Perceived usefulness is “the user’s perception of the degree to which using a particular system will improve his/her performance” within an organizational context.” (Davis & Venkatesh 1996, p20). The organizational effectiveness measures were derived from the work of Creed et al (1993) and Prasad & Prasad (1994). It is frequently the potential of the technology, not the prevailing political or social forces that drive concern for effectiveness. Certain technologies pose risks for an organization that must be addressed; an organization’s decision to implement technologies is often based on technical rather than organizational or human considerations. Considering technology as a fundamental of the culture of the organization can foster a greater understanding of the processes and practices that impact organizational effectiveness. It is necessary to think about the nature of an organization’s technology, the dynamics of the evolution AWRE’2002 201 of technology, and the implications of technological evolution on organizational effectives (Creed et al 1993). Damodoran & Olphert (2000), and Prasad & Prasad (1994) stress that technology can support and maintain a beneficial shift in organizational culture, and that this shift highlights the value of new technologies and promotes their use. Two aims of highly effective organizations are: 1) to manage complex demanding technologies ensuring they avoid failures that would adversely affect the organization and 2) to maintain the capacity to meet periods of very high demand often under considerable time pressure (LaPorte & Consolini, 1991). In assessing organizational effectiveness it is necessary to consider values and preferences, internal and person orientation, and to focus on external factors such as political and social pressure. Although staff and management may have little control over external events they nonetheless impact organizational performance. Technology is often used to leverage organizational change in an effort to improve effectiveness and efficiency. The effective use of technologies requires major organizational and individual learning (Eason, 1988; Senge, 1992). The other body of research utilised was that of usability which is closely tied to fitness for purpose and ease of use. Shackel (1991) suggested that usability, as he defined it, was comprised of four components or criteria: • Flexibility (e.g. a capacity to cope with some specified deviation from the specified environment), • Learnability (e.g. effective use shall be developed within a pre-defined training scheme and system of user support), • Effectiveness (e.g. the required range of tasks must be accomplished at equal or better than a specified performance level), and • Attitude (e.g. there will be acceptable levels of human cost in terms of fatigue, discomfort, frustration, personal effort - these being the factors that are most likely to colour a user's attitude toward the system). In 1988, Eason suggested that usage of an information system is the single most reliable indicator of usability. In those environments where the user has discretion over their use of a system, usage will decline as usability lessens and may decline to the point where the system will be discarded. The relationship between usability and usage will be moderated by the usefulness of the system. In effect, trade-offs are made between usefulness and usability. Method Participants were from a large Government Department and were assigned to attend questionnaire and interview sessions by their local line management. No further control could be exercised over the sample in terms of its structure and representativeness. Everyone attending a session completed the questionnaire and up to six persons per session were selected for structured interviews. At the time of this study the system had been in use for approximately two months and users were required to use part of the old system (a paper based, semi-computerised system) in combination with the new system to record maintenance information. Two primary means were used for data collection in this study: A self-report questionnaire and structured interviews. Focus groups were utilised during questionnaire development. Questionnaires and Structured Interviews The technology uptake questionnaire was comprised of previously developed scales such as the TAM (Davis, 1986, Davis & Venkatesh, 1996) as well as purpose-derived AWRE’2002 202 scales relating to organisationally relevant constructs and system usability (Creed et al 1993, Shackel 1991). Prior to interviews or completing questionnaires participants signed an informed consent statement. Participants were asked to respond based on their experiences with the system. This paper reports on the results of the interview data and will only make reference to data from the questionnaires where it is particularly salient. The interview structure followed that used in the questionnaires and contained a 15 probe topics designed to elicit and explore information about the system, its acceptance, and use that would not be apparent in the questionnaire data. Interviewees were also given the opportunity to raise any other issues. Most questions on Technology Uptake were included on the basis of previously published research (Davis, 1989), (Adams, et al 1992) and (Igbaria, 1994) and covered the areas of: past behaviour, behavioural intention, perceived usefulness, perceived ease of use and perceived behavioural control. Other questions were derived from three sources: Shackel's (1991) usability criteria (work processes, confidence and familiarity, training, help, and information supply). The questions relating to organizational effectiveness are based on the work of LaPorte & Consolini (1991) and Creed et al (1993) and have been adapted to study organisational behaviours and technology uptake. The third source was derived from the system promotional materials. A total of 21 Interview transcripts were analysed by abstracting and collating the statements made and, where appropriate, classifying them as positive, neutral or negative. The number of statements made on any given topic may exceed the number of participants in the structured interviews as users often utter more than one statement on the same topic. This qualitative data served to clarify the quantitative data collected using the questionnaires. Ninety-five participants completed the questionnaire but some respondents did not answer all questions. Requirements Issues Perceived Usefulness Perceived usefulness is the degree to which using a particular system will improve the users performance within an organizational context (Davis & Venkatesh 1996). Responses in the structured interviews to questions about the system’s usefulness were predominantly negative. Of the 181 statements considered with usefulness, only 12.7% were positive, 13.3% neutral, and 74% negative. Over 22% of these statements were to do with the extra time it takes to get work done with the system. The amount of time required to complete tasks appears to have dramatically increased. Some statements made about this include: “…The amount of time you have to spend on the computer it's lengthened the actual repair process. It definitely has not streamlined it….” “I haven't found a single thing that it can do that can help me. It can't give you any benefits that the old system did at this point that I've found.” There was a range of estimates of the increase in time needed to complete tasks: “it's taking twice as long…” to “It would slow me down five times.” Other negative statements concerned the increased workload: AWRE’2002 203 ”…you've got to do ten times as many transactions compared to the old system to get the same direct result.” “…it's been more a hindrance than anything… We've got to pull a part off on the system and make it serviceable get another part and make it serviceable, fit it … put all the hours in, it just takes a long time especially when you’re doing 12 things” However, while feelings about the system are predominantly negative, there are some positives. It was also proposed that the system has potential, if only some work is done on it. For example: “I think (the system) has got the potential to be really good, it’s got a lot of very good information there and probably very good as a management tool.” The questionnaire data collected regarding the perceived usefulness of the system (see Figure 2) indicates that in all areas respondents perceived the system highly negatively i.e. assigning scores very close to the ‘Strongly Disagree’ pole of the scale (Mean = 1.91, SD = 1.06 N = 92). The possible range of score was 1 to 7. Standard 7 6 5 4 3 2 1 The system Using the Using the Using the Using the Overall, I find allows me to system system system system makes the system accomplish increases my improves my enhances my it easier to do useful in my tasks more productivity. job effectiveness my job. job. quickly. performance. on the job. Figure 2. Mean Perceived usefulness of the system. (1 - strongly disagree; 4 - neutral; 7 - strongly agree). deviations for the Perceived Usefulness scale indicate a high level of consistency in scoring. Usability Usability is the capability of a system to be used by humans easily and effectively (Shackel, 1991). Good system design depends on solving the interactions between users, tasks, tools and the environment in which they work. The level of usability is judged by users in relation to their perceptions of how easy the system is to use and by measures of effectiveness. Elements of usability are reported below. AWRE’2002 204 i. Ease of Use The Ease of Use Criteria used were classified according to the Shackel Usability Criterion into which they best fit (See Table 1.); there is some overlap as there are several possible secondary mappings for each of the scale items. Item Ease of Use Scale Shackel Criterion 1 Learning to operate “system name” was easy for me. Learnability 2 Interacting with “system name” is clear and Learnability understandable. 3 I find it takes a lot of effort to become skilled at using Learnability “system name”. 4 I find it easy to get “system name” to do what I want it to. Effectiveness 5 “System name” is rigid and inflexible to interact with. Flexibility Table 1.Ease of use classified according to Shackel’s (1991) Usability criteria. It was made clear during the structured interviews that the system was not easy to use. Strong negative feelings toward the system were expressed by a number of different interviewees. Of the 94 statements relating to ease of use 26.5% were positive, 3% were neutral and 70% were negative. Most negative responses referred to producing reports, extra time required to operate the system, inconsistent system behaviour, frustrating delays and the lack of support for replacing optional equipment (e.g. various racks). With reference to the positive statements relating to ease of use, the following are typical of those surveyed: “Having the system spread over a wide area is one on the positives…rather than having one central area where you record maintenance.” “Once you’ve worked out how to pull a report the information you get on them is really good.” The comments made on ease of use were predominantly negative (70%). Some commented on access to the system, others on how time consuming it was, others on how frustrating it was and the lack of functionality. The following are examples of the criticisms relating to the use of the system: “It’s something that requires a lot of effort to use.” “In terms of being a replacement (for the paper-based system) it does it in a very cumbersome manner.” This quotation sums up the attitude of users: “They could have done it better … but I suppose we’ve been built up for so many years about this system, about how good it was going to be and it was such a disappointment.” In general users did not assess the system as being easy to use. Problems particularly mentioned included a lack of consistency in its working/availability, a lack of consistency in its human-computer interface, a ‘one-size-fits-all’ approach to interface design i.e. separate appropriate interfaces not designed for specific work groups/tasks, and questionable accuracy of the data in the system. AWRE’2002 205 ii. Training Training can be regarded as the prime factor directly contributing to the learnability of a system. Effective use of a system is developed as a result of a pre-defined training scheme. The desirable attributes of such a training scheme are that it: reflect the system to be used, and is appropriate to the user’s tasks. The training provided for the system was perceived negatively by most questionnaire respondents and was regarded as providing little assistance in using the system (Mean = 3.42, SD = 1.33, N=91). The users who were interviewed tended to be negative also (64% of comments made were negative and only 17% positive). Two related themes emerged when considering the negative comments that had been made: training was too generic and lacked relationship to task (not related to their skills or specialisation). In addition, users noted the need for training to be conducted close to the delivery of a system. When considering the single training related item from the Perceived Behavioural Control Section of the questionnaire it is apparent that users considered the assistance to use the system that was provided by the training regime to be marginal at best. The majority of interview statements relating to training were negative – 64%, 19% were neutral and 17% were positive statements. The positive statements tended to relate to what they thought of the course at the time they did it rather than what they thought of their training subsequently. “… the training package in general was good.” “When we actually did the Computer Based Instruction…everyone came away thinking this is going to be good.” The negative views reflect problems with either course mechanics, such as: “The training database…. There’d be a couple of ways it (the system) could do something, but it would only let you do it one way at a time then make you do it a different way another time. The core training information on the system I was using was corrupt and you couldn’t do some of the scenarios that were given.” There was a consistent theme that the training was too generic i.e. not designed to support their job or the type of work they did: “Basically they (training) were generic...It was really irrelevant” “It wasn't directed at my type of work or the work shop environment, it was just a general overview of every function they thought was necessary…. They just wasted days of training. It was a five day course and they could have done the transactions that we use in the workshop in one day.” The above criticisms strongly indicate that, in the users’ view, the training definitely did not reflect the system to be used and was not appropriate to the user’s tasks. A number of people felt a major flaw was a lack of relation to the actual job, and in particular the time lag between training and use at work which was, eventually, many months! Organizational Effectiveness A number of factors contribute to organizational effectiveness. This research project investigated work processes, work behaviours, workload, relationships in the workplace and communication. Each of these will be discussed in more detail below. AWRE’2002 206 i. Changes to Work Processes A total of 138 statements were collected in the structured interviews in relation to work processes. Of these, 5% were positively oriented toward Work Processes undertaken since the introduction of the system, 14% were neutral and 81% were negative. This data is remarkably consistent with the questionnaire data. The positive statements made included one person's assessment: ..... “So in many ways it has improved the quality of work." Hopes for the future were also expressed: "So with more training I'd expect it would become more useful." "I believe it will be better but that's a long way down the track." The remaining 'positive' statements are more related to being 'comrades in adversity' and to having to verify the system against what is ‘known’ than to the value of the system: " It's made each of us in the workshop rely more on someone else in the workshop to maybe answer a problem in regards to the system, so it's probably built on the teamwork that way." In general, two categories of negative statement were salient. Firstly, 29% of these statements related to the increase in time taken to perform tasks because of the system and, secondly, the 10% of statements directly addressed workload. The time taken to use the system (for whatever reason) was assessed by many respondents and a conservative estimate given for the extra time needed was: "At the moment we've virtually tripled our work." One of the major concerns was for the effect on productivity: “… do a couple of repairs and come back and then spend like three hours in there signing up and that's down time when … could have been doing more work." In the questionnaire users were asked to rate on a scale from 1 (strongly disagree) to 7 (strongly agree) if they believed the system to be targeted and suited to the work done by themselves, their unit, trade, and the organization as a whole (Mean = 3.08, SD = 1.31, N = 92). Other questions asked whether respondents believed that system is well targeted to the work and organisational needs of the organization, their unit, their trade group, and their level of specialisation (Mean = 3.00, SD = 1.28, N = 92). Responses to these questions were consistently below the neutral point (4) on the scale (See Figure 3) indicating users perceive the system to be poorly targeted to the work and organisational needs at a range of levels (from the organisation-wide, down to their own units and trade groups). Users also responded to the statement: “I consider that the work of the organization has been improved by the introduction of the system.” The mean of 1.97 is low and the standard deviation (1.30) indicates that an overwhelmingly large proportion of the respondents (frequency count = 93.5%) scored this item below the neutral point of the scale indicating that they do not consider the work of the organization to have been enhanced by the introduction of the system. AWRE’2002 207 7 6 5 4 3 2 1 The system is well targeted to the The system is well targeted to the work of The system is well targeted to the work of The system is well targeted to the work of The system is well targeted to the work of The system is well targeted to the The system is well targeted to the organisational needs of the Organization as organisational needs of my Group. organisational needs of people who are at people who are at my level of Specialization organisational needs of my unit. organisational needs of my Trade. The system is well targeted to the work of The system is well targeted to the The system is well targeted to the the organization as a whole. my level of Specialisation my Group. my Trade. my unit. a whole. Figure 3 Results from work processes items from the questionnaire (1 - strongly disagree; 4 - neutral; 7 - strongly agree). Users did not assess the system to be well targeted to the organisational and work needs of themselves, their units, their trade group or the organization as a whole. Users stated unequivocally and frequently that the system had slowed down their work rate and increased their workload. The system is seen as being more labour intensive and therefore it takes longer to complete each task. Limited access to terminals and printers was an exacerbating factor. ii. Changes to Work Behaviour Using recall (hindsight), users perceived minor but negative changes in work related behaviours since the introduction of the system. These behaviours involved small (but statistically significant) negative shifts in: attention to detail, being alert to potential accidents, doing the job well, not making work for others, ownership of problems, understanding the job boundaries of others and, working flexibly. The perceived increase in workload attributed to the system needs to be considered in the context of existing staff shortages. However, given the parallel perception that time- on-task has increased, it is possible that much of the perceived increase in workload is system related. The question of a decline in morale was also raised. Of the statements that relate to work behaviour the majority of statements (94%) indicated that there had been a negative effect on work behaviours since the introduction of the system. It is apparent that the depth of feelings expressed during the interviews was not captured by the questionnaires. The comments from the interviews show that the system has had negative impact on them in a number of areas AWRE’2002 208 relating to work behaviours including; making work for others, relying on others, attention to detail, being alert to the potential for serious accidents, working flexibly, and changes in job boundaries. Some typical examples of these quotes relating to the system making work for people include: “You’re always making more work for your mates now” “Now basically if you stop work at 10 o’clock or so, so you’ve got 2 hours to finish the system work, so you’re leaving more jobs for the other shift which is in turn increasing their workload….” “…before you’d get say three hours of productive work, we’re now getting one and the rest of the time is admin on the system.” Fifteen percent of the statements related to safety. There are a variety of issues raised here which are of concern because they are indicators of problems in terms of occupational health and safety. Typical of the issues raised is: “We’re having to think of a lot of ways to make the system do what we now do safely. Not only safely, but correctly and record what needs to be recorded.” One of the other questions discussed at the interviews was about how the system had changed the way people had to rely on others in order to do their job. Comments specifically referred to having to rely on others more in order to solve problems directly associated with the system. In relation to working flexibly it was reported that the system has reduced flexibility in getting work done. There also appear to be affects in relation to changing work boundaries such as expanding the manager’s role to include “counselling” and consoling workers: “We’re now having to deal with an induced morale problem because of the frustration of the system. So quite often, and more often than on a daily basis we are having to say to people “look calm down, relax …just settle down and take your time and do it” because people are just getting frustrated with it.” iii. Workload Interviewees raised workload issues and the general consensus was that the system had increased workload. It was stressed that the system has increased the workload during a time when there have been cuts in the number of staff. It is not possible to determine the extent to which each of these variables is affected by the introduction of system as staff cuts have also contributed to the current situation. Further research is needed in this area. The statements made about workload indicated a perception that it had increased and was more labour intensive: “You…spend too much time on the computer when the job is to fix the item up…” “ all the paperwork that used to be carried out in say five minutes is now taking half an hour.” “If anything it's just increased our workload at a time when spending cuts, manning cuts are just critical. Time and the system just absorbs man hours.” Relationships in the Workplace. Work relationships were considered under three separate headings: relationships with colleagues, superiors and juniors, co-operation in the workplace, and communication in the workplace. AWRE’2002 209 i. Relationships with Colleagues, Superiors and Juniors Work relationships were discussed during the interviews. The question focused on the relationships with colleagues, supervisors and juniors and whether these relationships had changed in any way since the system was introduced. Twenty percent of respondents said that the system had no effect on their work relationships, and only one interviewee said that their work relationships had been improved by the system. The remaining 73% reported their work relationships had been adversely affected. More than half of those who said their work relationships had been affected claimed that the level of frustration had increased. The following quotes are typical of the sentiment expressed: (the system) “might have increased the stress level more than anything …frustration maybe …it’s more stressful so it’s definitely impacted on the relationships at work.” “They’re continually arguing with people who come in. Tradesmen, arguing, discussing, pointing out where they’re going wrong with the system.” ii. Co-operation in the Workplace Issues regarding co-operation in the workplace are difficult to ascertain using questionnaires and other more structured techniques. Structured interviews were utilised in order to explore the level of cooperation in the workplace. Most respondents (74%) indicated that the system had a negative affect on co-operation. While (13%) indicated that the system had not had any affect on the level of co- operation in the workplace, but several then went on to describe how co-operation had been affected. There were also indicators that the system was causing frustration and delays and was adversely affecting productivity. The following statement was common throughout the structured interviews: “…You’re sort of hanging around while one person puts his stuff in and (then) the next person, and those two people could have been helping or (doing) something else straight away rather than having to wait around.” Where the level of co-operation has improved it was likely to be because of the situation hinted at in the following statement: “It’s made an improvement (in co-operation) because the guys are trying to help each other out.” The general effects on the workplace that were mentioned in the context of slowness of the work processes and system response time included those that will have effects on co-operation: “… frustration comes also because ….I think they're letting everyone else down because they can't do as much work as they used to do because they're being hamstrung by a system that's slow” In the case of negative statements, these covered quality of work, morale, frustration and the time taken to use the system and can be exemplified by one quotation: "It effects your quality because it directly affects morale because it is so frustrating and time consuming. It takes your mind set off your job because you're actually got to start concentrating on overcoming the problems in the system and you stop to think about what your actual job is that you should be doing." (our emphasis) AWRE’2002 210 iii. Communication in the Workplace There were 7 questions in the questionnaire relating to communication. These related to the type of communication and the mode of communication used in the workplace The overall response to the questions was neutral (Mean = 3.79). However, the responses to the first three questions referring to improvements in communications were negative. Respondents did not consider that the system improved informal communication or communications in the work place overall. The structured interview delved deeper into this issue in order to ascertain the changes in communication style and frequency that may have taken place since the introduction of the system. Only 25% of the statements were neutral (indicated no change in communication), 31% were positive indicating that communication had improved in some way since the introduction of the system and 43% were negative indicating communication has been disrupted in some way since the introduction of the system. The following comments are representative of those surveyed: “I use the phone a lot more to ring help, emails and stuff, no. I just write a lot more post-it notes….” “I don’t think it’s made much difference.” Interpreting the data was not easy as some participants state that communication had not changed, and went on to describe how communication has changed since the introduction of the system. Most of those who commented directly during interviews upon their relationships in the workplace with superiors, colleagues and juniors reported that relationships had been adversely affected since the introduction of the system. While relatively few statements were made concerning co-operative behaviours at work, most of those made suggested the system had reduced co- operation. An interesting viewpoint was the notion that the system had increased co- operation in the workplace by uniting users against a 'difficult' system. An increase in the sense of frustration experienced by users in general was noted as was its effects on work relationships. Users identified some facets of communication in the workplace that had suffered as a result of the system (informal communication, informal networks and workplace communication overall). They also reported increases in telephone use. Implementation Issues When a system is implemented there is an expectation that the system will work as it was intended and that is it has been implemented properly. It was brought to light during the interviews that there were a number of ‘implementation’ problems associated with this system. In particular the accuracy of data contained on the system was questioned, access to the system could be problematic, the system speed was considered inadequate and there were problems in relation to how the users were being forced to input information in a rigid and non-intuitive way. Comments directly referring to accuracy related to 3 different things: • The accuracy of the data contained in the system • Use of the system which led to information not being entered into the system, and • Corrupted and unreliable information. Comments that were made about the system being inaccurate included: AWRE’2002 211 “When you’re moving a part for a major service, because of the way the system is you have to virtually say you’re removing it for access and that’s incorrect.” “Sometimes the information is wrong” "Attention to detail now is much greater, it has to be because you've got to check the system against what you know to be right." A typical example of the comments relating to the use of the system and how this impacts on the accuracy of information: “I will often pull a report and look at the history of the part…We find out later that they’re not using the system for specific tasks, so that if I want to track a history of the part, all of a sudden I’ve got big holes in the history of what’s happened to it. …And in actual fact we’re actually getting a corrupt database because everything that happens to a part isn’t being seen by the system.” Additional comments made in relation to corrupt information included: “…with the data migration(moving data from one site to another) it came over extremely corrupt.” Some users commented on the processes forced upon them for no (identifiable) task related reason. For example: “When you put a part number in to do a print report you have to put a star…….. Why? If you don't put that star on it still goes through the whole process of generating a report, but you've got no data in that report if you miss the star…“ Other comments were made about the look-and-feel of the system in relation to creating a "look-alike-but-not-behave-alike" system because the developers had attempted to implement a windows look-alike with a result that was similar enough to result in expectations of MS Windows behaviour but which often did not exhibit the behaviours in subtly misleading ways, for example; “…you know how in Windows you've got little boxes up here (referring to the close & resizing boxes in the MS Windows Window Title Bar)…. But it's intuitive in some cases to go and hit that box to close the window. If you do that you crash the system. “This is a Windows look alike that runs on a Unix background but the amount of little quirks that up front get you was quite amazing.” The users experience of the causes of the slowness covered a stated lack of terminals in the workplace and the amount of time spent using a computer as a result of the work processes imposed by the system being more labour intensive. Comments typical of those concerning access to terminals are: “Limited amount of terminal access doesn't help.” “The thought of having to (go) from my work position to a terminal is distance (travel) time lost again. “ “…and I don't have immediate access to a system terminal for a start..." A lack of certain types of functionality also has safety implications such as: “When you remove an item for just access it doesn’t prompt you to say whether its in working order or not.” AWRE’2002 212 “If you pull your main item off, it (the system) pulls everything else off with it but it doesn’t give job numbers …but when you can pull it off and put it back on and the computer says it never came off.” Conclusion Reasons given for the failure of new IT systems to meet expectations have been cited in previous research as being largely social and behavioural and not technological Clegg et al (1997). Successful management of human and behavioural factors during system design and implementation phases is vital for system acceptance. All of these have figured in our research into the usage of technology over a number of years. The approach taken in this study to investigate factors contributing to successful technology uptake were chosen because they were felt by us to complement each other. The usability (Shackel 1991; Eason 1988) approach covers the human factors side of things as does the TAM (Davis 1986; Davis 1989; Davis et al 1989; Davis et al 1996) but from a different perspective. The organizational and workplace behavioural side have in general been neglected in the investigation of the acceptance of technologies. What we were in effect doing during this study was tapping into the attitudes of highly skilled job performers to the effect that this system had upon their job and their job performance. The external variables included in the original TAM model (Davis 1986) can be related to requirements issues; organizational, human and technical attributes of the system such as design features of the information system, educational programs, publicity, advertising and feedback generated by other user’s experiences, and to implementation issues relating to data accuracy and system reliability. By including the organizational factors (ethos, work practice, effect on workplace relationships) in our model we were attempted to move a proportion of the variables, considered by Davis (1989) as ‘external’, into the model. We included a number of additional scales in the questionnaire trying to tap into some of these attributes (work processes, information supply, training, help etc). The factors that the authors propose influence technology uptake include both organizational and technical factors our research highlights areas where the system being studied failed to meet user expectations. These included several requirements issues; HCI (individual), communication, aspects of the physical environment, organization structure, procedural change, formal and informal systems, social systems (work relations), and selection and training. Implementation issues that also failed to meet expectations including inaccurate data, information not being entered into the system (due to requirements issues not being met) and corrupted and unreliable information. It is apparent that the implementation process of this system was deficient in a number of ways and that these deficiencies resulted in the workforce becoming increasingly stressed and demoralized because they were unable to perform their jobs to the same high standard as previously. Particularly worrying were the affects noted about the organizational impact of the system including: a decrease in organizational effectiveness because of changed work processes and changes in work behaviours, increased workload, making work for others, decreased flexibility; and deterioration of relations with co-workers (including superiors and juniors). It is also questionable whether the changes necessary to make this system meet the requirements of users and of the organization are feasible and affordable. AWRE’2002 213 References Adams D., Nelson, R., Todd, P. (1992) Perceived usefulness, ease of use, and usage of information technology: A replication. MIS Quarterly, June, 227-247. Clegg, C., C. Axtell, L. Damodaran,, B. Farbey, R. Hull, R. Lloyd-Jones, J. Nicholls, R. Sells, C. Tomlinson. Information technology: A study of performance and the role of human and organizational factors. Ergonomics, 40, 851- 887. Creed W.E.D., Stout, S.K., Roberts, K.H. (1993) Organizational effectiveness as a theoretical foundation for research on reliability-enhancing organizations. In K. H. Roberts (Ed.), New challenges to understanding organizations. Macmillan, 1993, pp. 55-74. Damodoran L.W., Olphert (2000) Barriers and facilitators to the use of knowledge management systems. Behaviour & Information Technology, 19 (6) 405 – 413 Davis F.D. (1986) A Technology Acceptance Model for Empirically Testing New End- user Information Systems: Theory and Results. Doctoral dissertation. Sloan School of Management. Massachusetts Institute of Technology. Davis F.D., (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, September, 319-340. Davis F.D., Bagozzi, R., Warshaw, P(1989) User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35, 982-1003. Davis F.D., & Venkatesh, V. (1996) A critical assessment of potential measurement biases in the technology acceptance model: Three experiments. International Journal of Human-Computer Studies, 45, 19-45. Eason K.D.(1988) Information technology and organizational change. London, Taylor Francis. Fishbein M., Ajzen I. (1975) Belief, Attitude, Intention and Behaviour: An Introduction to Research and Theory. Reading, MA: Addison-Wesley. Igbaria, M., Schiffman, S., Wieckowski, T. (1994). The respective roles of perceived usefulness and perceived fun in the acceptance of microcomputer technology. Behaviour and Information Technology, 13, 349-361. Ouellette J.A., Wood W. (1998) Habit and intention in everyday life: The multiple processes by which past behaviour predicts future behaviour. Psychological Bulletin, 124, 54-74. LaPorte T.R., Consolini P.M. (1991) Working in practice but not wrong in theory: theoretical challenges of high reliability organisations. Journal of Public Administration Research and Theory. 1, 19 – 47. Prasad P., Prasad A. (1993) The ideology of professionalism and work computerization: An institutionalist study of technological change. Human Relations, 47 (12) 1443 – 1458. Senge P.M. (1992) The Fifth Discipline: The Art and Practice of the Learning Organisation. Random House, Australia. Shackel B. (1991) Human Factors for Informatics Usability. Cambridge University Press, Cambridge. Walton E.J., Dawson S. (2001) Managers’ perceptions of criteria of organizational effectiveness. Journal of Management Studies, 38 (2) 173 – 199.