=Paper=
{{Paper
|id=Vol-69/paper-16
|storemode=property
|title=The Human and Organizational Impacts of Technology Uptake and Use
|pdfUrl=https://ceur-ws.org/Vol-69/paper16.pdf
|volume=Vol-69
}}
==The Human and Organizational Impacts of Technology Uptake and Use==
https://ceur-ws.org/Vol-69/paper16.pdf
AWRE’2002 199
The Human and Organizational Impacts of Technology
Uptake and Use
Christine Wood and Terry Webb
DSTO Fern Hill, Dept of Defence, CANBERRA ACT 2601
christine.wood@dsto.defence.gov.au
terry.webb@dsto.defence.go.au
Abstract
Any particular IT system can be specified in many different ways, reflecting the
processes and practices of the organization and organizational culture. It is important
to understand the interconnections between IT and the organization and to understand
the implications of impacts of different systems on the organization. This paper
explores some of the human and organizational needs and requirements of an IT
system in relation to:
• requirements issues where although the complete system and/or individual
components of the system work as intended by the developers users are
dissatisfied and
• implementation issues where the complete system and/or individual components
of the system does not work as was intended by the developers (e.g. crashes or it
produces erroneous data).
A user perception survey was conducted in a large government department after the
introduction of a computerized maintenance management system. Results indicated
that users were largely dissatisfied with the system.
Keywords:
requirements, usability, perceived ease of use, perceived usefulness, technology
acceptance model, organizational effectiveness, training, technology uptake.
Introduction
A number of factors contribute to overall user satisfaction and uptake of new IT
systems within organizations. These can be broadly classified into two categories;
requirements issues which includes usability, perceived ease of use, perceived
usefulness, communication, information about the system and training. The other is
implementation issues which require that a system work as it was intended and is
reliable. This includes a lack system crashes, access to the system, data integrity,
system speed, and that the system does not produce any errors.
Users and organizations develop expectations regarding how a system will perform
before the system is implemented. Both formal and informal methods are employed
by organizations and within organizations to build expectations. Prasad and Prasad
(1994) described a study of the computerization of a health administration system in a
Health Care Organization. They found that potential users had discussed and
anticipated the arrival of the new system and had categorized the implementation of
the system as a professional and professionalising process. This helped create a
climate of acceptance where personal concerns were reduced. Although management
AWRE’2002 200
did little to involve employees in the implementation process, or to understand the
social roles and networks as a result of the computerization process, commitment to
using the new system was high.
We report the results of a user perception survey of the implementation of a Computer
Maintenance Management system in a large government department. In this case the
organization used a variety of promotional materials (ranging from printed matter to
information contained in an intranet site). Potential users were also invited to
comment on the design of the system prior to and during its introduction. An
extensive training program was also implemented. All of these factors contributed to
users having a high expectation of the new system.
The foundation of our research derives from three areas of research; the Technology
Acceptance Model also known as TAM (Davis 1986, 1989, Davis Bagozzi & Warsaw
1989, Davis & Venkatesh 1996), Organizational Effectiveness (Creed, Stout &
Roberts 1993; LaPorte & Consolini 1991, Damadoran & Olphert 2000) and Usability
(Shackel, 1986, Eason, 1988). The TAM (see Figure 1) was developed by Davis
(1986) in an effort to predict, explain and increase user acceptance of technology. It
is based on the Theory of Reasoned Action (Fishbein & Ajzen, 1975) and investigates
the ability to predict peoples’ acceptance of technology including computer
acceptance and to predict intentions from attitudes, perceived usefulness, perceived
ease of use and other related variables. External variables affect perceived ease of use
and perceived usefulness, which in turn influence behavioural intention which
contributes directly to actual usage. In this model external variables provide a link
between internal beliefs, attitudes and intentions and may include system design
features, training, user involvement in design and information about the system.
Perceived
Usefulness
Behavioural Intention Actual System Usage
External Variables to Use
Perceived Ease
of Use
Figure 1. The Technology Acceptance Model (Davis, et al, 1989).
Perceived ease of use has a direct effect on perceived usefulness and is defined as “the
user’s perception of the extent to which using a particular system will be free from
effort” (Davis & Venkatesh 1996, p20). Perceived usefulness is “the user’s perception
of the degree to which using a particular system will improve his/her performance”
within an organizational context.” (Davis & Venkatesh 1996, p20).
The organizational effectiveness measures were derived from the work of Creed et al
(1993) and Prasad & Prasad (1994). It is frequently the potential of the technology,
not the prevailing political or social forces that drive concern for effectiveness.
Certain technologies pose risks for an organization that must be addressed; an
organization’s decision to implement technologies is often based on technical rather
than organizational or human considerations. Considering technology as a
fundamental of the culture of the organization can foster a greater understanding of
the processes and practices that impact organizational effectiveness. It is necessary to
think about the nature of an organization’s technology, the dynamics of the evolution
AWRE’2002 201
of technology, and the implications of technological evolution on organizational
effectives (Creed et al 1993). Damodoran & Olphert (2000), and Prasad & Prasad
(1994) stress that technology can support and maintain a beneficial shift in
organizational culture, and that this shift highlights the value of new technologies and
promotes their use. Two aims of highly effective organizations are: 1) to manage
complex demanding technologies ensuring they avoid failures that would adversely
affect the organization and 2) to maintain the capacity to meet periods of very high
demand often under considerable time pressure (LaPorte & Consolini, 1991). In
assessing organizational effectiveness it is necessary to consider values and
preferences, internal and person orientation, and to focus on external factors such as
political and social pressure. Although staff and management may have little control
over external events they nonetheless impact organizational performance.
Technology is often used to leverage organizational change in an effort to improve
effectiveness and efficiency. The effective use of technologies requires major
organizational and individual learning (Eason, 1988; Senge, 1992).
The other body of research utilised was that of usability which is closely tied to fitness
for purpose and ease of use. Shackel (1991) suggested that usability, as he defined it,
was comprised of four components or criteria:
• Flexibility (e.g. a capacity to cope with some specified deviation from the
specified environment),
• Learnability (e.g. effective use shall be developed within a pre-defined training
scheme and system of user support),
• Effectiveness (e.g. the required range of tasks must be accomplished at equal or
better than a specified performance level), and
• Attitude (e.g. there will be acceptable levels of human cost in terms of fatigue,
discomfort, frustration, personal effort - these being the factors that are most likely
to colour a user's attitude toward the system).
In 1988, Eason suggested that usage of an information system is the single most
reliable indicator of usability. In those environments where the user has discretion
over their use of a system, usage will decline as usability lessens and may decline to
the point where the system will be discarded. The relationship between usability and
usage will be moderated by the usefulness of the system. In effect, trade-offs are
made between usefulness and usability.
Method
Participants were from a large Government Department and were assigned to attend
questionnaire and interview sessions by their local line management. No further
control could be exercised over the sample in terms of its structure and
representativeness. Everyone attending a session completed the questionnaire and up
to six persons per session were selected for structured interviews. At the time of this
study the system had been in use for approximately two months and users were
required to use part of the old system (a paper based, semi-computerised system) in
combination with the new system to record maintenance information. Two primary
means were used for data collection in this study: A self-report questionnaire and
structured interviews. Focus groups were utilised during questionnaire development.
Questionnaires and Structured Interviews
The technology uptake questionnaire was comprised of previously developed scales
such as the TAM (Davis, 1986, Davis & Venkatesh, 1996) as well as purpose-derived
AWRE’2002 202
scales relating to organisationally relevant constructs and system usability (Creed et al
1993, Shackel 1991). Prior to interviews or completing questionnaires participants
signed an informed consent statement. Participants were asked to respond based on
their experiences with the system. This paper reports on the results of the interview
data and will only make reference to data from the questionnaires where it is
particularly salient. The interview structure followed that used in the questionnaires
and contained a 15 probe topics designed to elicit and explore information about the
system, its acceptance, and use that would not be apparent in the questionnaire data.
Interviewees were also given the opportunity to raise any other issues.
Most questions on Technology Uptake were included on the basis of previously
published research (Davis, 1989), (Adams, et al 1992) and (Igbaria, 1994) and
covered the areas of: past behaviour, behavioural intention, perceived usefulness,
perceived ease of use and perceived behavioural control. Other questions were
derived from three sources: Shackel's (1991) usability criteria (work processes,
confidence and familiarity, training, help, and information supply). The questions
relating to organizational effectiveness are based on the work of LaPorte & Consolini
(1991) and Creed et al (1993) and have been adapted to study organisational
behaviours and technology uptake. The third source was derived from the system
promotional materials.
A total of 21 Interview transcripts were analysed by abstracting and collating the
statements made and, where appropriate, classifying them as positive, neutral or
negative. The number of statements made on any given topic may exceed the number
of participants in the structured interviews as users often utter more than one
statement on the same topic. This qualitative data served to clarify the quantitative
data collected using the questionnaires. Ninety-five participants completed the
questionnaire but some respondents did not answer all questions.
Requirements Issues
Perceived Usefulness
Perceived usefulness is the degree to which using a particular system will improve the
users performance within an organizational context (Davis & Venkatesh 1996).
Responses in the structured interviews to questions about the system’s usefulness
were predominantly negative. Of the 181 statements considered with usefulness, only
12.7% were positive, 13.3% neutral, and 74% negative. Over 22% of these
statements were to do with the extra time it takes to get work done with the system.
The amount of time required to complete tasks appears to have dramatically
increased. Some statements made about this include:
“…The amount of time you have to spend on the computer it's lengthened the actual repair
process. It definitely has not streamlined it….”
“I haven't found a single thing that it can do that can help me. It can't give you any benefits
that the old system did at this point that I've found.”
There was a range of estimates of the increase in time needed to complete tasks:
“it's taking twice as long…” to “It would slow me down five times.”
Other negative statements concerned the increased workload:
AWRE’2002 203
”…you've got to do ten times as many transactions compared to the old system to get the
same direct result.”
“…it's been more a hindrance than anything… We've got to pull a part off on the system and
make it serviceable get another part and make it serviceable, fit it … put all the hours in, it
just takes a long time especially when you’re doing 12 things”
However, while feelings about the system are predominantly negative, there are some
positives. It was also proposed that the system has potential, if only some work is
done on it. For example:
“I think (the system) has got the potential to be really good, it’s got a lot of very good
information there and probably very good as a management tool.”
The questionnaire data collected regarding the perceived usefulness of the system (see
Figure 2) indicates that in all areas respondents perceived the system highly
negatively i.e. assigning scores very close to the ‘Strongly Disagree’ pole of the scale
(Mean = 1.91, SD = 1.06 N = 92). The possible range of score was 1 to 7. Standard
7
6
5
4
3
2
1
The system Using the Using the Using the Using the Overall, I find
allows me to system system system system makes the system
accomplish increases my improves my enhances my it easier to do useful in my
tasks more productivity. job effectiveness my job. job.
quickly. performance. on the job.
Figure 2. Mean Perceived usefulness of the system. (1 - strongly disagree; 4 -
neutral; 7 - strongly agree).
deviations for the Perceived Usefulness scale indicate a high level of consistency in
scoring.
Usability
Usability is the capability of a system to be used by humans easily and effectively
(Shackel, 1991). Good system design depends on solving the interactions between
users, tasks, tools and the environment in which they work. The level of usability is
judged by users in relation to their perceptions of how easy the system is to use and by
measures of effectiveness. Elements of usability are reported below.
AWRE’2002 204
i. Ease of Use
The Ease of Use Criteria used were classified according to the Shackel Usability
Criterion into which they best fit (See Table 1.); there is some overlap as there are
several possible secondary mappings for each of the scale items.
Item Ease of Use Scale Shackel Criterion
1 Learning to operate “system name” was easy for me. Learnability
2 Interacting with “system name” is clear and Learnability
understandable.
3 I find it takes a lot of effort to become skilled at using Learnability
“system name”.
4 I find it easy to get “system name” to do what I want it to. Effectiveness
5 “System name” is rigid and inflexible to interact with. Flexibility
Table 1.Ease of use classified according to Shackel’s (1991) Usability criteria.
It was made clear during the structured interviews that the system was not easy to use.
Strong negative feelings toward the system were expressed by a number of different
interviewees. Of the 94 statements relating to ease of use 26.5% were positive, 3%
were neutral and 70% were negative. Most negative responses referred to producing
reports, extra time required to operate the system, inconsistent system behaviour,
frustrating delays and the lack of support for replacing optional equipment (e.g.
various racks). With reference to the positive statements relating to ease of use, the
following are typical of those surveyed:
“Having the system spread over a wide area is one on the positives…rather than having one
central area where you record maintenance.”
“Once you’ve worked out how to pull a report the information you get on them is really
good.”
The comments made on ease of use were predominantly negative (70%). Some
commented on access to the system, others on how time consuming it was, others on
how frustrating it was and the lack of functionality. The following are examples of
the criticisms relating to the use of the system:
“It’s something that requires a lot of effort to use.”
“In terms of being a replacement (for the paper-based system) it does it in a very
cumbersome manner.”
This quotation sums up the attitude of users:
“They could have done it better … but I suppose we’ve been built up for so many years about
this system, about how good it was going to be and it was such a disappointment.”
In general users did not assess the system as being easy to use. Problems particularly
mentioned included a lack of consistency in its working/availability, a lack of
consistency in its human-computer interface, a ‘one-size-fits-all’ approach to interface
design i.e. separate appropriate interfaces not designed for specific work groups/tasks,
and questionable accuracy of the data in the system.
AWRE’2002 205
ii. Training
Training can be regarded as the prime factor directly contributing to the learnability of
a system. Effective use of a system is developed as a result of a pre-defined training
scheme. The desirable attributes of such a training scheme are that it: reflect the
system to be used, and is appropriate to the user’s tasks. The training provided for the
system was perceived negatively by most questionnaire respondents and was regarded
as providing little assistance in using the system (Mean = 3.42, SD = 1.33, N=91).
The users who were interviewed tended to be negative also (64% of comments made
were negative and only 17% positive). Two related themes emerged when
considering the negative comments that had been made: training was too generic and
lacked relationship to task (not related to their skills or specialisation). In addition,
users noted the need for training to be conducted close to the delivery of a system.
When considering the single training related item from the Perceived Behavioural
Control Section of the questionnaire it is apparent that users considered the assistance
to use the system that was provided by the training regime to be marginal at best. The
majority of interview statements relating to training were negative – 64%, 19% were
neutral and 17% were positive statements. The positive statements tended to relate to
what they thought of the course at the time they did it rather than what they thought of
their training subsequently.
“… the training package in general was good.”
“When we actually did the Computer Based Instruction…everyone came away thinking this is
going to be good.”
The negative views reflect problems with either course mechanics, such as:
“The training database…. There’d be a couple of ways it (the system) could do something,
but it would only let you do it one way at a time then make you do it a different way another
time. The core training information on the system I was using was corrupt and you couldn’t
do some of the scenarios that were given.”
There was a consistent theme that the training was too generic i.e. not designed to
support their job or the type of work they did:
“Basically they (training) were generic...It was really irrelevant”
“It wasn't directed at my type of work or the work shop environment, it was just a general
overview of every function they thought was necessary…. They just wasted days of training.
It was a five day course and they could have done the transactions that we use in the
workshop in one day.”
The above criticisms strongly indicate that, in the users’ view, the training definitely
did not reflect the system to be used and was not appropriate to the user’s tasks. A
number of people felt a major flaw was a lack of relation to the actual job, and in
particular the time lag between training and use at work which was, eventually, many
months!
Organizational Effectiveness
A number of factors contribute to organizational effectiveness. This research project
investigated work processes, work behaviours, workload, relationships in the
workplace and communication. Each of these will be discussed in more detail below.
AWRE’2002 206
i. Changes to Work Processes
A total of 138 statements were collected in the structured interviews in relation to
work processes. Of these, 5% were positively oriented toward Work Processes
undertaken since the introduction of the system, 14% were neutral and 81% were
negative. This data is remarkably consistent with the questionnaire data. The positive
statements made included one person's assessment:
..... “So in many ways it has improved the quality of work."
Hopes for the future were also expressed:
"So with more training I'd expect it would become more useful."
"I believe it will be better but that's a long way down the track."
The remaining 'positive' statements are more related to being 'comrades in adversity'
and to having to verify the system against what is ‘known’ than to the value of the
system:
" It's made each of us in the workshop rely more on someone else in the workshop to maybe
answer a problem in regards to the system, so it's probably built on the teamwork that way."
In general, two categories of negative statement were salient. Firstly, 29% of these
statements related to the increase in time taken to perform tasks because of the system
and, secondly, the 10% of statements directly addressed workload. The time taken to
use the system (for whatever reason) was assessed by many respondents and a
conservative estimate given for the extra time needed was:
"At the moment we've virtually tripled our work."
One of the major concerns was for the effect on productivity:
“… do a couple of repairs and come back and then spend like three hours in there signing up
and that's down time when … could have been doing more work."
In the questionnaire users were asked to rate on a scale from 1 (strongly disagree) to 7
(strongly agree) if they believed the system to be targeted and suited to the work done
by themselves, their unit, trade, and the organization as a whole (Mean = 3.08, SD =
1.31, N = 92). Other questions asked whether respondents believed that system is well
targeted to the work and organisational needs of the organization, their unit, their
trade group, and their level of specialisation (Mean = 3.00, SD = 1.28, N = 92).
Responses to these questions were consistently below the neutral point (4) on the
scale (See Figure 3) indicating users perceive the system to be poorly targeted to the
work and organisational needs at a range of levels (from the organisation-wide, down
to their own units and trade groups). Users also responded to the statement: “I
consider that the work of the organization has been improved by the introduction of
the system.” The mean of 1.97 is low and the standard deviation (1.30) indicates that
an overwhelmingly large proportion of the respondents (frequency count = 93.5%)
scored this item below the neutral point of the scale indicating that they do not
consider the work of the organization to have been enhanced by the introduction of
the system.
AWRE’2002 207
7
6
5
4
3
2
1
The system is well targeted to the
The system is well targeted to the work of
The system is well targeted to the work of
The system is well targeted to the work of
The system is well targeted to the work of
The system is well targeted to the
The system is well targeted to the
organisational needs of the Organization as
organisational needs of my Group.
organisational needs of people who are at
people who are at my level of Specialization
organisational needs of my unit.
organisational needs of my Trade.
The system is well targeted to the work of
The system is well targeted to the
The system is well targeted to the
the organization as a whole.
my level of Specialisation
my Group.
my Trade.
my unit.
a whole.
Figure 3 Results from work processes items from the questionnaire (1 - strongly
disagree; 4 - neutral; 7 - strongly agree).
Users did not assess the system to be well targeted to the organisational and work
needs of themselves, their units, their trade group or the organization as a whole.
Users stated unequivocally and frequently that the system had slowed down their
work rate and increased their workload. The system is seen as being more labour
intensive and therefore it takes longer to complete each task. Limited access to
terminals and printers was an exacerbating factor.
ii. Changes to Work Behaviour
Using recall (hindsight), users perceived minor but negative changes in work related
behaviours since the introduction of the system. These behaviours involved small
(but statistically significant) negative shifts in: attention to detail, being alert to
potential accidents, doing the job well, not making work for others, ownership of
problems, understanding the job boundaries of others and, working flexibly. The
perceived increase in workload attributed to the system needs to be considered in the
context of existing staff shortages. However, given the parallel perception that time-
on-task has increased, it is possible that much of the perceived increase in workload is
system related. The question of a decline in morale was also raised.
Of the statements that relate to work behaviour the majority of statements (94%)
indicated that there had been a negative effect on work behaviours since the
introduction of the system. It is apparent that the depth of feelings expressed during
the interviews was not captured by the questionnaires. The comments from the
interviews show that the system has had negative impact on them in a number of areas
AWRE’2002 208
relating to work behaviours including; making work for others, relying on others,
attention to detail, being alert to the potential for serious accidents, working flexibly,
and changes in job boundaries. Some typical examples of these quotes relating to the
system making work for people include:
“You’re always making more work for your mates now”
“Now basically if you stop work at 10 o’clock or so, so you’ve got 2 hours to finish the
system work, so you’re leaving more jobs for the other shift which is in turn increasing their
workload….”
“…before you’d get say three hours of productive work, we’re now getting one and the rest of
the time is admin on the system.”
Fifteen percent of the statements related to safety. There are a variety of issues raised
here which are of concern because they are indicators of problems in terms of
occupational health and safety. Typical of the issues raised is:
“We’re having to think of a lot of ways to make the system do what we now do safely. Not
only safely, but correctly and record what needs to be recorded.”
One of the other questions discussed at the interviews was about how the system had
changed the way people had to rely on others in order to do their job. Comments
specifically referred to having to rely on others more in order to solve problems
directly associated with the system. In relation to working flexibly it was reported
that the system has reduced flexibility in getting work done. There also appear to be
affects in relation to changing work boundaries such as expanding the manager’s role
to include “counselling” and consoling workers:
“We’re now having to deal with an induced morale problem because of the frustration of the
system. So quite often, and more often than on a daily basis we are having to say to people
“look calm down, relax …just settle down and take your time and do it” because people are
just getting frustrated with it.”
iii. Workload
Interviewees raised workload issues and the general consensus was that the system
had increased workload. It was stressed that the system has increased the workload
during a time when there have been cuts in the number of staff. It is not possible to
determine the extent to which each of these variables is affected by the introduction of
system as staff cuts have also contributed to the current situation. Further research is
needed in this area. The statements made about workload indicated a perception that
it had increased and was more labour intensive:
“You…spend too much time on the computer when the job is to fix the item up…”
“ all the paperwork that used to be carried out in say five minutes is now taking half an
hour.”
“If anything it's just increased our workload at a time when spending cuts, manning cuts are
just critical. Time and the system just absorbs man hours.”
Relationships in the Workplace.
Work relationships were considered under three separate headings: relationships with
colleagues, superiors and juniors, co-operation in the workplace, and communication
in the workplace.
AWRE’2002 209
i. Relationships with Colleagues, Superiors and Juniors
Work relationships were discussed during the interviews. The question focused on
the relationships with colleagues, supervisors and juniors and whether these
relationships had changed in any way since the system was introduced. Twenty
percent of respondents said that the system had no effect on their work relationships,
and only one interviewee said that their work relationships had been improved by the
system. The remaining 73% reported their work relationships had been adversely
affected. More than half of those who said their work relationships had been affected
claimed that the level of frustration had increased. The following quotes are typical of
the sentiment expressed:
(the system) “might have increased the stress level more than anything …frustration maybe
…it’s more stressful so it’s definitely impacted on the relationships at work.”
“They’re continually arguing with people who come in. Tradesmen, arguing, discussing,
pointing out where they’re going wrong with the system.”
ii. Co-operation in the Workplace
Issues regarding co-operation in the workplace are difficult to ascertain using
questionnaires and other more structured techniques. Structured interviews were
utilised in order to explore the level of cooperation in the workplace. Most
respondents (74%) indicated that the system had a negative affect on co-operation.
While (13%) indicated that the system had not had any affect on the level of co-
operation in the workplace, but several then went on to describe how co-operation had
been affected. There were also indicators that the system was causing frustration and
delays and was adversely affecting productivity. The following statement was
common throughout the structured interviews:
“…You’re sort of hanging around while one person puts his stuff in and (then) the next
person, and those two people could have been helping or (doing) something else straight
away rather than having to wait around.”
Where the level of co-operation has improved it was likely to be because of the
situation hinted at in the following statement:
“It’s made an improvement (in co-operation) because the guys are trying to help each other
out.”
The general effects on the workplace that were mentioned in the context of slowness
of the work processes and system response time included those that will have effects
on co-operation:
“… frustration comes also because ….I think they're letting everyone else down because they
can't do as much work as they used to do because they're being hamstrung by a system that's
slow”
In the case of negative statements, these covered quality of work, morale, frustration
and the time taken to use the system and can be exemplified by one quotation:
"It effects your quality because it directly affects morale because it is so frustrating and time
consuming. It takes your mind set off your job because you're actually got to start
concentrating on overcoming the problems in the system and you stop to think about what
your actual job is that you should be doing." (our emphasis)
AWRE’2002 210
iii. Communication in the Workplace
There were 7 questions in the questionnaire relating to communication. These related
to the type of communication and the mode of communication used in the workplace
The overall response to the questions was neutral (Mean = 3.79). However, the
responses to the first three questions referring to improvements in communications
were negative. Respondents did not consider that the system improved informal
communication or communications in the work place overall.
The structured interview delved deeper into this issue in order to ascertain the changes
in communication style and frequency that may have taken place since the
introduction of the system. Only 25% of the statements were neutral (indicated no
change in communication), 31% were positive indicating that communication had
improved in some way since the introduction of the system and 43% were negative
indicating communication has been disrupted in some way since the introduction of
the system. The following comments are representative of those surveyed:
“I use the phone a lot more to ring help, emails and stuff, no. I just write a lot
more post-it notes….”
“I don’t think it’s made much difference.”
Interpreting the data was not easy as some participants state that communication had
not changed, and went on to describe how communication has changed since the
introduction of the system. Most of those who commented directly during interviews
upon their relationships in the workplace with superiors, colleagues and juniors
reported that relationships had been adversely affected since the introduction of the
system. While relatively few statements were made concerning co-operative
behaviours at work, most of those made suggested the system had reduced co-
operation. An interesting viewpoint was the notion that the system had increased co-
operation in the workplace by uniting users against a 'difficult' system.
An increase in the sense of frustration experienced by users in general was noted as
was its effects on work relationships. Users identified some facets of communication
in the workplace that had suffered as a result of the system (informal communication,
informal networks and workplace communication overall). They also reported
increases in telephone use.
Implementation Issues
When a system is implemented there is an expectation that the system will work as it
was intended and that is it has been implemented properly. It was brought to light
during the interviews that there were a number of ‘implementation’ problems
associated with this system. In particular the accuracy of data contained on the
system was questioned, access to the system could be problematic, the system speed
was considered inadequate and there were problems in relation to how the users were
being forced to input information in a rigid and non-intuitive way.
Comments directly referring to accuracy related to 3 different things:
• The accuracy of the data contained in the system
• Use of the system which led to information not being entered into the system, and
• Corrupted and unreliable information.
Comments that were made about the system being inaccurate included:
AWRE’2002 211
“When you’re moving a part for a major service, because of the way the system is you have to
virtually say you’re removing it for access and that’s incorrect.”
“Sometimes the information is wrong”
"Attention to detail now is much greater, it has to be because you've got to check the system
against what you know to be right."
A typical example of the comments relating to the use of the system and how this
impacts on the accuracy of information:
“I will often pull a report and look at the history of the part…We find out later that they’re
not using the system for specific tasks, so that if I want to track a history of the part, all of a
sudden I’ve got big holes in the history of what’s happened to it. …And in actual fact we’re
actually getting a corrupt database because everything that happens to a part isn’t being seen
by the system.”
Additional comments made in relation to corrupt information included:
“…with the data migration(moving data from one site to another) it came over extremely
corrupt.”
Some users commented on the processes forced upon them for no (identifiable) task
related reason. For example:
“When you put a part number in to do a print report you have to put a star…….. Why? If
you don't put that star on it still goes through the whole process of generating a report, but
you've got no data in that report if you miss the star…“
Other comments were made about the look-and-feel of the system in relation to
creating a "look-alike-but-not-behave-alike" system because the developers had
attempted to implement a windows look-alike with a result that was similar enough to
result in expectations of MS Windows behaviour but which often did not exhibit the
behaviours in subtly misleading ways, for example;
“…you know how in Windows you've got little boxes up here (referring to the close & resizing
boxes in the MS Windows Window Title Bar)…. But it's intuitive in some cases to go and hit
that box to close the window. If you do that you crash the system.
“This is a Windows look alike that runs on a Unix background but the amount of little quirks
that up front get you was quite amazing.”
The users experience of the causes of the slowness covered a stated lack of terminals
in the workplace and the amount of time spent using a computer as a result of the
work processes imposed by the system being more labour intensive. Comments
typical of those concerning access to terminals are:
“Limited amount of terminal access doesn't help.”
“The thought of having to (go) from my work position to a terminal is distance (travel) time
lost again. “
“…and I don't have immediate access to a system terminal for a start..."
A lack of certain types of functionality also has safety implications such as:
“When you remove an item for just access it doesn’t prompt you to say whether its in working
order or not.”
AWRE’2002 212
“If you pull your main item off, it (the system) pulls everything else off with it but it doesn’t
give job numbers …but when you can pull it off and put it back on and the computer says it
never came off.”
Conclusion
Reasons given for the failure of new IT systems to meet expectations have been cited
in previous research as being largely social and behavioural and not technological
Clegg et al (1997). Successful management of human and behavioural factors during
system design and implementation phases is vital for system acceptance. All of these
have figured in our research into the usage of technology over a number of years. The
approach taken in this study to investigate factors contributing to successful
technology uptake were chosen because they were felt by us to complement each
other. The usability (Shackel 1991; Eason 1988) approach covers the human factors
side of things as does the TAM (Davis 1986; Davis 1989; Davis et al 1989; Davis et
al 1996) but from a different perspective. The organizational and workplace
behavioural side have in general been neglected in the investigation of the acceptance
of technologies. What we were in effect doing during this study was tapping into the
attitudes of highly skilled job performers to the effect that this system had upon their
job and their job performance.
The external variables included in the original TAM model (Davis 1986) can be
related to requirements issues; organizational, human and technical attributes of the
system such as design features of the information system, educational programs,
publicity, advertising and feedback generated by other user’s experiences, and to
implementation issues relating to data accuracy and system reliability.
By including the organizational factors (ethos, work practice, effect on workplace
relationships) in our model we were attempted to move a proportion of the variables,
considered by Davis (1989) as ‘external’, into the model. We included a number of
additional scales in the questionnaire trying to tap into some of these attributes (work
processes, information supply, training, help etc).
The factors that the authors propose influence technology uptake include both
organizational and technical factors our research highlights areas where the system
being studied failed to meet user expectations. These included several requirements
issues; HCI (individual), communication, aspects of the physical environment,
organization structure, procedural change, formal and informal systems, social
systems (work relations), and selection and training. Implementation issues that also
failed to meet expectations including inaccurate data, information not being entered
into the system (due to requirements issues not being met) and corrupted and
unreliable information.
It is apparent that the implementation process of this system was deficient in a number
of ways and that these deficiencies resulted in the workforce becoming increasingly
stressed and demoralized because they were unable to perform their jobs to the same
high standard as previously. Particularly worrying were the affects noted about the
organizational impact of the system including: a decrease in organizational
effectiveness because of changed work processes and changes in work behaviours,
increased workload, making work for others, decreased flexibility; and deterioration
of relations with co-workers (including superiors and juniors). It is also questionable
whether the changes necessary to make this system meet the requirements of users
and of the organization are feasible and affordable.
AWRE’2002 213
References
Adams D., Nelson, R., Todd, P. (1992) Perceived usefulness, ease of use, and usage
of information technology: A replication. MIS Quarterly, June, 227-247.
Clegg, C., C. Axtell, L. Damodaran,, B. Farbey, R. Hull, R. Lloyd-Jones, J. Nicholls, R. Sells,
C. Tomlinson. Information technology: A study of performance and the role of human and
organizational factors. Ergonomics, 40, 851- 887.
Creed W.E.D., Stout, S.K., Roberts, K.H. (1993) Organizational effectiveness as a
theoretical foundation for research on reliability-enhancing organizations. In K. H.
Roberts (Ed.), New challenges to understanding organizations. Macmillan, 1993, pp. 55-74.
Damodoran L.W., Olphert (2000) Barriers and facilitators to the use of knowledge
management systems. Behaviour & Information Technology, 19 (6) 405 – 413
Davis F.D. (1986) A Technology Acceptance Model for Empirically Testing New End-
user Information Systems: Theory and Results. Doctoral dissertation. Sloan School of
Management. Massachusetts Institute of Technology.
Davis F.D., (1989). Perceived usefulness, perceived ease of use and user acceptance
of information technology. MIS Quarterly, September, 319-340.
Davis F.D., Bagozzi, R., Warshaw, P(1989) User acceptance of computer technology:
A comparison of two theoretical models. Management Science, 35, 982-1003.
Davis F.D., & Venkatesh, V. (1996) A critical assessment of potential measurement
biases in the technology acceptance model: Three experiments. International Journal
of Human-Computer Studies, 45, 19-45.
Eason K.D.(1988) Information technology and organizational change. London,
Taylor Francis.
Fishbein M., Ajzen I. (1975) Belief, Attitude, Intention and Behaviour: An
Introduction to Research and Theory. Reading, MA: Addison-Wesley.
Igbaria, M., Schiffman, S., Wieckowski, T. (1994). The respective roles of perceived
usefulness and perceived fun in the acceptance of microcomputer technology.
Behaviour and Information Technology, 13, 349-361.
Ouellette J.A., Wood W. (1998) Habit and intention in everyday life: The multiple
processes by which past behaviour predicts future behaviour. Psychological Bulletin,
124, 54-74.
LaPorte T.R., Consolini P.M. (1991) Working in practice but not wrong in theory:
theoretical challenges of high reliability organisations. Journal of Public
Administration Research and Theory. 1, 19 – 47.
Prasad P., Prasad A. (1993) The ideology of professionalism and work
computerization: An institutionalist study of technological change. Human Relations,
47 (12) 1443 – 1458.
Senge P.M. (1992) The Fifth Discipline: The Art and Practice of the Learning
Organisation. Random House, Australia.
Shackel B. (1991) Human Factors for Informatics Usability. Cambridge University
Press, Cambridge.
Walton E.J., Dawson S. (2001) Managers’ perceptions of criteria of organizational
effectiveness. Journal of Management Studies, 38 (2) 173 – 199.