Information quality has been proposed as one of the determining factors for perceived information system success. Fehrenbacher and Helfert have studied contextual factors of information system use (e.g. different user types, types of business activities supported, etc.) that influence the perceived importance and the trade-offs of information quality criteria. In this paper, we will use their framework in a discussion of the findings from a study we conducted on the implementation of an electronic messaging system (emessaging) in Norwegian health care aimed at supporting collaboration between different health-care actors. The system has a high perceived success even though the information quality of the message content offers room for improvement according to health-care personnel using the system.
An evaluation of the use of an electronic messaging system has been presented, which aims at improving the collaboration between hospitals and community care [1, 2]. The overall outcome of the qualitative evaluation is that, in the eyes of the users: ‘The introduction of e-messaging in Norwegian health care can be considered a success story in that it has led to more efficient, higher-quality and safer patient transitions’[1]. The evaluation study does not address information quality systematically, although some challenges with respect to this in the evaluated implementation are reported [1]. These challenges relate to missing or incomplete information (e.g. an updated medication list) and too little standardised message content (e.g. the discharge report), indicating that at least information quality is not perceived as optimal.
Information quality is proposed as one of the independent variables that determines information system success [3]. Especially in the case of electronic messaging systems, this seems very reasonable. A messaging system that exchanges messages with a questionable information quality cannot be expected to be a success.
In this paper, we will address the question of the information quality of the messaging system in a more systematic way, by discussing its relation to the systems’ success. The paper is structured as follows. In the next section will we introduce the concept of information quality. In the following section, we will briefly describe our case and the research method applied. After that, the results are presented, followed by a discussion of our findings. 2
Information quality is a multi-model concept and many researchers have proposed different characteristics that identify it [4]. Research has furthermore demonstrated that information quality is a subjective feature, which can be referred to as: ‘...data or information that is fit for use’ [5]. As demonstrated, this assessment is influenced by contextual factors such as: the role of the person assessing information quality; the activity in which the information is used; the organisational context in which the information is used (which department or organisational unit); and available resources (for example time) [5].
In this paper, will we use characteristics previously proposed [5], as this list was derived from reviewing a large number of information quality frameworks. The characteristics included are listed in Table 1 below (the descriptions are adapted from [6], p.7).
Description The information is error free Information is easily accessible by authorised users, in the right format Information contains all the relevant facts Information is available when needed Information can be trusted Information cannot be accessed by unauthorised users The same information is represented in the same way Information is to the point Fehrenbacher and Helfert showed, based on an extensive survey, that these characteristics are weighted based on the context. Therefore, there seems to be a trade-off between them. For example, they found that the characteristic Timely is weighted as being much more important than Consistent Representation by people involved in primary activities than by people involved in support activities [5]. As another example, IT people valued the characteristic Conciseness as more important than Completeness, whereas non-IT people had the opposite assessment [5].
Therefore, information quality is a multi-modal, subjective assessment, where contextual factors determine how the characteristics contribute to the overall outcome. 3
E-messaging system: The figure below, which is taken from [1], illustrates the exchange of messages supported by the system that we studied. E-messaging has been introduced nationwide in Norwegian health care. The development and implementation of the e-messaging system was initiated by national health-care authorities [7] to improve information exchange and communication between community health-care services, GPs and hospitals. The implementation of the e-messaging system followed the acknowledgement that communication and information exchange between the providers was predominantly done orally, either via telephone or in face-to-face meetings, as well as via fax or postal letters. This meant that the communication of important patient information could be slow and fragmented and that health-care workers found it difficult to contact one another. As a result, insufficient understanding of patients’ needs could arise, jeopardising the quality of care [8, 9]. The e-messaging system was consequently introduced to ‘secure seamless patient trajectories across the health and care sector through electronic all-to-all communication’ [7 p.6].
The e-messaging system was developed as a module that could be integrated with the various electronic patient record (EPR) systems in use in Norway, among which there are substantial variations. Community health-care services throughout the country use three EPR systems, while hospitals use two EPR systems. Because these systems are not integrated, information cannot be exchanged automatically between them. However, the e-messaging system enables users to exchange some of the information stored in the EPRs. When composing a message, a user can retrieve some of the content of the message directly from an EPR; thus, it is not necessary to re-type information. Furthermore, information contained in a received message can be stored in an EPR. This integration of the e-messaging system with various EPR systems facilitates the implementation of the legal requirement that patient information must be exchanged when necessary[10].
Health-care setting: One large university hospital and three adjacent municipal homecare units were used as the setting for this study. The hospital and one of the municipalities were strategically selected because they had the longest experience with e-messaging. The other two municipalities were randomly selected. As for the information infrastructure, the e-messaging solution is integrated into the providers’ EPR systems. The staff started to use the e-message system progressively during the period 2011–2013.
Study design: Semi-structured interviews were conducted with 41 health-care personnel with a focus on their experiences with e-messaging during patient transitions between hospital and municipal-based home care services. Data material: The data collection took place between February and November 2014. The inclusion criterion was that informants must have worked for a minimum of six months to gain a certain level of experience with e-messaging. Staff were handed written information about the study and recruited by their managers. All authors, except Toussaint, participated in the data collection.
The informants were either interviewed individually or together in groups of two, three or four. Nurses constituted the largest group of informants whereas a few others were key personnel either working with e-messaging in care situations or involved with implementation and support of e-message systems. The interview guide focused on three main issues related to the e-messaging system: its efficiency; its influence on the quality of care; and its consequences for patient safety. The interviews lasted 30– 60 minutes and were audio-recorded and later transcribed verbatim by student assistants. Once transcribed, data were coded by hand to identify themes and patterns of themes. More in-depth descriptions of the analysis have been reported elsewhere [1]. Ethical issues: Approval was granted by the Norwegian Social Science Data Services. Written informed consent was obtained from all the participants. 4
Several positive effects of the introduction of the e-messaging system have been reported [1], which can be related to some of the information quality characteristics listed in Table 1. First, information on a patient’s illness history is more easily available for those who need it and there is less need to spend time in phone cues searching for information. This indicates that Accessibility is improved. Furthermore, it is noted that in a new situation, information on a patients’ health status is provided to the hospital unsolicited through sending an admission report by community care. Before the introduction of the e-messaging system, hospital nurses had to call community care nurses to obtain this information. This indicates a positive effect on Timeliness.
However, the interview data also shows negative effects on some of the information quality characteristics. Let us look at the interview excerpt below, taken from an interview with a community care nurse, when talking about the admission report message:
You can attach the note written in our EPR system [Gerica] If it states the reason why a patient is admitted, then you can just use that instead of writing your own. You can write your own, if you want, that is your own choice (...) there, the hospital can see how big the need for help is based on the ADL [Activities of Daily Living]. If it is updated. That is a little challenge in the middle of all this. There was a big focus on that when we started but now it has moved a bit to the back. The ADL is not in focus but is rather important. (SH 12/2 community care nurse)
So, the reason for admission can be taken from the EPR used in the community care setting, but it is unclear whether it has been properly updated. It can also be filled out by the nurse. Two characteristics are in play here. Firstly, can Credibility be questioned, when it is unclear what the source of the ADL is. Secondly, it could be error prone, when taken from the EPR if it has not been updated properly.
In the next interview excerpt, the focus is on the patient health information message and is taken from an interview with a hospital nurse:
Community care likes to have more concise information: what is the background? What are our assessments? What do we think? What are the plans? It is very important that these are included in the patient health information (...) So I think that we and community care can be better at being concise, being more informative in the messages. (...) I see some of the patient health information messages coming from here that are terrible. They only state the planned discharge date and that is not very informative for community care. (AP 04, hospital nurse) In this excerpt, it is clear that Conciseness and Completeness are at stake. Message content – at least in the case of the patient health information messages – happens to be little to the point and/or incomplete.
The last interview excerpt is taken from an interview with two community nurses, discussing discharge messages received from the hospital:
Nurse A: I have seen some discharge reports that were not very good. Some in which almost nothing was stated. Some can be empty, while in others there is very little about how we should follow up [the patients] based on what is done in the hospital.
Nurse B: It doesn't say what they have concluded. It just says what they have done: "He has got liquid and did the examinations". There is no conclusion based on the whole stay. Often. (NH 21/2, community care nurse)
Here, there are three characteristics at stake. Firstly, there is Completeness. As noted, many discharge reports are a little uninformative. Next, there is the issue of Conciseness. As noted by the second nurse, the information provided is not all to the point, while the critical information – a conclusion – is missing. Finally, the characteristic Consistent Representation is in focus here. It is stated that the discharge report is far from standardised, in both form and content.
The table below summarises the findings we presented, where a '+' indicates a positive effect and a '−' a negative effect.
Effect − + − − − −
A more systematic investigation of information quality in the e-messaging system made clear that from a user’s perspective a number of characteristics comprising information quality, can be discussed. Not less than five out of seven of the characteristics are assessed as less than optimal. If information quality is a determining independent variable for perceived information systems’ success, it is difficult to explain the success of the e-messaging system given the problematic state of its information quality.
Firstly, we must note that the assessment of the different information quality characteristics by the informants was a more moderated way to problematise some of the aspects of information quality than a strict categorisation into 'good' and 'bad' quality. Therefore, even though problems in relation to for example Completeness were identified, the data do not warrant the conclusion that information exchanged was useless due to a lack of Completeness. Our study design does not enable us to distinguish between 'good enough to be useful' and 'not good enough to be useful'.
Secondly, the systems’ success could also be partly explained by the simple fact that it replaced a work practice that was so inefficient and ineffective that anything was better than the old way of doing it.
However, our results could also point in a direction previously proposed [5]. The context in which the system is applied leads to a trade-off between the characteristics. In the context of collaboration and coordination of work between hospitals and community care when transferring patients, the characteristics of Accessibility and Timeliness are more important than having optimal Completeness or Conciseness. This might be related to the time constrained nature of this type of work. It is most important to have the information that the patient will be transferred to the hospital or back home as soon as possible, as that drives the planning logistics on both sides of the collaboration. Having available complete and concise information on the patient's status is needed, but not critical to drive the logistics.
For further research, it would be interesting to see if and how these less than optimal characteristics are prioritised among themselves. Is Credibility for example more or less important than Completeness in this context? Such insights could inform the further development of the e-messaging system and help in making decisions about which parts to improve first. Furthermore, it would be interesting to see how these prioritisations change when another context is considered? If the e-messaging system would be used to support the referral process, for example, would we see the same prioritisations or not? Acknowledgements We thank the health professionals involved in the interviews for sharing their experiences with us. This research was funded by the Research Council of Norway, grant number 229623/H10 and is part of the evaluation of the Coordination Reform.