Student gets a low grade on an e-learning course and blames the e-learning system for their failure. Clearly, the user is unsatisfied, but can it be stated that the system is unsuccessful?
This article focuses on e-learning, which according to Sangrà, Vlachopoulo and Cabrera [1] can be described with the following: "E-learning is an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication, and interaction and that facilitates the adoption of new ways of understanding and developing learning." In elearning, technology is used as one enabler of the learning process [2]. E-learning systems aid among other things in presenting content, assessing learner outcomes, promoting collaboration, and facilitating problem solving [2].
In 1992, DeLone and McLean [3] proposed the IS success model that models the dimensions of IS TKTP 2024: Annual Doctoral Symposium of Computer Science, 10.-11.6.2024 Vaasa, Finland laura.havinen@uwasa.fi 0009-0000-3364-7407 success. The model has been widely applied and modified in the field of IS, and it serves as a theoretical background for many studies on the success of elearning systems [4]- [8].
One of the success dimensions in the IS success model is user satisfaction. User satisfaction is a subjective measure and thus prone to bias. Yet many empirical studies on IS success use user satisfaction as a key measure or even as a surrogate of IS success [9]. This can be noted also in the study of e-learning systems success [10].
In this article, the role of user satisfaction as measure for IS success is discussed by considering self-serving bias, a phenomenon that has been identified in the field of social psychology. Self-serving bias means that individuals tend to blame external factors for failures and credit success to themselves [11]. This phenomenon can be seen in practice for example so, that a student blames the teacher when he receives a low grade in an exam [12]. Self-serving bias has been distinguishable also in other contexts than classroom, for example with users interacting with robots [13] and with e-commerce users attributing negative outcomes to computers rather than positive outcomes [14]. In the field of IS, selfserving bias has been pointed out to be one possible explanation for bias in user satisfaction surveys, so that user blames the system for their failure [15]- [17].
The aim of this article is to discuss the possibility of self-serving bias acting as a confounding variable affecting the relationship between net benefits and user satisfaction in the IS success model. This is done in the context of e-learning and from the viewpoint of a student as a user. Given that self-serving bias is a concept that has been constructed for achievement contexts, it is assumed that the theory benefits especially the research in the field of e-learning. As a contribution, this study gives implications for future research on e-learning systems success about how self-serving bias should be considered. This article can be categorized as a critical review, where the aim is to highlight an untrustworthy area of existing knowledge [18]. Furthermore, this article seeks to provide explanation [19] by adding understanding about factors affecting user satisfaction. This article answers the call to analyze the interrelationship between dimensions in the IS success model [20], [9] and to research causal attributions in IS context and for IS artifacts [21].
The article is constructed as follows. First, the theoretical background is described including the DeLone and McLean IS success model and the concept of self-serving bias. Then the results of the critical review on e-learning success are reported. Lastly, the article concludes with the implications for research based on the results.
This article is rooted on two theories which have been validated by several empirical studies and give a solid ground for the study: the IS success model [3] from IS research, and Weiner´s [22] attribution theory from social psychology. Regarding the IS success model, the view is especially on the relationship between user satisfaction and net benefits. In the context of attribution theory, this article focuses on locus of control and especially on the self-serving bias. In the following chapters, both theories are described.
The DeLone and McLean IS Success model is a taxonomy of information systems success that aims to categorize central IS success measures. The framework is widely adopted in IS research. The first version of the model was presented in 1992, and it was revised in 2003 [3], [20].
The original model (see Figure 1) included six interdependent dimensions that are system quality, information quality, use, user satisfaction, individual impact, and organizational impact [3]. System quality refers to the desired characteristics of the information system and measures technical success. Information quality refers to the characteristics of the information product, such as accuracy, and measures semantic success. Use and user satisfaction refer to measuring interaction between the information product and users. Individual impact refers to the influence on management decisions, and organizational impact on organizational performance. These last four dimensions measure effectiveness success. The six dimensions are reliant on one another [3]. System quality and information quality affect use and user satisfaction both singularly and jointly. The amount of use can affect the degree of user satisfaction as well as the other way around. This effect can be either positive or negative. Use and user satisfaction are antecedents for individual impact. Impact on individual performance leads to organizational impact.
In 2003, the model was updated (see Figure 2) [20]. Service quality dimension was added besides the information quality and system quality. Service quality refers to the quality of support for the users from the IS department and IT support. Individual impact and organizational impact were replaced by net benefits. The net benefits is a broader dimension describing more precisely the range of entities that the IS activity can impact, which include e.g. work groups, industries and societies. In addition, the concept of use was clarified by dividing it into intention to use and use. Intention to use describes the attitude and use the behavior, and the division also helps to complete the model in both process and causal sense. Use precedes user satisfaction in process view, but in causal view user satisfaction can also affect use via the intention to use. As DeLone and McLean [20] point out, the model is based on both process and causal considerations. For example, in process view use must precede user satisfaction but in causal view increased user satisfaction will lead to increase in intention to use and therefore increase in use. The arrows show the associations of the dimensions in the process sense. Whether the causality is positive or negative depends on the context. E.g. a high-quality system could lead to higher user satisfaction and positive net benefits. More use of a system with poor quality could lead to lower user satisfaction and negative net benefits.
As a conclusion of the generation of the IS success model, DeLone and McLean [3] claim that IS success is a multidimensional construct that should be evaluated based on systematically combined individual measures from different categories, to make measuring comprehensive. Later, they have also stressed that it is important to measure the interactions between success dimensions to reveal the impact of independent variables [20].
Attribution is a field of study in social psychology that focuses on causal explanations. Attribution-based theories examine the causes individuals' credit for events and outcomes [23], [24]. The major contribution to forming attribution theory was made by Heider [25], and later among others by Kelley [24] and Weiner [22]. Weiner´s [22] attribution theory is directed especially on academic achievement and the causes of success and failure. Weiner focuses on classroom settings, where he claims that people ask "why" questions especially related to achievement contexts. Because Weiner´s attribution theory was crafted especially for academic settings, it is in special interest in this study of e-learning.
Weiner´s attribution theory divides causalities in three-dimensions: locus, stability, and controllability. In this study, the focus is especially on the locus of causality. Locus refers to whether an individual interprets cause to be internal or external. For example, if a student gets good feedback, they might credit it to internal cause, such as high ability ("I succeeded because I am clever") or external cause, such as help from others ("I succeeded because others helped me").
People tend to credit positive outcomes with internal causes and blame external causes for negative outcomes (see e.g. [26]- [29]). This phenomenon is referred to as self-serving bias [11].
There are several possible explanations for this phenomenon. Reasons for self-serving bias can be to consciously protect desirable self-view or to influence others´ perceptions, but often it happens unintentionally and even unconsciously [29]. People tend to seek causes for outcomes that differ from their positive expectations ("I got a low grade although I thought I would succeed on this course") or from their self-schema ("I got a low grade although I am a good student"), or which are inconsistent with actions ("I got a low grade although I have studied hard") [29].
Individuals make self-serving attributions in situations where the outcome is important for them and thus has implications for self-worth [29]. According to Campbell and Sedikides [26], selfserving bias happens in a situation where self-threat, that is threat to the self-concept, is high. These kinds of situations are especially those where an individual acts as an actor, views the task at hand important, conducts tasks that are skills-oriented, and acts in a competitive setting.
Especially in the context of user-satisfaction surveys, Hufnagel and Conca [17] point out the following factors to be associated with self-serving bias: performance outcomes, prior expectation, expertise, experience, perceived responsibility for outcomes, and extent of volitional control. It is important to note that the extent and presence of selfserving bias varies across conditions. In addition to contextual differences, culture and age also have an effect [28].
In the context of exam performance, Arkin and Maruyama [30] found the self-serving bias causing students to attribute external factors for failing in a test more likely in their own case than they assume external factors to have affected the average student. Also, Noel, Forsyth and Kelley [12] found that students who performed poorly on a course examination blamed external factors such as teacher, ambiguity of the textbook and unfairness of items on the test. Gotlieb [31] applied attribution theory in the study of student evaluations and found that grades may affect student evaluations of professors (i.e. if the student gets a high grade, they will give more positive evaluation for the teaching and vice versa).
Attribution theory and its implications have been applied in IS research, both in general and in the context of IS success, although not to a great extent. Kelley et al. [21] state that "attribution theory makes contributions to explaining and understanding IS phenomena". They see especially that the theory on causal attributions benefit the research on postadoption usage. Alony, Hasan and Paris [32] used attribution theory to analyze how well biases in attribution predict non-interpersonal relationship and saw also potential in post-adoption research. In the field of HCI, Niels and Janneck [33] appoint attribution theory as an applicable background theory to understand users and their behavior. They have applied attribution theory to generate personas on how individuals attribute computer-related failure and success.
The self-serving bias in connection to the IS success model has been brought up especially related to the worry about possible confounding variables on the dimension of user satisfaction. Especially under criticism has been the tendency to use user satisfaction as a measure for IS success, which means replacing or removing net benefits. Some studies confirm the possibility to use user satisfaction as a measure for system effectiveness [34], while others don´t. This is understandable, since user satisfaction is a complex variable, and there is no clear view of all the user and environment characteristics that affect it (see e.g. [35]).
Already in the original paper about IS Success model, DeLone and McLean [3] bring up the wide use of user satisfaction/user information satisfaction as a measure for IS success and state the worry that "usersatisfaction measures might be biased by user computer attitudes". User attitude towards IS has been found to have an impact on user satisfaction e.g. in the meta-analysis on IS success model by Sabherwal, Jeyaraj and Chowa [36].
Self-serving bias as a confounding variable on user satisfaction has been noted in several studies. Snead et al. [37] claim that causal attributions can act as an intervening variable between the independent variable and dependent variable in research related to IS success. Mathieson [16] argues that for user satisfaction instruments to accurately reflect IS success, users must have accurate beliefs about the IS. As an example of possible bias, Mathieson [16] refers to user attributing the cause of failure to system instead of himself. Iivari & Ervasti [38] acknowledge the possibility of self-serving bias as confounding factor between user satisfaction and IS effectiveness but considered the effect to be able to be eliminated. Hufnagel [15] claims that evaluating system effectiveness based on user satisfaction ratings may be biased if it measures more individual's attribution to their own performance outcome, and Hufnagel and Conca [17] point out self-serving bias as a possible bias in user satisfaction surveys and call for researchers to recognize potential sources of bias.
The IS success model focuses on the dependent variable on the field of IS, that means the dimensions which describe what is IS success. The independent variables are those variables that influence IS success, such as user characteristics [39]. In addition, there are also control variables and variables that have a moderating effect instead of direct effect on IS success [39]. This study claims that self-serving bias should be considered as a confounding variable in the model, which is as a variable that influences the relationship between other variables. This study focuses on the dependent variables net benefits and user satisfaction. As stated before, net benefits refer to the success of outcome stage, and the measures vary depending on the case. Net benefits describe the extent of contribution IS gives to the stakeholders (individuals, groups, organizations, industries, and nations) [39]. In the context of elearning systems the stakeholders can be customers (e.g. students), suppliers (e.g. teachers, educational institutions, content providers), board and shareholders (e.g. education ministry), professional associations (e.g. teachers´ association) and other special interest groups (e.g. students´ commissions) [2]. This study focuses on the students as stakeholders.
What the net benefits are is left for the researcher to define in the given context and related to the given stakeholders [20]. There is a large variety of methods measuring and thus defining the net benefits [9]. Net benefits can be e.g. improved decision-making, improved productivity, increased sales, or improved profits [9].
From the student perspective in the e-learning context the net benefits are on the individual level. Most used net benefits on individual level are perceived usefulness or job impact [9]. In the context of e-learning systems perceived usefulness can be seen for example as experienced usefulness in studies [6], and job impact can be seen for example as improved performance, more effective learning, or cost and time savings [5]. In the context of learning, net benefits can also be thought to be related to gaining knowledge, attaining learning outcomes and improving student competence (e.g. [40], [5]).
This study focuses on those kinds of net benefits that imply to the user whether they have succeeded or failed. From the student's point of view, these kinds of net benefits would be e.g. those that are communicated via course grade and/or teacher feedback, such as the forementioned gaining knowledge, attaining learning outcomes and improving student competence.
According to Petter. DeLone and McLean [9], studies have shown strong support for the relationship between user satisfaction and net benefits, and between net benefits and user satisfaction on individual level. The connection between net benefits and user satisfaction is valid both when net benefits are positive and when they are negative. The nature of the interaction depends on the case [20]. The connection between net benefits and user satisfaction is of primary interest in this study.
In the following, a qualitative literature review is conducted to illustrate the role of user satisfaction in studies related to e-learning system success. The literature review focuses on studies on e-learning system success that discuss the dimension of user satisfaction and use as a theoretical framework the DeLone and McLean IS success model.
The search was conducted in the journal Computers & Education which is a high-level journal that focuses especially on e-learning. The used search term was (elearning OR e-learning) AND success AND "user satisfaction" and the search was performed in full-text articles for the period of 1.1.2015-31.12.2023. The search resulted in 26 articles. These articles were reviewed according to titles and abstracts to ensure that they discuss e-learning system success from the student viewpoint, which eliminated 13 articles. After that, the remaining 13 articles were reviewed by reading the full text to ensure that the IS success model had a significant role as a background theory in the studies. This led to the rejection of 10 articles, leaving finally three articles that were selected for the qualitative review. In the following, the content of the articles is shortly described and after that, the findings are discussed.
Isaac et al. [41] study task-technology fit (TTF) and compatibility as mediating variables for IS success in online learning. The conceptual model of their study combines compatibility and TFF with the IS success model and has as the IS success indicator performance impact. Performance indicator is described as "the extent to which online learning influences student performance based on productivity, knowledge acquisition, and resource savings". The performance impact is affected by user satisfaction, actual usage and TFF. Moreover, user satisfaction and actual usage also affect performance impact indirectly through TFF. Research data was collected through a survey, and it consisted of 448 responses from university students. In the results of the study, user satisfaction was seen as the second most significant affecting factor to academic performance after task technology fit. Also, user satisfaction was seen to have a meaningful impact on TTF together with actual usage.
Cidral et al. [42] focus in their study on finding the determinants for e-learning systems success. They applied IS success model together with e-learning satisfaction theory. The aim of the study is to find determinants of user perceived satisfaction, use and individual impact in the context of e-learning. They suggest a model that modifies the IS success model based on the theory of e-learning satisfaction. In this model, the main success factor is individual impact, which is defined as "the degree of benefit perceived by students when using an e-learning system". Individual impact is affected by use and user perceived satisfaction. Furthermore, use is affected by user perceived satisfaction. The data was gathered through a survey and consisted of 301 responses from students in higher education institutions. The user perceived satisfaction was found to be a significant factor affecting the individual impact, as the following citation highlights: "The significant impact of user perceived satisfaction on individual impacts supports the suggestion that user perceived satisfaction can serve as a valid substitute for individual impact".
Ung, Labadin and Mohamad [43] study the feasibility of a localized e-learning system myCTGWBL aimed at training computational thinking skills to teachers. The research data was gathered through pre-experiment and post-experiment surveys, of which in the latter they inquired the respondents' perceptions towards the e-learning system. The post-experiment survey was answered by 369 teachers after 14 days of self-learning using the system. The IS success model was used as a framework for determining the system success. The study implies that the system's success is determined by net benefits but does not clearly specify what those net benefits are. Instead, the study refers to general theory about net benefits being improved task performance and productivity. Regarding IS success the article mainly discusses the meaning of user satisfaction and user intention. User satisfaction is seen having a central role affecting the IS success as can be seen from the following citations "Moreover, the proposed myCTGWBL success model indicates that user intention and user satisfaction are closely related to system success." and "Additionally, one of the most vital features influencing the performance of myCTGWBL is user satisfaction" [43].
In all the three reviewed studies, user satisfaction was considered significantly influencing net benefits. This was most visible in the study by Cidral [42], which suggested that user satisfaction could be applicable to replace the individual impacts as indicator for IS success. These notions are in line with the findings of Petter, Delone and McLean [9], who state that in the field of e-learning systems success user satisfaction is used as a key measure and even solely measure of IS success. As can be seen from the study by Ung, Labadin and Mohamad [43], in the context of e-learning clearly defining the net benefits can be difficult as e.g. performance in the context of learning is not as straightforward to define as in other contexts. Each of the studies used surveys as a data gathering method relying on self-reported data, but only Isaac et al. [41] mentioned the self-reported actions as a possible limitation for the study. Not surprisingly none of the studies mentioned selfserving bias as a possible confounding factor or addressed attribution theory in other ways.
The covered theoretical background gives implications that self-serving bias could emerge in an e-learning context and affect student attitudes related to IS success. Deriving from the discussed literature and from the review on the three studies on e-learning systems success, the following suggestion for research in e-learning systems success can be made. Avoid using user satisfaction as a sole measure for IS success in the context of e-learning from the student viewpoint.
As Petter, DeLone and McLean [9] point out, user satisfaction should not be used as a sole indicator of IS success. It has been stated that the possibility of selfserving bias can affect user satisfaction ratings in general [15]- [17]. This should be especially considered in the context of e-learning, where the elements promoting self-serving bias are strongly present, such as the outcome being important for selfworth [29] and the possible threat to the self-concept [26]. Previous literature gives implications that failing on a course can lead to distortion in the student evaluation of factors such as the teacher, textbook or even IS [12], [14], [31]. As DeLone and McLean [3] state, it is important to use a variety of measures to minimize the effect of confounding variables. User satisfaction is a complex variable [35], as is the selfserving bias [31] and it is difficult to pinpoint exactly what affects what in certain circumstances. The use of control variables can help to reveal possible dependencies.
This study aimed to find implications for research by discussing the phenomenon of self-serving bias in the context of e-learning systems success from a student viewpoint. Based on the qualitative review of three studies on e-learning system success it was noted that user satisfaction has a central role in e-learning system success, and it can even be seen as possible sole measure for IS success, without discussing the possibility of self-serving bias. Taking into consideration the discussed theoretical background related to self-serving bias this was seen problematic, and thus as the main result of the study was given one key implication for future research, that is to avoid using user satisfaction as a sole measure for IS success in this context. This article grasps only the surface of implications that attribution theory could have for IS success, but succeeds in opening new directions for research in IS. This follows on the path of Kelley et al. [21] who see attribution theory being in its "spring" of existence in IS research. In the context of studying IS success, this study increases the understanding of variables affecting IS success in e-learning context, and in general.
The suggestion made in this study is not unique, but it can be found already in the works of DeLone and McLean [3] and Petter, DeLone and McLean [9]. The contribution of this study is to strengthen them in the context of e-learning systems success with the notion of the phenomenon of self-serving bias.
As a recommendation to practitioners, taking the self-serving bias into account can benefit both education providers and IS professionals. Identifying and changing focus of attribution cause from external to internal might lead to improved IS success but also improved performance [12]. Acknowledging the possibility of self-serving bias can be considered in the evaluation practices of e-learning systems, e.g. by collecting system related feedback before assigning the final grades.
This study gives multiple recommendations to researchers. Firstly, it serves as a good background for further empirical study. Finding significant differences between different user groups could serve as a guide for further research and to understand prior results [17]. It could also be considered if selfserving bias has impact on possible mediating variables, such as task-technology fit in the relationship between user satisfaction and net benefits [41].
As guidelines for further research, it should be noted that Weiner´s [22] attribution theory focuses on success and failure. To reveal self-serving bias, it is important to find out if the student experienced a success or a failure. If the course grade is used as an implication for success or failure [31], it is important to monitor if the course grade matches with student expectations [29]. As Campbell and Sedikides [26] point out, self-serving bias is higher in situations with high self-threat, which gives an implication to evaluate also whether the student experiences the course as important for himself.
Also, it should be noted, that if the student chooses to attribute the cause of success or failure to an external factor, which factor does he favor. Instead of the IS it could be for example the teacher, other students, luck, or task difficulty [22]. There has been noted a difference that individuals attribute cause to a controllable external factor rather than uncontrollable external factor. For example, people in severe car crashes tend to attribute causes to other drivers rather than uncontrollable events such as weather or road conditions [44]. From this viewpoint there is a question whether students preferably attribute the cause to controllable event, and whether they experience the e-learning system as a controllable factor. This can vary also depending on the nature of the e-learning course, for example based on the course's self-study degree.
As a critical review, this study may suffer from subjectivity because of the limited selection of reviewed literature [18]. This risk has been minimized by reporting the literature search explicitly and grounding the work on two prevalent theories and on previous research applying those theories.