The Effects of Job and User Characteristics on the Perceived Usefulness and Use Continuance Intention of Generative Artificial Intelligence Chatbots at Work

Abstract

Although generative artificial intelligence (AI) chatbots have recently attracted a lot of attention, the antecedents of their user perceptions as well as their use intention and actual use at work remain poorly understood. In this study, we aim to address this gap from the socio-technical perspective of information systems (IS) research by examining how the perceived usefulness and use continuance intention of generative AI chatbots at work are affected by the job and personal characteristics of their users. The examination is based on a sample of 338 current or prior users of generative AI chatbots at work that was collected via an online survey in the summer of 2023 and is analysed with covariance-based structural equation modelling (CB-SEM). We find the effects of both job and user characteristics on the perceived usefulness and use continuance intention of generative AI chatbots at work to be relatively weak, whereas perceived usefulness is found to act as a strong antecedent of use continuance intention. Finally, we discuss the contributions of the study from both theoretical and practical perspectives.

Keywords

• eol>Generative artificial intelligence chatbots
• job characteristics
• user characteristics
• perceived usefulness
• use continuance intention
• future of work1

1. Introduction

Artificial intelligence (AI) is a technology with the potential to bring about substantial changes in our society, and it can be considered to have immense implications also for the future of work (e.g., [ 1–3 ]). Thus, it is not surprising that AI has been studied more and more (e.g., [ 4–7 ]) also from the socio-technical perspective of information systems (IS) research, which focuses on the interactions between the social and technical components of various socio-technical systems [ 8 ]. One particular type of AI that has recently attracted a lot of attention, both in academia and in the mainstream media, is generative AI, which refers to AI that can be used to generate text, images, video, audio, code, or practically any other type of content as a response to a prompt provided by the user [ 9 ]. This attention has been driven by the launch of several novel generative AI chatbots based on large language models (LLMs), such as OpenAI’s ChatGPT (launched on 30 November 2022 and based on OpenAI’s proprietary generative pre-trained transformer [GPT] models GPT3.5 and GPT-4), Microsoft’s Bing Chat (launched in February 2023 and also based on OpenAI’s GPT-4 model), and Google’s Bard (launched in March 2023 and based on Google’s proprietary Language Model for Dialogue Applications [LaMDA] model) [ 10 ]. These novel generative AI chatbots have proven not only to clearly exceed the performance of their predecessors but also to be very versatile – true “jacks of all trades” [ 11 ]. For example, they are able to conduct conversations that are practically indistinguishable from conversations between two humans, answer almost any questions, tell (at least half-decent) jokes, write essays, poems, lyrics, and other types of literary works, translate, edit, and summarise text, and even act as programmers by writing or debugging code in various programming languages.

Because of these impressive abilities, generative AI chatbots have attracted much attention also in terms of the future of work, and several prior studies have already highlighted their transformative potential for many industries and professions, such as education (e.g., [ 9, 10, 12–14 ]), health care (e.g., [ 15–19 ]), hospitality and tourism (e.g., [ 20–24 ]), knowledge work (e.g., [ 25 ]), and organisational management (e.g., [ 26 ]). However, in contrast to the macro-level perspective adopted in most prior studies, few prior studies have so far adopted a more micro-level perspective and examined, for example, what kinds of factors act as the antecedents of the various user perceptions of generative AI chatbots (e.g., their perceived usefulness) or their use intention and actual use at work. For example, little is known about how useful generative AI chatbots are actually perceived to be by people who use them at work or how motivated these people are to continue using them in their jobs in the future. Similarly, even less is known about how these perceptions or motivations potentially differ between people with different job and personal characteristics. In this study, we aim to address these research gaps from the socio-technical perspective of IS research by examining how the perceived usefulness and use continuance intention of generative AI chatbots at work (i.e., the perceptions and conations concerning the technical component of a socio-technical system) are affected by the job and personal characteristics of their users (i.e., the task-related and individual-related aspects of the social component of a socio-technical system). The examination is based on a sample of 338 current or prior users of generative AI chatbots at work that was collected via an online survey in the summer of 2023 and is analysed with covariance-based structural equation modelling (CB-SEM). As a contribution, we advance the theoretical understanding of the antecedents of the user perceptions and use continuance of generative AI chatbots at work as well as provide several important implications for practice.

After this introductory section, we present the research model and research hypotheses of the study in Section 2. The research methodology and research results of the study are reported in Sections 3 and 4, and the research results are discussed in more detail in Section 5. Finally, we conclude the paper with a brief discussion of the limitations of the study and some potential paths for future research in Section 6.

2. Research model and research hypotheses

The theoretical foundation of our research model is based on Task–Technology Fit (TTF) theory [ 27 ], in which the degree to which a technology is able to assist an individual in performing his or her portfolio of tasks (or a job), and consequently the perceived usefulness and use continuance intention of that particular technology (e.g., [ 28, 29 ]), are hypothesised to be determined by the interaction of three antecedents: (1) task characteristics, (2) technology characteristics, and (3) individual characteristics. That is, in order for a particular technology to perform optimally and to be perceived as useful and motivating to be used also in the future, it has to match both the characteristics of the job in which it is being used and the characteristics of the individual who is using it. Thus, we hypothesise that also the perceived usefulness and use continuance intention of generative AI chatbots at work are affected by two main groups of antecedents: job characteristics and user characteristics. This hypothesis can be seen to fit well the socio-technical perspective of IS research (e.g., [ 8 ]) in terms of focusing on how the perceptions and conations concerning the technical component of a socio-technical system (i.e., perceived usefulness and use continuance intention) are affected by the task-related and individual-related aspects of the social component of that same system (i.e., job and user characteristics).

Of course, in terms of the outcome constructs in our research model, we could have also focused on the effects of job and user characteristics only on perceived usefulness and hypothesised that their effects on use continuance intention are fully mediated by it because perceived usefulness has been found to act as one of the main antecedents of use intention and use continuance intention in prior research (e.g., [ 29–32 ]). However, instead of focusing only on such indirect effects of job and user characteristics on use continuance intention via perceived usefulness, we see it as important to focus also on the direct effects of job and user characteristics on use continuance intention because not all the effects are necessarily fully mediated by perceived usefulness. For example, while users with specific job and personal characteristics may not perceive generative AI chatbots as particularly useful, they may still have a strong intention to continue using them at work for some other reasons, such as being able to match the work performance of other people in the same job who are using generative AI chatbots at work. In the following three subsections, we discuss the research constructs and the research hypotheses of our research model in more detail, beginning with the aforementioned job and user characteristics.

2.1. Research hypotheses on job characteristics

In terms of job characteristics, we focus on four characteristics that are specified in the Work Design Questionnaire (WDQ) by Morgeson and Humphrey [33] and can be seen as especially relevant for the use of generative AI chatbots at work: (1) job creativity requirement, (2) job task variety, (3) job specialisation, and (4) job social interaction. First, job creativity requirement or job problem-solving requirement (of which we use the former term in this paper) refers to the degree to which individuals perceive that creativity or generating unique or innovative ideas or solutions is required to perform their job effectively [33–35]. In turn, job task variety refers to the degree to which a job involves performing a wide range of tasks, whereas job specialisation refers to the degree to which a job involves performing specialised tasks or possessing specialised knowledge and skills [33]. In a sense, these two characteristics can be seen as opposites of each other because whereas the former characteristic focuses more on the width of knowledge and skills required in the job, the latter characteristic focuses more on the depth of knowledge and skills required in the job. Finally, job social interaction is based on the interaction outside the organisation characteristic of the WDQ, which we extend here to cover social interaction not only outside but also within the organisation, thus reflecting the degree to which the job involves interacting and communicating with individuals either external or internal to the organisation.

Of the job characteristics, we hypothesise both job creativity requirement and job task variety to have positive effects on the perceived usefulness and use continuance of intention of generative AI chatbots at work. On one hand, this is based on the ability of generative AI chatbots to mimic human creativity [ 25 ], which can be assumed to promote their perceived usefulness and use continuance intention particularly in knowledge-intensive jobs with high creativity requirement [ 25 ]. On the other hand, it is based on the versatility of generative AI chatbots [ 11 ], which can be assumed to promote their perceived usefulness and use continuance intention particularly in jobs with high task variety [36, 37]. In contrast, we hypothesise both job specialisation and job social interaction to have negative effects on the perceived usefulness and use continuance of intention of generative AI chatbots at work. On one hand, this is based on the assumption that the more specialised one’s job-related tasks are, the less chance there is for generative AI chatbots or any other general-purpose technologies to assist one in these tasks, thus impeding particularly the routinisation of their use [38]. On the other hand, it is based on the assumption that if one’s job-related tasks consist mostly of interacting with other people, there is less chance for one to interact with technologies like generative AI chatbots and use them to assist one in these tasks. We summarise the eight hypotheses concerning the effects of job characteristics on the perceived usefulness and use continuance intention of generative AI chatbots at work as follows: H1a: Job creativity requirement positively affects the perceived usefulness of generative AI chatbots at work.

H1b: Job creativity requirement positively affects the use continuance intention of generative AI chatbots at work.

H2a: Job task variety positively affects the perceived usefulness of generative AI chatbots at work.

H2b: Job task variety positively affects the use continuance intention of generative AI chatbots at work. H3a: Job specialisation negatively affects the perceived usefulness of generative AI chatbots at work.

H3b: Job specialisation negatively affects the use continuance intention of generative AI chatbots at work.

H4a: Job social interaction negatively affects the perceived usefulness of generative AI chatbots at work.

H4b: Job social interaction negatively affects the use continuance intention of generative AI chatbots at work.

2.2. Research hypotheses on user characteristics

In terms of user characteristics, we focus on four characteristics that have been commonly hypothesised and found to affect technology acceptance and use in prior research (e.g., [ 31, 32 ]): (1) gender, (2) age, (3) education, and (4) job experience (in the current job). These characteristics have been found to affect, for example, the technology readiness of individuals, with men, younger individuals, and more highly educated individuals having higher technology readiness and women, older individuals, and less highly educated individuals having lower technology readiness [39, 40]. In addition, in recent meta-analyses (e.g., [41]), men have also been found to have a more positive attitude toward technology compared with women. In turn, higher technology readiness and a more positive attitude toward technology in general can be assumed to result in more positive perceptions of a particular technology, thereby also supporting its use continuance intention. Thus, we hypothesise that men, younger individuals, and more highly educated individuals perceive generative AI chatbots as more useful and have a stronger intention to continue using them compared with women, older individuals, and less highly educated individuals. In addition, we hypothesise that individuals who have more experience in their current job perceive generative AI chatbots as more useful and have a stronger intention to continue using them at work compared with individuals who have less experience in their current job. This is based on the assumption that more job experience results in a better understanding of the job-related tasks and, thus, also of how generative AI chatbots may be used to assist one in these tasks. We summarise the eight hypotheses concerning the effects of user characteristics on the perceived usefulness and use continuance intention of generative AI chatbots at work as follows: H5a: Gender affects the perceived usefulness of generative AI chatbots at work, with men perceiving them as more useful compared with women.

H5b: Gender affects the use continuance intention of generative AI chatbots at work, with men having a stronger intention to continue using them compared with women. H6a: Age negatively affects the perceived usefulness of generative AI chatbots at work. H6b: Age negatively affects the use continuance intention of generative AI chatbots at work. H7a: Education positively affects the perceived usefulness of generative AI chatbots at work. H7b: Education positively affects the use continuance intention of generative AI chatbots at work. H8a: Job experience positively affects the perceived usefulness of generative AI chatbots at work.

H8b: Job experience positively affects the use continuance intention of generative AI chatbots at work.

2.3. Other research hypotheses and a summary of the research model

In addition to the four job characteristics and four user characteristics mentioned above, we also add one control variable concerning the non-work use of generative AI chatbots to our research model. This control variable can be seen as relevant in that individuals who use generative AI chatbots in non-work contexts in addition to the work context have likely gained more experience in using the technology, which can be assumed to affect their perceptions of its usefulness and their intention to continue using it. We hypothesise that the effect of this use experience on the perceived usefulness and use continuance intention of generative AI chatbots at work is positive rather than negative because it can be assumed to provide the users with a better understanding of the technology and of how it may be used to assist one not only in non-work contexts but also in the work context. These two hypotheses are summarised as follows:

H9a: Non-work use of generative AI chatbots affects their perceived usefulness at work, with those who use them in non-work contexts in addition to the work context perceiving them as more useful.

H9b: Non-work use of generative AI chatbots affects their use continuance intention at work, with those who use them in non-work contexts in addition to the work context having a stronger intention to continue using them.

Finally, in our research model, we also hypothesise a positive effect of perceived usefulness on use continuance intention. This is based on the role of perceived usefulness as one of the main antecedents of use intention and actual use in numerous IS theories, such as the Technology Acceptance Model (TAM) [ 30 ] and the Unified Theory of the Acceptance and Use of Technology (UTAUT) [ 31, 32 ]. This final hypothesis is summarised as follows:

H10: Perceived usefulness of generative AI chatbots at work positively affects their use continuance intention at work.

The resulting research model is summarised in Figure 1, illustrating the hypothesised effects of the four job characteristics, four user characteristics, and one control variable on the perceived usefulness and use continuance intention of generative AI chatbots at work as well as the effect of perceived usefulness on use continuance intention.

3. Methodology

The data for testing the research hypotheses in our research model was collected in the summer of 2023 via an online survey that was conducted by using the LimeSurvey service. The respondents of the survey were recruited by sending an invitation to 1,207 respondents of our previous online survey that was conducted about a year earlier in the spring of 2022. In this previous survey, these invited respondents had indicated that they had used some kind of a robot or intelligent system (more specifically a physical robot, software robot, chatbot, or virtual assistant) at work, which is why we considered them ideal informants for the present survey in terms of likely having higher than average adoption rates for using generative AI chatbots at work.

The respondents of this previous survey were originally recruited by using an online crowdsourcing service, which have been deemed a reliable and valid method of collecting data also in IS research [42]. More specifically, we used the Prolific service, which has been found to provide better or at least equal data quality and a more heterogeneous population of participants than its alternatives, such as the Amazon Mechanical Turk (MTurk) service [43, 44]. This same service was also used for sending the invitations to participate in the present survey. In the previous survey, because we were mainly interested in the use of robots at work, we recruited only respondents who were employed either full-time (≥ 30 h / week) or part-time (< 30 h / week) and who resided in the UK, the US, or Canada, which are all countries that have been found to have high usage rates of robots at work [45] and can also be considered to constitute a homogeneous Anglospheric cultural domain. In order to promote data quality, we followed both the more general [46] and the more IS-specific [47] guidelines for using online crowdsourcing services for research. For example, we recruited only respondents who had a minimum approval rate of 98% for their submitted tasks or studies as well as a minimum of 20 submissions and a maximum of 10,000 submissions. All the respondents were paid a monetary reward for their participation in both the previous survey and the present survey that exceeded the minimum hourly reward recommended by the Prolific service.

In the survey, use continuance intention (UCI), perceived usefulness (PU), job creativity requirement (JCR), job task variety (JTV), job specialisation (JS), and job social interaction (JSI) were each measured reflectively by using multiple items. The wordings of these items are reported in Appendix A. Use continuance intention was measured with three items that were adapted from [ 31, 32 ], whereas perceived usefulness was measured with four items that were adapted from [ 30–32 ]. In turn, job creativity requirement was measured with four items that were adapted from [35, 48], whereas job task variety, job specialisation, and job social interaction were each measured with four items that were adapted from [33]. The measurement scale of all these items was the traditional five-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree). In turn, gender, age, education, job experience, and the non-work use of generative AI chatbots were each measured by using one item only. Gender was measured with a binary scale (0 = man and 1 = woman), age with a continuous scale (age in years), education with a three-point categorical scale (1 = no degree, diploma, or certificate, 2 = undergraduate degree, diploma, or certificate, and 3 = graduate or postgraduate degree, diploma, or certificate), job experience with a six-point categorical scale (1 = less than a year, 2 = 1–2 years, 3 = 3–5 years, 4 = 6–10 years, 5 = 11–20 years, and 6 = more than 20 years), and the non-work use of generative AI chatbots with a binary scale (0 = has not used and 1 = has used). In order to avoid forced responses, responding to all the aforementioned items was voluntary, and not responding to a particular item resulted in a missing value.

The collected data was analysed with covariance-based structural equation modelling (CBSEM) by using the Mplus version 8.8 software [49] and following the guidelines for SEM in administrative and social science research [50]. Because the items that were measured on the Likert scale were all treated as continuous variables and some of the items had non-normally distributed data, model estimation was conducted by using robust maximum likelihood (MLR) estimation. In turn, the potential missing values in the items were handled by using full information maximum likelihood (FIML) estimation, which uses all the available data in model estimation.

4. Results

In total, we received a valid response from 838 out of the 1,207 invited respondents, resulting in a response rate of 69.4%. Of these 838 respondents, 338 (40.3%) reported having used generative AI chatbots at work, and these respondents were used as the sample of this study. The descriptive statistics of this sample in terms of the gender, age, education, and country of residence of the respondents as well as their experience in their current job are reported in Table 1. In addition, the current industry of the respondents is reported in Appendix B. As can be seen, the sample was quite evenly balanced in terms of gender and age, with the age of the respondents ranging from 19 to 70 years and having a mean of 37.2 years and a standard deviation of 10.2 years. Most of the respondents (83.1%) had attained an undergraduate, graduate, or postgraduate degree, diploma, or certificate, and most (92.9%) resided either in the UK or in the US. Almost half of the respondents (45.2%) also had more than five years of experience in their current job. In addition, Table 2 reports descriptive statistics about the use of generative AI chatbots among the respondents in terms of the generative AI chatbots that they have used at work, their use frequency of these generative AI chatbots at work, and whether they have used generative AI chatbots in non-work contexts in addition to the work context. Unsurprisingly, the most used generative AI chatbot at work was Open AI’s ChatGPT, which had been used by 83.1% of the respondents. It was followed by Microsoft’s Bing Chat at 27.8%, Google’s Bard at 14.8%, and other generative AI chatbots at 4.7%. These generative AI chatbots were used very frequently by the respondents at work, as more than six out of ten respondents (60.6%) used them at least weekly. In contrast, about one out of four respondents (27.6%) used generative AI chatbots at work relatively infrequently, as they used them less frequently than monthly, had only tried or trialled them a few times, or had quit using them. Most of the respondents (85.2%) had also used generative AI chatbots not only in the work context but also in non-work contexts. In the following three subsections, we first evaluate the estimated model in terms of the reliability and validity of its constructs and indicators as well as its goodness-of-fit with the data. Finally, we report the model estimates.

4.1. Construct reliability and validity

Construct reliability was evaluated from the perspective of internal consistency by using the composite reliability (CR) of the constructs [51], which is commonly expected to be at least 0.7 [52]. The CR of each construct is reported in the first column of Table 3, showing that all the constructs met this criterion. In turn, construct validity was evaluated from the perspectives of convergent and discriminant validity by using the two criteria based on the average variance extracted (AVE) of the constructs [51], which is the average proportion of variance that a construct explains in its indicators. The first criterion concerning convergent validity expects each construct to have an AVE of at least 0.5. This means that, on average, each construct should explain at least half of the variance in its indicators. The AVE of each construct is reported in the second column of Table 3, showing that all the constructs met this criterion. In turn, the second criterion concerning discriminant validity expects each construct to have a square root of AVE that is at least equal to its absolute correlations with the other constructs in the model. This means that, on average, each construct should share at least an equal proportion of variance with its indicators compared with what it shares with the other constructs. The square root of AVE of each construct (on-diagonal) and the correlations between all the constructs in the model (off-diagonal) are reported in the remaining columns of Table 3, and they show that also this criterion was met by all the constructs.

4.2. Indicator reliability and validity

Indicator reliability and validity were evaluated by using the standardised loadings of the indicators, which are reported in Table 4 together with the means and standard deviations (SD) of the indicator scores as well as the percentages of missing values. In the typical case of each indicator loading on only one construct, the standardised loading of each indicator is commonly expected to be statistically significant and at least 0.707 [51]. This is equivalent to the standardised residual of each indicator being at least 0.5, meaning that at least half of the variance in each indicator is explained by the construct on which it loads. This criterion was met by all the indicators except JS2. However, because its slightly lower loading was not found to compromise the reliability or validity of the job specialisation construct (cf. Section 4.1), we decided to retain it in the model.

Missing 0.6% 0.9% 0.9% 0.6% 0.9% 1.2% 0.6% 0.3% 0.0% 0.0% 0.3% 0.0% 0.3% 0.0% 0.0% 0.0% 0.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

Loading 0.956*** 0.965*** 0.950*** 0.909*** 0.917*** 0.926*** 0.915*** 0.921*** 0.838*** 0.909*** 0.846*** 0.801*** 0.820*** 0.808*** 0.862*** 0.759*** 0.636*** 0.837*** 0.781*** 0.854*** 0.919*** 0.843*** 0.927***

4.3. Model fit and model estimates

The results of model estimation in terms of the standardised effect sizes and their statistical significance and the proportions of explained variance (R2) in the perceived usefulness and use continuance intention constructs are reported in Table 5. Model fit was evaluated by using both the χ2 test of model fit and the four model fit indices recommended in recent methodological guidelines [ 50, 53 ]: the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardised root mean square residual (SRMR). Of these, the χ2 test of model fit rejected the null hypothesis of the model fitting the data (χ2(300) = 500.339, p < 0.001), but this can be considered common in the case of large samples [ 54 ]. In contrast, the values of the four model fit indices (CFI = 0.966, TLI = 0.959, RMSEA = 0.044, and SRMR = 0.037) all met the cut-off criteria recommended in recent methodological guidelines [ 53 ]: CFI ≥ 0.95, TLI ≥ 0.95, RMSEA ≤ 0.06, and SRMR ≤ 0.08. Thus, we consider the overall fit of the model to be acceptable. We also found no serious signs of multicollinearity or common method bias in the model in terms of its latent constructs (i.e., the constructs that were measured by using multiple indicators). For example, the variance inflation factor (VIF) statistics calculated by using the factor scores of the latent constructs were all less than three [ 55 ], and the Harman’s single factor test [ 56 ] that was conducted for the latent constructs suggested a very bad fit with the data (χ2(230) = 4,317.360, p < 0.001, CFI = 0.236, TLI = 0.159, RMSEA = 0.229, and SRMR = 0.227). As shown in Table 5, three constructs were found to have statistically significant effects on the perceived usefulness of generative AI chatbots at work. First, those who had higher job task variety were found to perceive generative AI chatbots as more useful at work. Second, men were found to perceive generative AI chatbots as more useful at work compared with women. Finally, third, those who had used generative AI chatbots not only in the work context but also in other contexts were found to perceive them as more useful at work. In turn, four constructs were found to have statistically significant effects on the use continuance intention of generative AI chatbots at work. First, as expected based on theories like TAM [ 30 ] and UTAUT [ 31, 32 ], those who perceived generative AI chatbots as more useful at work were also found to have a stronger intention to continue using them at work. This was by far the strongest effect of all the estimated effects in the model. Second, surprisingly and contrary to our original hypothesis, older respondents were found to have a stronger intention to continue using generative AI chatbots at work. Third, those who had higher education were found to have a stronger intention to continue using generative AI chatbots at work. Finally, fourth, those who had more experience in their current job were found to have a stronger intention to continue using generative AI chatbots at work. In total, the estimated model was able to explain 10.8% of the variance in the perceived usefulness and 80.9% of the variance in the use continuance intention of generative AI chatbots at work.

5. Discussion and conclusion

In this study, which adopts the socio-technical perspective of IS research, we examined how the perceived usefulness and use continuance intention of generative AI chatbots at work (i.e., the perceptions and conations concerning the technical component of a socio-technical system) are affected by the job and personal characteristics of their users (i.e., the task-related and individualrelated aspects of the social component of a socio-technical system). This was done by using data from 338 current or prior users of generative AI chatbots at work to test our research model, which comprised a total of 19 research hypotheses concerning the effects of four job characteristics, four user characteristics, and one control variable on the perceived usefulness and use continuance intention of generative AI chatbots at work as well as the effect of perceived usefulness on use continuance intention. The results of this hypothesis testing are summarised in Table 6, showing that we found support for a total of six research hypotheses in our research model. In summary, we made four main findings. First, we found that the four job characteristics in our research model (i.e., job creativity requirement, job task variety, job specialisation, and job social interaction) had relatively weak effects on the perceived usefulness and use continuance intention of generative AI chatbots at work. More precisely, we found that none of these job characteristics had a statistically significant effect on the use continuance intention of generative AI chatbots at work, and only job task variety had a statistically significant effect on the perceived usefulness of generative AI chatbots at work. As hypothesised, this effect was found to be positive, meaning that those who work in a job that involves performing a wide range of tasks tend to perceive generative AI chatbots as more useful at work. This is most likely explained by the versatile and general-purpose nature of this technology in terms of being able to assist individuals in a variety of job-related tasks ranging from answering simple questions to writing or debugging complex code in various programming languages. However, like the other effects of job characteristics, also this effect was found to be relatively weak. Overall, these weak effects may be explained from both theoretical and methodological perspectives. From a theoretical perspective, one possible explanation is that, although there is great potential in generative AI chatbots for assisting workers in their job-related tasks [ 25, 57 ], workers may not yet be able to fully capitalise on this potential in real life, for example, because of a misfit between the technology and the tasks or a lack of knowledge and skills. Thus, for example, although one might work in a job characterised by high creativity requirement or high task variety, one would still not necessarily perceive generative AI chatbots as particularly useful or have a particularly strong intention to continue using them at work. Another possible explanation that relates especially to job creativity requirement is that, although workers in jobs with high creativity requirement may see more opportunities for the utilisation of generative AI chatbots at work [ 25, 57 ], they may also simultaneously see generative AI chatbots as a threat to their job security [ 57 ], thus causing their perceptions of the usefulness of generative AI chatbots and their intention to continue using them at work to be rather mixed. In turn, from a methodological perspective, one possible explanation may be that our operationalisations of the four job characteristics as well as perceived usefulness and use continuance intention were too general, and more specific operationalisations would have been needed to capture the hypothesised effects between the constructs. For example, instead of assessing these constructs only in terms of one’s current job, there might have been the need to assess them in terms of particular job-related tasks. Another possible explanation may relate to the high means and low standard deviations of the scores of the indicators that were used to measure the job characteristic constructs (cf. Table 4). These statistics suggest that most of the respondents in our sample were working in jobs with high job creativity requirement, high job task variety, high job specialisation, and high job social interaction. This may have falsely weakened the strength of the examined effects compared with a more balanced situation where our sample would have contained more respondents working also in jobs with lower job creativity requirement, lower job task variety, lower job specialisation, and lower job social interaction.

Second, we found that also the four user characteristics in our research model (i.e., gender, age, education, and job experience) had relatively weak effects on the perceived usefulness and use continuance intention of generative AI chatbots at work. Here, however, we found more effects that were statistically significant. On one hand, as hypothesised, we found that men perceived generative AI chatbots as more useful at work compared with women, which is in line with the findings of prior studies on the higher technology readiness of men and their more positive attitude toward technology compared with women [39–41]. On the other hand, we found that older individuals, more highly educated individuals, and individuals with more experience in their current job had a stronger intention to continue using generative AI chatbots at work. Of these, the positive effect of education is consistent with our research hypotheses and in line with the findings of prior studies on the higher technology readiness of more highly educated individuals compared with less highly educated individuals [39–40]. Similarly, the positive effect of job experience is consistent with our research hypotheses and most likely explained by the fact that those who have more experience in their current job also have a better understanding of their job-related tasks and, thus, of how generative AI chatbots may be used to assist them in these tasks. In contrast, the positive effect of age conflicts with our research hypotheses, which originally proposed that this effect would be negative. It also conflicts with the findings of prior studies on the higher technology readiness of younger individuals compared with older individuals [39– 40]. Here, however, it is important to note that age was found to have a positive effect only on the use continuance intention of generative AI chatbots at work but no effect on the perceived usefulness of generative AI chatbots at work. This means that, although older individuals tend to have a stronger intention to continue using AI chatbots at work, they do not tend to perceive them as more useful at work compared with younger individuals. Thus, this conflicting finding may be explained by the motivation of older individuals to use this technology to better compete with their younger colleagues, which seems to override the potential differences in technology readiness or the attitude toward technology between individuals of different ages.

Third, as hypothesised, we found that the perceived usefulness of generative AI chatbots at work had a statistically significant and very strong positive effect on their use continuance intention at work. This is in line with IS theories like TAM [ 30 ] and UTAUT [ 31, 32 ] and confirms their applicability also to the context of generative AI chatbots. Fourth, as hypothesised, we also found that our control variable concerning the non-work use of generative AI chatbots had a statistically significant and positive effect on the perceived usefulness of generative AI chatbots at work. This suggests a strong cross-contextual transferability of the use experience of generative AI chatbots from non-work to the work context, in which it is likely to provide users with a better understanding of the technology and how it may be used to assist them in their job-related tasks.

We see that our study makes several theoretical and practical contributions. From a theoretical perspective, to the best of our knowledge, this study is the first one to focus on the antecedents of the user perceptions and use continuance of generative AI chatbots among their current or prior users. Thus, its findings considerably advance the current understanding of these antecedents, especially from the socio-technical perspective of IS research in terms of examining how the perceptions and conations concerning the technical component of a socio-technical system (i.e., perceived usefulness and use continuance intention) are affected by the task-related and individual-related aspects of the social component of the same system (i.e., job and user characteristics). The main theoretical insight in this respect is that both job and user characteristics seem to act as surprisingly weak antecedents of the perceived usefulness and use continuance intention of generative AI chatbots at work. This is interesting because it somewhat challenges the important role that task and individual characteristics have traditionally been proposed to play in IS research by theories like TTF [ 27 ]. Some possible explanations for this from both theoretical and methodological perspectives have already been proposed above. In turn, from a practical perspective, the findings of the study provide several important managerial implications for organisations that are already using or are planning to use generative AI chatbots at work. For example, on one hand, if the weak effects of job characteristics on the perceived usefulness and use continuance intention of generative AI chatbots at work are indeed caused by issues related to a misfit between the technology and the tasks, a lack of knowledge and skills among the workers, or worries about job security, the organisations should address these issues with appropriate managerial actions, such as by thinking about the best practices for using generative AI chatbots for various job-related tasks, developing the knowledge and skill levels of their workers, and communicating more openly about the potential consequences of using generative AI chatbots at work in terms of topics like job displacement and reskilling [ 57 ]. On the other hand, although the findings of the study imply few differences in the acceptance of generative AI chatbots at work in terms of different job and user characteristics, they still seem to be most readily accepted by individuals who work in jobs with high task variety as well as by men, older individuals, more highly educated individuals, individuals with more experience in their current job, and individuals with more experience in the non-work use of generative AI chatbots. Thus, these individuals are most likely to act as the early adopters of generative AI chatbots at work and represent a key segment in terms of promoting their successful adoption in organisations particularly at the early stages of the diffusion process. Of course, at the later stages of the diffusion process, special attention must also be paid to women, younger individuals, less highly educated individuals, individuals with less experience in their current job, and individuals with less experience in the non-work use of generative AI chatbots, who are more likely to act as late adopters of generative AI chatbots at work and may need more support in their adoption decisions.

6. Limitations and future research

In our view, this study has three main limitations. First, of the potential job and user characteristics, our study focused only on the four job and four user characteristics that we considered most relevant for the perceived usefulness and use continuance intention of generative AI chatbots at work. Future studies should focus also on other job and user characteristics that have been proposed in prior literature so that their research models are able to capture all the relevant characteristics of both the users and their jobs that act as antecedents of the perceived usefulness and use continuance intention of generative AI chatbots at work. Second, our study focused only on perceived usefulness as an outcome construct instead of other perceptions of the used technology that have been found to act as antecedents of use intention and actual use in prior research, such as perceived ease of use [ 30–32 ] and perceived enjoyment of use [ 32, 58, 59 ]. Future studies should focus also on these other perceptions as outcome constructs. Third, as already mentioned above, in our study, the operationalisations of the four job characteristics as well as perceived usefulness and use continuance intention focused on assessing the constructs only in terms of one’s current job instead of some more particular job-related tasks. Future research should focus on these kinds of more specific operationalisations instead of the more general operationalisations that were used in this study. In addition to addressing the aforementioned limitations, another potential path for future research could also be to focus on the prospective instead of current or prior users of generative AI chatbots at work.

Acknowledgements

This study was funded by the Academy of Finland (341359), the Emil Aaltonen Foundation, and the Foundation for Economic Education. [33] F. P. Morgeson, S. E. Humphrey, The Work Design Questionnaire (WDQ): Developing and validating a comprehensive measure for assessing job design and the nature of work, Journal of Applied Psychology 91.6 (2006) 1321–1339. doi:10.1037/0021-9010.91.6.1321. [34] C. E. Shalley, L. L. Gilson, T. C. Blum, Matching creativity requirements and the work environment: Effects on satisfaction and intentions to leave, Academy of Management Journal 43.2 (2000) 215–223. doi:10.2307/1556378. [35] L. L. Gilson, C. E. Shalley, A little creativity goes a long way: An examination of teams’ engagement in creative processes, Journal of Management 30.4 (2004) 453–470. doi:10.1016/j.jm. 2003.07.001. [36] R. W. Harris, Teaching, learning and information technology: Attitudes towards computers among Hong Kong’s faculty, Journal of Computing in Higher Education 9.2 (1997) 89–114. doi:10.1007/BF02948780. [37] R. W. Harris, Attitudes towards end-user computing: A structural equation model, Behaviour & Information Technology 18.2 (1999) 109–125. doi:10.1080/014492999119156. [38] L. Lapointe, S. Rivard, A triple take on information system implementation, Organization Science 18.1 (2007) 89–107. doi:10.1287/orsc.1060.0225. [39] J. I. Rojas-Méndez, A. Parasuraman, N. Papadopoulos, Demographics, attitudes, and technology readiness: A cross-cultural analysis and model validation, Marketing Intelligence & Planning 35.1 (2017) 18–39. doi:10.1108/MIP-08-2015-0163. [40] M. Blut, C. Wang, Technology readiness: A meta-analysis of conceptualizations of the construct and its impact on technology usage, Journal of the Academy of Marketing Science 48.4 (2020) 649–669. doi:10.1007/s11747-019-00680-8. [41] Z. Cai, X. Fan, J. Du, Gender and attitudes toward technology use: A meta-analysis, Computers & Education 105 (2017) 1–13. doi:10.1016/j.compedu.2016.11.003. [42] P. B. Lowry, J. D’Arcy, B. Hammer, G. D. Moody, “Cargo Cult” science in traditional organization and information systems survey research: A case for using nontraditional methods of data collection, including Mechanical Turk and online panels, Journal of Strategic Information Systems 25.3 (2016) 232–240. doi:10.1016/j.jsis.2016.06.002. [43] E. Peer, L. Brandimarte, S. Samat, A. Acquisti, Beyond the Turk: Alternative platforms for crowdsourcing behavioral research, Journal of Experimental Social Psychology 70 (2017) 153–163. doi:10.1016/j.jesp.2017.01.006. [44] E. Peer, D. Rothschild, A. Gordon, Z. Evernden, E. Damer, Data quality of platforms and panels for online behavioral research, Behavior Research Methods 54.4 (2022) 1643–1662. doi: 10.3758/s13428-021-01694-3. [45] World Economic Forum, The future of jobs report 2020, 2020. URL: https://www.weforum.org/reports/the-future-of-jobs-report-2020. [46] E. Peer, J. Vosgerau, A. Acquisti, Reputation as a sufficient condition for data quality on Amazon Mechanical Turk, Behavior Research Methods 46.4 (2014) 1023–1031. doi:10.3758/ s13428-013-0434-y. [47] R. Jia, Z. R. Steelman, B. H. Reich, Using Mechanical Turk data in IS research: Risks, rewards, and recommendations, Communications of the Association for Information Systems 41.14 (2017) 301–318. doi:10.17705/1CAIS.04114. [48] T.-Y. Kim, A. H. Y. Hon, D.-R. Lee, Proactive personality and employee creativity: The effects of job creativity requirement and supervisor support for creativity, Creativity Research Journal 22.1 (2010) 37–45. doi:10.1080/10400410903579536. [49] L. K. Muthén, B. O. Muthén, Mplus home page, 2023. URL: https://www.statmodel.com. [50] D. Gefen, E. E. Rigdon, D. Straub, Editor’s comments: An update and extension to SEM guidelines for administrative and social science research, MIS Quarterly 35.2 (2011) iii–xiv. doi: 10.2307/23044042. [51] C. Fornell, D. F. Larcker, Evaluating structural equation models with unobservable variables and measurement error, Journal of Marketing Research 18.1 (1981) 39–50. doi:10.1177/ 002224378101800104. [52] J. C. Nunnally, I. H. Bernstein, Psychometric Theory, 3rd. ed., McGraw-Hill, New York, NY, 1994.

Appendix A: Item wordings

References

[1] M. H. Jarrahi , Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making , Business Horizons 61.4 ( 2018 ) 577 - 586 . doi: 10 .1016/j.bushor. 2018 . 03 .007.

[2] J. Howard , Artificial intelligence: Implications for the future of work , American Journal of Industrial Medicine 62.11 ( 2019 ) 917 - 926 . doi: 10 .1002/ajim.23037.

[3] W. Wang , K. Siau , Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: A review and research agenda , Journal of Database Management 30.1 ( 2019 ) 61 - 79 . doi: 10 .4018/JDM.2019010104.

[4] E. E. Makarius , D. Mukherjee , J. D. Fox , A. K. Fox , Rising with the machines: A sociotechnical framework for bringing artificial intelligence into the organization , Journal of Business Research 120 ( 2020 ) 262 - 273 . doi: 10 .1016/j.jbusres. 2020 . 07 .045.

[5] A. Asatiani , P. Malo , P. R. Nagbøl , E. Penttinen, T. Rinta-Kahila , A. Salovaara , Sociotechnical envelopment of artificial intelligence: An approach to organizational deployment of inscrutable artificial intelligence systems , Journal of the Association for Information Systems 22.2 ( 2021 ) 325 - 352 . doi: 10 .17705/1jais. 00664 .

[6] L. Sartori , A. Theodorou , A sociotechnical perspective for the future of AI: Narratives, inequalities, and human control , Ethics and Information Technology 24.1 ( 2022 ) 4 . doi: 10 .1007/ s10676-022-09624-3.

[7] P. Akbarighatar , I. Pappas , P. Vassilakopoulou , A sociotechnical perspective for responsible AI maturity models: Findings from a mixed-method literature review , International Journal of Information Management Data Insights 3.2 ( 2023 ) 100193 . doi: 10 .1016/j.jjimei. 2023 . 100193 .

[8] S. Sarker , S. Chatterjee , X. Xiao , A. Elbanna , The sociotechnical axis of cohesion for the IS discipline: Its historical legacy and its continued relevance , MIS Quarterly 43.3 ( 2019 ) 695 - 720 . doi: 10 .25300/MISQ/ 2019 /13747.

[9] R. Peres , M. Schreier , D. Schweidel , A. Sorescu , On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice , International Journal of Research in Marketing 40.2 ( 2023 ) 269 - 275 . doi: 10 .1016/j.ijresmar. 2023 . 03 .001.

[10] J. Rudolph , S. Tan , S. Tan , War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond: The new AI gold rush and its impact on higher education , Journal of Applied Learning & Teaching 6.1 ( 2023 ) 364 - 389 . doi: 10 .37074/jalt. 2023 . 6 .1.23.

[11] J. Kocoń , I. Cichecki , O. Kaszyca , M. Kochanek , D. Szydło , J. Baran , J. Bielaniewicz , M. Gruza , A. Janz , K. Kanclerz , A . Kocoń , B. Koptyra , W. Mieleszczenko-Kowszewicz , P. Miłkowski , M. Oleksy , M. Piasecki , Ł. Radliń ski, K. Wojtasik , S. Woźniak, P. Kazienko, ChatGPT: Jack of all trades, master of none , Information Fusion 99 ( 2023 ) 101861 . doi: 10 .1016/j.inffus. 2023 . 101861 .

[12] E. Kasneci , … G. Kasneci, ChatGPT for good? On opportunities and challenges of large language models for education , Learning and Individual Differences 103 ( 2023 ) 102274 . doi: 10 .1016/j.lindif. 2023 . 102274 .

[13] J. Rudolph , S. Tan , S. Tan, ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? , Journal of Applied Learning & Teaching 6.1 ( 2023 ) 342 - 363 . doi: 10 .37074/jalt. 2023 . 6 . 1 .9.

[14] A. Tlili , B. Shehata , M. A. Adarkwah , A. Bozkurt , D. T. Hickey , R. Huang , B. Agyemang , What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education , Smart Learning Environments 10.1 ( 2023 ) 15 . doi: 10 .1186/s40561-023-00237-x.

[15] S. K. Ahmed , The impact of ChatGPT on the nursing profession: Revolutionizing patient care and education , Annals of Biomedical Engineering ( 2023 ). doi:10.1007/s10439-023-03262-6.

[16] M. Cascella , J. Montomoli , V. Bellini , E. Bignami, Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios , Journal of Medical Systems 47.1 ( 2023 ) 33 . doi: 10 .1007/s10916-023-01925-4.

[17] J. Ferdush , M. Begum , S. T. Hossain, ChatGPT and clinical decision support: Scope, application, and limitations , Annals of Biomedical Engineering ( 2023 ). doi:10.1007/s10439-023-03329-4.

[18] J. Liu , C. Wang , S. Liu, Utility of ChatGPT in clinical practice , Journal of Medical Internet Research 25 ( 2023 ) e48568 . doi: 10 .2196/48568.

[19] M. Sallam , ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns , Healthcare 11 .6 ( 2023 ) 887 . doi: 10 .3390/ healthcare11060887.

[20] I. Carvalho , S. Ivanov, ChatGPT for tourism: Applications, benefits and risks , Tourism Review ( 2023 ). doi: 10 .1108/TR-02-2023-0088.

[21] T. Dogru , … T. Zhang, Generative artificial intelligence in the hospitality and tourism industry: Developing a framework for future research , Journal of Hospitality & Tourism Research ( 2023 ). doi: 10 .1177/10963480231188663.

[22] Y. K. Dwivedi , N. Pandey , W. Currie , A. Micu , Leveraging ChatGPT and other generative artificial intelligence (AI ) -based applications in the hospitality and tourism industry: Practices, challenges and research agenda, International Journal of Contemporary Hospitality Management ( 2023 ). doi: 10 .1108/IJCHM-05-2023-0686.

[23] D. Gursoy , Y. Li , H. Song , ChatGPT and the hospitality and tourism industry: An overview of current trends and future research directions , Journal of Hospitality Marketing & Management 32.5 ( 2023 ) 579 - 592 . doi: 10 .1080/19368623. 2023 . 2211993 .

[24] I. A. Wong , Q. L. Lian , D. Sun , Autonomous travel decision-making: An early glimpse into ChatGPT and generative AI , Journal of Hospitality and Tourism Management 56 ( 2023 ) 253 - 263 . doi: 10 .1016/j.jhtm. 2023 . 06 .022.

[25] P. Ritala , M. Ruokonen , L. Ramaul, Transforming boundaries: How does ChatGPT change knowledge work? , Journal of Business Strategy ( 2023 ). doi: 10 .1108/JBS-05-2023-0094.

[26] L. Ayinde , M. P. Wibowo , B. Ravuri , F. B. Emdad , ChatGPT as an important tool in organizational management: A review of the literature , Business Information Review 40.3 ( 2023 ) 137 - 149 . doi: 10 .1177/02663821231187991.

[27] D. L. Goodhue , R. L. Thompson , Task-technology fit and individual performance , MIS Quarterly 19.2 ( 1995 ) 213 - 236 . doi: 10 .2307/249689.

[28] T. J. Larsen , A. M. Sørebø , Ø Sørebø, The role of task-technology fit as users' motivation to continue information system use , Computers in Human Behavior 25.3 ( 2009 ) 778 - 784 . doi: 10 .1016/j.chb. 2009 . 02 .006.

[29] B. Wu , X. Chen , Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model , Computers in Human Behavior 67 ( 2017 ) 221 - 232 . doi: 10 .1016/j.chb. 2016 . 10 .028.

[30] F. D. Davis , Perceived usefulness, perceived ease of use, and user acceptance of information technology , MIS Quarterly 13.3 ( 1989 ) 319 - 340 . doi: 10 .2307/249008.

[31] V. Venkatesh , M. G. Morris , G. B. Davis , F. D. Davis , User acceptance of information technology: Toward a unified view , MIS Quarterly 27.3 ( 2003 ) 425 - 478 . doi: 10 .2307/30036540.

[32] V. Venkatesh , J. Y. L. Thong , X. Xu , Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology , MIS Quarterly 36.1 ( 2012 ) 157 - 178 . doi: 10 .2307/41410412.

[53] L.-t. Hu, P. M. Bentler , Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives , Structural Equation Modeling 6.1 ( 1999 ) 1 - 55 . doi: 10 .1080/10705519909540118.

[54] P. M. Bentler , D. G. Bonett , Significance tests and goodness of fit in the analysis of covariance structures , Psychological Bulletin 88.3 ( 1980 ) 588 - 606 . doi: 10 .1037/ 0033 - 2909 . 88 .3.588.

[55] J. F. Hair , Jr., W. C. Black , B. J. Babin , R. E. Anderson , Multivariate Data Analysis, 8th . ed., Cengage , Andover, United Kingdom, 2018 .

[56] P. M. Podsakoff , S. B. MacKenzie , J.- Y. Lee , N. P. Podsakoff , Common method biases in behavioral research: A critical review of the literature and recommended remedies , Journal of Applied Psychology 88.5 ( 2003 ) 879 - 903 . doi: 10 .1037/ 0021 - 9010 . 88 .5.879.

[57] F. F.-H. Nah , R. Zheng , J. Cai , K. Siau , L. Chen, Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration , Journal of Information Technology Case and Application Research 25.3 ( 2023 ) 277 - 304 . doi: 10 .1080/15228053. 2023 . 2233814 .

[58] F. D. Davis , R. P. Bagozzi , P. R. Warshaw , Extrinsic and intrinsic motivation to use computers in the workplace , Journal of Applied Social Psychology 22 .14 ( 1992 ) 1111 - 1132 . doi: 10 .1111/j.1559- 1816 . 1992 .tb00945.x.

[59] H. van der Heijden , User acceptance of hedonic information systems , MIS Quarterly 28.4 ( 2004 ) 695 - 704 . doi: 10 .2307/25148660.