“When and How Artificial Intelligence Augments Employee Creativity”, Nan Jia, Xueming Luo, Zheng Fang, Chengcheng Liao2023-03-28 (, )⁠:

Can artificial intelligence (AI) assist human employees in increasing employee creativity? Drawing on research on AI-human collaboration, job design, and employee creativity, we examine AI assistance in the form of a sequential division of labor within organizations: in a task, AI handles the initial portion which is well-codified and repetitive, and employees focus on the subsequent portion involving higher-level problem-solving.

First, we provide causal evidence from a field experiment conducted at a telemarketing company. We find that:

AI assistance in generating sales leads, on average, increases employees’ creativity in answering customers’ questions during subsequent sales persuasion. Enhanced creativity leads to increased sales. However, this effect is much more pronounced for higher-skilled employees.

Next, we conducted a qualitative study using semi-structured interviews with the employees. We found that:

AI assistance changes job design by intensifying employees’ interactions with more serious customers. This change enables higher-skilled employees to generate innovative scripts and develop positive emotions at work, which are conducive to creativity. By contrast, with AI assistance, lower-skilled employees make limited improvements to scripts and experience negative emotions at work.

We conclude that employees can achieve AI-augmented creativity, but this desirable outcome is skill-biased by favoring experts with greater job skills.

…We used double randomization: 3,144 customers were randomly assigned to be served by AI human teams or human agents alone, and 40 human agents were randomly assigned to work in AI human teams or independently [20 from the top third by sales volume as the ‘best’, 20 from the bottom as the ‘worst’]. To measure employee creativity, we used voice recognition and text mining analysis to process the audio recordings of agents’ conversations during sales persuasion, identifying whether customer questions fell outside the scope of agents’ training and whether agents successfully answered these untrained questions. We also observed whether customers applied for credit cards after the sales calls. The results show that, on average, agents with AI assistance were 2.33× as successful in solving untrained questions as those without AI assistance, but the magnitude of this increase was much more pronounced for top agents—2.81× that of bottom agents. Further, causal mediation analysis demonstrates that increased success in answering untrained questions is critical for AI-assisted agents to achieve higher customer purchase rates than those obtained independently.

Subsequently, we conducted semi-structured interviews with the 28 sales agents involved in the field experiment. The agents confirmed that AI assistance changed their job design by screening out uninterested customers, thereby intensifying their interactions with more serious customers. This change impacted agents’ skills and psychology but with a distinct divergence based on the agent’s job skills. Higher-skilled agents discussed several paths through which such a change enabled them to produce more innovative scripts to address untrained questions from customers. This change also engenders positive psychological outcomes for higher-skilled agents, including better mood, higher morale, a greater sense of freedom in their position, and a more positive view of the firm. In contrast, lower-skilled agents expressed that they had limited abilities to take advantage of the opportunities presented by this change to solve untrained questions and reported greater stress, a stronger sense of defeat, and lower morale. The findings corroborate and enrich our theory by generating deeper and more nuanced insights into the underlying mechanisms through which AI assistance affects employee creativity.

…Field Experiment Setting: We conducted a randomized field experiment in a large telemarketing company in Asia, the name of which will remain confidential owing to company preferences. This company specializes in selling a wide variety of products and services to more than 30 million customers across multiple industries, including telecom, retail, fintech, and real estate. At the time of the experiment, the company was preparing to launch a new business line for marketing credit cards in partnership with a major bank. None of the employees had prior experience selling credit cards before the launch. Our experiment was conducted at the beginning of the new business launch after employees received basic training on selling credit cards with relevant scripts. This ensured that all employees had equal prior exposure and knowledge specific to credit card sales.

The company has adopted the common practice of designing sales tasks as two sequential components. In the first stage, employees call customers to introduce general information about the product and probe the initial interest of customers to generate “sales leads”, described as customers who are interested in learning more about the product (without yet committing to make a purchase). Customers who were not interested were filtered out. The sales lead generation was a well-codified activity for which the company provided numerous protocols and scripts. The second stage pertained to sales persuasion, wherein employees continued serving the leads by finding out more about their needs, trying to match their needs with the product, and convincing the lead to make a purchase (ie. to apply for a credit card in our setting). Sales persuasion was considered a much less structured activity than sales lead generation. While the company provided training to employees with a knowledge bank, unexpected questions commonly occurred, and the knowledge bank needed to be updated.5 However, these two stages were closely connected as a single sales task because the initial lead generation critically enhanced the effectiveness of subsequent second-stage sales persuasion by saving effort and mental power that would otherwise be wasted on trying to persuade customers who are inherently uninterested in the product (Sabnis et al 201311ya).

The company used AI conversational bot technology to generate sales leads and reduce labor costs. The AI conversational bot was empowered by cutting-edge deep learning neural networks, voice recognition algorithms, and natural language understanding via bidirectional encoder representations from transformers (Brynjolfsson and McAfee2014; Davenport et al 2021; Luo et al 2021). It was trained with terabytes of telemarketing call data and could engage in natural, human-like conversations with customers. Its “speech-to-text” process recognized human language and converted audio data to a machine-understandable language. Moreover, “grammatical parts-of-speech tagging” identifies each word in the corpus based on its definition and context.

Furthermore, the AI conversational bot applied deep learning algorithms to dynamically understand the answers to customer questions based on both correct answers (positive samples), which increased the probability of sales and incorrect answers (negative samples). Via the “text-to-speech” function, the trained AI conversational bot could understand customer questions and communicate correct answers drawn from the knowledge bank to the customers in natural conversations. According to the company’s records, the AI conversational bot passed the Turing test because, during the short (2–3 minute) phone conversation, nearly 97% of the customers failed to distinguish the AI conversational bot from human agents. A high-tech firm developed and commercialized AI technology before licensing the focal company.6

AI assistance freed up more time for us to think more about how to overcome some difficulties. For example, when there was no AI assistance, about half of our day was spent dialing numbers and dealing with no answers, hang-ups, short conversations, and so on. Thus, we could not handle many real cases in one day. However, after AI intervenes, we can also handle the same number of cases in one day as we previously did but have a lot more time to think. [High-performing employee #6]

Conversely, there was a major divergence in lower-skilled agents. Although lower-skilled agents also agreed that they saved time and energy and had increased opportunities to interact with customers from AI assistance, they felt that these changes did not make a difference in helping them find new or better answers because of their limited abilities and thus reported limited innovative outcomes (denoted as [2] in Figure 8).

Paying more attention and spending more time [on solving questions] probably do not make a difference; I can’t think of a better solution. [Low-performing employee #5]

Even with more time, I am not sure if I can find a better solution because solving some problems does not necessarily hinge on spending more time to think but on my limited abilities. [Low #6]

I have low ability and a weak foundation, and it is difficult for me to innovate when encountering challenging cases. [Low #1]

Nevertheless, lower-skilled agents observed their highly skilled colleagues to solve challenging questions by developing new innovative answers:

[I benefit from] the scripts developed by higher-performing colleagues. As AI manages cases that do not require skills, the remaining cases passed to humans are relatively more difficult. It is difficult for us to innovate for these cases, but my higher-performing colleagues can continue to break through and innovate, and it will also benefit us. [Low #14]

In fact, [AI assistance] can indeed help us to explore see if we can innovate the answers to the problems for which we have been trained. Although I cannot do that myself, I have seen some outstanding colleagues coming up with new answers.

…In stark contrast, dealing with such clients increased the pressure felt by lower-skilled agents who reported feelings of “depression and distress” (#L1):

I don’t feel relieved. However, it makes life more stressful because I have to deal with many more complex businesses. They give me headaches throughout the day; how can I be more relaxed? [Low #9]

See Also:

View PDF:

When and How Artificial Intelligence Augments Employee Creativity