“AI and Jobs: Has the Inflection Point Arrived? Evidence from an Online Labor Platform”, Dandan Qiao, Huaxia Rui, Qian Xiong2023-12-07 (tech economics)⁠:

Artificial intelligence (AI) refers to the ability of machines or software to mimic or even surpass human intelligence in a given cognitive task. While humans learn by both induction and deduction, the success of current AI is rooted in induction, relying on its ability to detect statistical regularities in task input—an ability learnt from a vast amount of training data using enormous computation resources. We examine the performance of such a statistical AI in a human task through the lens of 4 factors, including task learnability, statistical resource, computation resource, and learning techniques, and then propose a three-phase visual framework to understand the evolving relation between AI and jobs.

Based on this conceptual framework, we develop a simple economic model of competition to show the existence of an inflection point for each occupation. Before AI performance crosses the inflection point, human workers always benefit from an improvement in AI performance, but after the inflection point, human workers become worse off whenever such an improvement occurs.

To offer empirical evidence, we first argue that AI performance has passed the inflection point for the occupation of translation but not for the occupation of web development. We then study how the launch of ChatGPT, which led to improvement of AI performance on many tasks, has affected workers in these two occupations on a large online labor platform. Consistent with the inflection point conjecture, we find that translators are negatively affected by the shock both in terms of the number of accepted jobs and the earnings from those jobs, while web developers are positively affected by the very same shock.

Given the potentially large disruption of AI on employment, more studies on more occupations using data from different platforms are urgently needed.

…Our second objective is to test the above inflection point conjecture using empirical data. For that, we conducted an empirical study using the launch of ChatGPT on November 30, 2022, as an exogenous shock that raised certain regions of the CIS. Our data comes from a large online freelancing platform, and we focus on two job categories: translation and web development. We hypothesize that the occupation of translation has passed the inflection point, especially after the launch of ChatGPT, while the occupation of web development has not, even after the launch of ChatGPT. For translation, we note that the transformer architecture, which is at the core of GPT, was initially proposed to tackle the challenge of machine translation. In contrast, web development involves high-level designs and complex interactions among different components, making it more challenging for AI to completely surpass the minimal intelligence levels of tasks required for web development. On the other hand, AI tools like ChatGPT do make the job of programming more efficient by assisting programmers with debugging, code snippets, and so on. Consequently, we believe the launch of ChatGPT did shock the area of the CIS corresponding to web development. As a control group, we use the occupation of construction design because jobs in this category are 4 currently completed by humans using specialized software with very limited inputs from AI, most likely because of insufficient data for training AI.

We match workers in a treated occupation (ie. translation, web development) with workers in the control occupation (ie. the construction design) and conduct a difference-in-differences (DID) analysis at the worker-month level. Our first dependent variable is transaction volume measured either as a worker’s accepted number of focal jobs each month or the ratio between those focal jobs and the total number of jobs accepted by the worker each month. Our second dependent variable is total earnings from those focal jobs. Consistent with the inflection point conjecture, we find that for translators, their transaction volumes dropped after the launch of ChatGPT, and they earned less from translation jobs. In contrast, web developers experienced an increase in their transaction volume and earnings after the launch of ChatGPT.

…To the best of our knowledge, the closest work to ours is a concurrent working paper by Liu et al 2023, which investigates how the launch of ChatGPT affected transaction volume on an online labor platform. Their main finding is a statistically-significant decrease in transaction volume for gigs and freelancers directly exposed to ChatGPT. In contrast, our study reveals a more complex relation between AI and jobs, both theoretically and empirically. In particular, we propose the inflection point conjecture based on our conceptual framework and economic modeling that leads to different implications of ChatGPT’s launch on different job categories. It should be noted that our findings do not contradict the main finding of Liu et al 2023, because their treated job category consists of writing and programming which, according to our analyses, should have experienced opposite effects with the launch of ChatGPT. It’s likely that the platform in their study differs from the platform in our study, which we believe makes the two studies complementary to each other for the robustness of empirical research in our field.

…To further ensure the observed effects occurred after ChatGPT’s launch, all measurements were constructed based on the focal jobs accepted within a given month, rather than those completed. We excluded data from November & December 2022 to account for potential pre-launch impacts of ChatGPT and holiday effects. Hence, the study’s time frame spans 6 months before and after the shock, from May 1, 2022, to June 30, 2023.

…in terms of magnitude, the transaction volume dropped by 7.4%…suggesting a decrease in worker’s earnings from focal jobs after ChatGPT’s launch, by 30.2%…In terms of magnitude, we do find that workers in the Writing OLM experience less of a decline, with transaction volume down 4.4%, and earnings down 19.2%, compared to workers in the Translation OLM.