Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality PDF
This paper reports the results of a large field experiment on the impact of artificial intelligence (AI) on the productivity and quality of knowledge workers, including
- In collaboration with BCG, a global consulting firm, a randomized controlled experiment was conducted with 758 consultants to determine the impact of having access to AI (GPT-4).
- There is a “jaggy (bumpy) technology frontier” in AI’s capabilities, with a mix of tasks that AI can easily perform and tasks that are difficult for AI today.
- For 18 tasks within the AI frontier, using AI increased productivity by 12.2%, speed by 25.1%, and quality by over 40%. In particular, the results for those who were originally below average in ability improved by 43%.
- On the other hand, for tasks outside the AI frontier, using AI resulted in a 19% decrease in the percentage of correct answers.
- Two patterns were observed among consultants who successfully use AI: those who divide tasks between humans and AI, such as “centaurs,” and those who work with AI in an integrated manner, such as “cyborgs.
These results state that while AI has the potential to significantly increase the productivity of knowledge workers, it is important to identify and properly utilize AI’s capabilities.
On the Centaur and Cyborg Metaphors
The centaur and cyborg metaphors were explained as follows.
- Centaur-like behavior:.
- A method of close collaboration between humans and AI, analogous to the mythical half-human, half-horse creature, in which the roles of humans and AI are strategically shared; identifying the strengths of AI and humans and assigning tasks appropriate to each; understanding well the unevenness of the AI frontier and subdividing tasks into areas in which AI and humans excel; and developing a system of “human-centered” AI that is able to work with AI and humans in a way that is both human-centered and human-centered, and that is able to work with both.
- Cyborg-like behavior:.
- It is analogous to the melding of machines and humans as seen in science fiction. The human and AI are so tightly integrated that from the outside it is impossible to distinguish whether the task is being performed by a human or an AI. Tasks are performed completely collaboratively by AI and human. The paper seemed to point out that in order to master AI, it is important not only to introduce AI, but also to explore this new centaur-like and cyborg-like way of collaboration. It was also stated that further research and analysis of the differences between these two approaches, the advantages and disadvantages of each, and suitable situations would be desirable.
- bumpy boundary.
- The reason why it looks bumpy is because the standard of “this is difficult, this is easy” is created based on human development.
- It looks uneven because you apply a human measure to a non-human intelligence.
This page is auto-translated from [/nishio/Navigating the Jagged Technological Frontier](https://scrapbox.io/nishio/Navigating the Jagged Technological Frontier) using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.