Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance

The research explores the underlying mechanisms through which AI agents influence productivity and performance. A large-scale experiment was conducted on "Pairit," a platform designed for studying human-AI collaboration. The study involved 2,234 participants assigned to either human-human or human-AI teams, tasked with producing over 11,000 advertisements. Evaluations were based on independent human ratings and a field experiment on X, which garnered approximately 5 million impressions. Key findings indicate that human-AI teams demonstrated 50% higher ad production per worker and superior text quality. In contrast, human-human teams achieved better image quality, suggesting a nuanced capability frontier for AI agents. The study also observed that human-AI teams generated more homogeneous outputs. Three primary mechanisms were identified to explain these effects. Firstly, human-AI collaboration was more task-oriented, characterized by a 25% increase in task-oriented messages and an 18% reduction in interpersonal communication. Secondly, human-AI interactions showed greater delegation, with participants delegating 17% more work to AI agents and performing 62% fewer direct text edits. Thirdly, recognizing that the collaborator was an AI agent moderated these effects, leading to more task-oriented behavior and increased delegation among participants who correctly identified their AI partner. These mechanisms collectively explain how task-oriented communication improved ad quality, particularly with AI, while interpersonal communication decreased it. Delegation enhanced text quality but did not affect image quality and was linked to a reduction in output diversity. The overall results suggest that AI agents reshape teamwork dynamics, impacting productivity, performance, and output diversity.