Test the Person and the AI Together
Most companies judge an AI tool by how well the tool performs. The better test is how well your people perform when they use it.
Erik Brynjolfsson, who directs Stanford University’s Digital Economy Lab, makes this argument. A model that performs well on its own does not establish that employees will perform better with it. He recommends comparing three things in the actual workflow: people working alone, AI working alone, and people working with AI. He calls it a centaur benchmark, after the mythical creature that is half human and half horse.¹
One experiment shows why all three numbers are important. Researchers asked 150 professional loan specialists to approve or deny loan applications drawn from real cases. Working alone, the specialists were right 68 percent of the time. When they saw the AI’s risk score, they were right 70 percent of the time. When they saw only a yes or no based on that same score, with the cutoff chosen carefully, they were right 74 percent of the time.² The AI did not change. The way its answer was presented to the person did.
There was a fourth number. Following the AI alone would have been right about 79 percent of the time, which is better than when a human was involved. That result comes from lending, where the specialists had little information the AI did not already have, and it will not hold for every job. It does show that you cannot assume which of the three will be the superior approach.
You can run a simple version of this test before you roll out a tool. Pick one process and a small sample of real work. Have the work done three ways: by the person, by the AI, and by the person using the AI. Compare the quality, the time taken and the errors that got through, and note whether the person caught the AI’s mistakes. Include one measure of the whole process, such as total turnaround time, because a task can get faster while causing a bottleneck in a slow process somewhere else.³
Each result requires a different approach from the business. If the person with AI wins, roll it out and train people to work that way. If the person alone wins, the tool is not ready for that work. If the AI alone wins, the question becomes who checks its work and who is accountable for it, which I covered in Take Ownership of AI’s Mistakes.
To get the most from your next AI tool, test it early in the design and build process. Compare how people perform alone, how the AI performs alone, and how they perform together. Use those results to design the workflow that produces the best outcome.
Sources
¹ Erik Brynjolfsson, interviewed by Nicholas Thompson, “What AI Is Really Doing to Jobs Right Now,” The Atlantic, video, May 27, 2026. The approach is set out in Andreas Haupt and Erik Brynjolfsson, “Position: AI Should Not Be An Imitation Game: Centaur Evaluations,” Proceedings of the 42nd International Conference on Machine Learning, 2025.
² Bnaya Dreyfuss and Ruru Hoong, “Calibrated Coarsening: Designing Information for AI-Assisted Decisions,” working paper, July 23, 2026.
³ Erik Brynjolfsson, Andrew McAfee and James Milin, “The ROI Gap: What 2026 Data Reveals About Enterprise AI Returns,” video, August 18, 2026. The three are co-founders of Workhelix, an AI consulting firm.