AI can shave considerable time off many traditional tasks. Proposals, quotes, sales copy, job postings and customer emails can come together in minutes, looking polished enough to pass an internal review. But producing work quickly is not the same as doing the job well. AI is a tool, and using it effectively means knowing what the job requires, where the tool helps and where it falls short.
One advertising study illustrates how AI-generated work can look convincing yet score lower on measures of effectiveness. Researchers working with the market research firm Ipsos showed 3,000 U.S. consumers 20 video ads for major brands. Half were made by people. The other half were made entirely with AI, working from briefs written from the original ads. Viewers struggled to tell them apart. The scores for sales potential and brand equity were different. On average, the AI ads scored 14 percent weaker on short-term sales potential and 17 percent weaker on long-term brand equity.¹ The humans outperformed AI.
The pattern behind the results is useful. AI matched the performance of people on conventional work, such as a product demonstration or a list of benefits, where the job is to carry out a familiar structure. It did worst where the result depended on emotional precision, humour or a surprising point of view. It also struggled with weak or vague briefs, and it came close to the human version when the brief was built on a real insight about the customer.¹
A study of consultants shows a similar gap between persuasive presentation and correct answers. In a controlled experiment with 758 consultants at Boston Consulting Group, those using AI on realistic tasks within its abilities completed 12.2 percent more tasks, 25.1 percent faster, at better quality. On one complex task chosen to fall outside the AI’s abilities, 84.5 percent of consultants without AI reached the correct answer, compared with 60 percent and 70.6 percent in the two groups using AI. Even so, the AI users’ recommendations were rated more coherent and persuasive, including the ones that were wrong.² Persuasive presentation was no guarantee of a correct answer.
AI is a tool, not a fit for every job. Using it effectively requires understanding both the work and the tool: what a good result looks like, what AI can do well, and where it needs your judgment or a different approach. Before you use AI-produced work in your business, ask what it needs to achieve and how you will judge its accuracy and effectiveness. Cost per document, production time and volume are easy to count, but they hide the economics of performance. Producing quotes faster saves time, but that gain may be outweighed if the quotes win less business.
Build that understanding by testing your work before and after AI. Choose one type of document your company now produces with AI, such as quotes or sales emails. Compare its performance with earlier examples used in similar circumstances. You need enough knowledge of the job to tell good results from good-looking ones, and enough knowledge of AI to decide which parts of the work to give it. That is the judgment I described in Match Experience to the Task. Where AI matches or beats the earlier results, use it more. Where it falls short, improve the instructions, add human review or use a different approach. The goal is not to use AI for every job. It is to know when and how the tool helps you do the job well.
Sources
¹ Adam Peruta, “Research: AI-Generated Ads Perform Worse Than Human-Made Ones—Even When Customers Can’t Tell Them Apart,” Harvard Business Review, September 3, 2026. Peruta co-led the study with Ipsos.
² Fabrizio Dell’Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon and Karim R. Lakhani, “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality,” Organization Science, published online March 11, 2026. The experiment was conducted with Boston Consulting Group and used GPT-4.