Measuring AI Performance
Most AI reporting measures activity. How many people have licenses, how many prompts were run, how many documents were processed. Activity is easy to count and it tells you almost nothing about whether the AI budget you’ve allocated is working.
There is a better measure. What does it cost us to produce one successful outcome?
Using it means identifying the outcome, which is the part most companies skip. A few practical examples:
A steel fabricator defines the outcome as a completed quote sent to a customer. AI drafts the take-off and specification, the estimator reviews and prices it. What does a sent quote cost now compared to what it cost last year?
An accounting firm measures the time to complete a month-end file that is ready for partner review.
A wholesale distributor processes a supplier invoice matched to a purchase order and posted without anyone touching it.
An insurance brokerage measures the time taken to prepare and deliver a renewal package to the client.
A transportation company uses a load booked, confirmed and dispatched.
Each of those is a unit of finished work that people in the business already recognize. Determine the cost you incur to deliver this result before you adopt AI, and then measure the cost again after adoption.
The discipline is rare. In a 2026 survey of chief executives, 56 percent named weak value tracking as a barrier to getting results from AI, yet only 14 percent said they define the expected profit and loss impact for all their AI initiatives, and only 20 percent measure AI primarily by that impact.¹
One more thing to watch is the handoff between steps in a process. This is often where measurement breaks down. AI creates a draft, a person refines it, and then passes it to a colleague. From that point on, it can be difficult to determine what value, if any, the AI contributed. Did AI reduce the total cost of the process, or did it simply shift work and cost to another stage of the workflow? To find out, follow an AI-assisted item from start to finish and compare the total cost of the workflow before and after AI was introduced. That comparison will tell you whether the process is actually delivering a financial benefit.
Pick one outcome in your business this month. Measure what that activity costs before AI, measure what it costs after AI, and compare that to the benefit created. Regardless of the result, the comparison will be enlightening.
Sources
¹ BCG AI Transformation CEO Survey, 2026, reported in Boston Consulting Group, “CEOs Are Starting to See Value from AI. Now Comes Execution,” 2026.