Migration & Adoption

Measuring ROI on LLM Projects: Metrics That Matter

Vendors will happily hand you spectacular ROI numbers. Your CFO will not accept theirs — and should not accept yours unless you measured a baseline first. Here is a framework for numbers that survive scrutiny.

Claude 3P 101 · Updated July 2026 · Unofficial guide

This article deliberately contains no claimed results — no "companies typically save X%." Any such number would be invented, and the entire point of ROI measurement is to replace borrowed numbers with your own. What generalizes is the method: pick metrics tied to the specific use case, measure the baseline before launch, and compare like for like afterward. Teams that skip the baseline step do not measure ROI; they estimate it after the fact, which is a polite phrase for making it up.

Baseline first, or the numbers mean nothing

Before the tool launches, measure the process as it runs today: how long the task takes, how many units a person handles per week, what the error or rework rate is, what the queue backlog looks like. This is unglamorous work and it is the foundation of everything else — a post-launch "we handle tickets 30% faster" claim is only as credible as the pre-launch measurement it is compared against. If a workflow was never measured before, the honest sequence is: measure it for a few weeks as-is, then launch, then compare. Build the baseline collection into the project plan as a formal phase, not a retrofit.

The metrics that hold up

Time per task. The workhorse metric for assistant-style use cases: minutes to draft a response, process a document, produce a report — measured before and after, on the same task mix. Convert to value using loaded labor cost only if the freed time is actually redeployed; "hours saved" that vanish into slack are a softer claim, and honest reporting labels them as such.

Deflection and automation rates. For support and triage use cases: the share of items fully handled without human touch, and the share where the human started from an AI draft instead of a blank page. Pair deflection with a quality guardrail (reopen rate, escalation rate, customer satisfaction) so you never celebrate deflecting tickets into unresolved frustration.

Quality deltas. Error rates, rework rates, compliance-check failures, first-pass acceptance of drafts — whichever quality signal the process already cares about, measured against baseline. Quality improvements are often worth more than speed and are chronically under-measured because they are harder to count.

Cycle time and throughput. For pipeline use cases: end-to-end time from intake to done, and backlog trend. Sometimes the win is not that each task is faster but that the queue stops growing.

Fully loaded cost per unit. On the cost side, count everything: token spend (input and output are billed separately — see pricing explained), engineering and maintenance time, and the human review time the workflow still requires. Divide by units processed. This is the number to trend over time and the honest denominator for any ROI ratio.

Rule of thumb: Every ROI claim needs three ingredients: a baseline measured before launch, a metric the business already cared about, and a cost figure that includes human review time. A claim missing any of the three should be labeled an estimate, not a measurement.

What to ignore

Ignore activity metrics dressed up as value: total prompts sent, tokens consumed, number of users who logged in once. They measure motion, not outcomes (repeat usage is a fair adoption signal — see change management — but it is not ROI). Ignore demo impressions and anecdotes as evidence, useful as they are for storytelling. Ignore industry benchmark percentages, which never share your baseline, task mix, or definition of "saved." And resist the temptation to project one pilot's results across the whole company on a slide — scale changes the economics in both directions.

Report it like an adult

A credible ROI readout is small and specific: the use case, the baseline and how it was measured, the post-launch numbers on the same definitions, the fully loaded cost, and the caveats. State what you did not measure. Report the soft benefits — employee satisfaction, faster onboarding, capacity absorbed without hiring — as observations alongside the hard numbers, not blended into them. And schedule a re-measurement after several months: early results carry novelty effects in both directions, and the durable number is the one that shows up after the workflow settles. A modest, well-evidenced result builds the credibility that funds the next three projects; an inflated one spends it.

Where to go next

The cost half of the ratio deserves its own attention: estimating your monthly bill covers forecasting spend before commitment, and FinOps for LLMs covers attributing it per feature once you are live. For picking a use case likely to produce measurable ROI at all, start with the one-week proof of concept plan.