Start
Lilium
Journal
Operations

Measuring agent cost without losing context

Total spend is useful. Cost attributed to the agent, user, tool path, and outcome is what helps a team make a decision.

July 8, 20261 min read

Every team instrumenting agents eventually builds a spend dashboard. The dashboard is rarely the hard part. Attribution is.

#Averages hide expensive paths

A stable average can conceal a small cohort of runaway traces. Long contexts, repeated tool failures, and sub-agent fan-out often affect only a fraction of runs while accounting for a large share of spend.

Start with the distribution, then preserve the ability to open the traces behind every bucket. Metrics should lead back to evidence.

#Attribute cost to decisions

Provider and model are only the first dimensions. Useful attribution includes the logical agent, prompt version, tool path, end user, environment, and final outcome.

Those dimensions let a team ask whether an expensive model is improving a target outcome or compensating for a broken workflow somewhere else.

DimensionQuestion it answers
Agent and versionDid the last prompt change move cost per run?
Tool pathWhich sequences burn tokens without improving the outcome?
End userAre a handful of accounts driving most of the spend?
OutcomeWhat does a successful run cost compared to a failed one?

#Optimize the whole run

The cheapest individual model call does not always produce the cheapest run. A stronger early decision can remove retries, tools, or escalations later in the trace.

Agent cost work should compare complete outcomes. The goal is not simply fewer tokens; it is reliable work at a cost the product can sustain.

Cost is attached to spans, so every number on a chart opens back into the run that produced it.