We gate build time, bundle size, and test coverage in CI โ and fail the PR when they regress. But the token/compute cost of the LLM-agent steps in our pipelines? For a long time that only showed up on the monthly bill, well after the fact.
A refactor that quietly ~5x'd the tokens a review-agent burned per PR was completely invisible until someone squinted at the invoice. By then it had been merged for weeks.
Three tool-agnostic habits that have made agent cost tractable:
1. Treat token cost as a CI metric, not an ops afterthought. The same way you'd fail a build that balloons bundle size, flag a PR whose agent step jumps in tokens/cost versus the base branch. Making it a check turns a surprise bill into a surprise review comment โ orders of magnitude cheaper to act on.
2. Attribute cost to a diff, not a month. "This PR adds 40k tokens/run to the review agent because it dumped a 900-line fixture into context" is actionable. "We spent $X in July" is not.
3. Budget before merge, not after deploy. A cheap pre-run estimate over just the changed files catches the 10x cases before they land โ no invoice required.
The gate itself is the part most teams still wire up by hand every time. If you want it off-the-shelf: npx @wartzar-bee/tokenscope measures the token/$ cost of a run, and there's an Apache-2.0 GitHub Action (wartzar-bee/ci-cost-guardrail) that fails a PR when an agent step's cost regresses past a budget โ same ergonomics as a bundle-size check.
Do you gate the cost of your agent steps the way you gate build time? Or is it still a monthly-invoice surprise?