AI costs out of control? Budgets, attribution and visibility for LLM spend

The first sign that AI costs are out of control is rarely a large number. It is a number nobody can explain. The provider invoice shows tokens by model and by day. It does not show which team, which application, which feature or which runaway agent loop spent them.

Before anything can be reduced, it has to be attributed. Before it can be capped, someone has to decide who owns which part of it.

Step 1: attribute every request

Every request should carry an answer to “who made this, and for what?”:

• A key per application and per person, rather than one shared provider key copied into every service. A shared key makes every request anonymous.
• Application identity that callers cannot spoof. If attribution comes from a header the caller sets, a misconfigured service reports its traffic as someone else’s. Bind the application to the key instead.
• Labels for the dimensions you report on: feature, environment, customer, cost center.
• The cost of each request, recorded at the time, from the token counts the provider returns (including cached-input tokens, and cache writes where the provider reports them) and the price of the model that actually answered.

With that in place, the questions finance asks have answers: spend by team, by application, by model, by day, and which requests drove a spike.

Step 2: look for the usual causes

Attributed data usually points at the same handful of causes:

• One model for everything. A capable model set as the default during development, now answering easy requests at frontier prices. See model routing.
• Agent loops. Coding and tool-using agents resend a growing context on every step. A long task can send the same system prompt and files dozens of times. See prompt caching.
• Repeated questions. Support and knowledge assistants answering the same questions all day. See semantic caching.
• Retries and fan-out. Clients that retry on timeouts, and parallel workers sending identical requests.
• Offline work at real-time prices. Nightly jobs written synchronously. See batch APIs.
• Bloated prompts. Duplicated retrieval chunks and padding that cost tokens and carry no meaning.

Step 3: set budgets that warn before they block

A budget is a spending limit for a scope (the organization, a team, a person) over a period (a day, a week, a month). Good budgets have three properties.

They warn first. A request blocked without notice breaks a workflow and erodes trust in the control. Warnings at increasing thresholds give people time to act before the limit is reached.

The warning reaches the person spending. An email to an administrator is not enough when the spender is a script or a developer in an editor. A notice in the response itself, and a machine-readable header for programs, reaches them where they are.

The tightest applicable limit wins. A request that counts against an organization budget, a team budget and a personal budget is checked against all three.

Pair budgets with rate limits for burst protection and model access rules so that only the teams who need the most expensive models can call them.

Step 4: measure savings against a baseline

Once optimizations are in place, report two numbers for every request: what it actually cost, and what it would have cost sent directly to the provider and model that was asked for, at list price. The difference is the saving, and it can be checked against the provider invoice: a request served from cache, for example, does not appear on it at all.

How the Tessarac AI Cost Optimizer helps

The AI Cost Optimizer records, for every request, the model asked for, the model that answered, the tokens used and avoided, the actual cost and what it would have cost direct. Reports break usage and savings down by user, team, model, provider, API key, application and label, and a savings-proof export reconciles each request against your provider invoices.

Budgets can be set per organization, team or person, daily, weekly or monthly. As a budget passes its warning thresholds, a one-line notice is appended to the model’s reply and a response header carries the same information for programs; webhook events notify your own systems. When a budget is used up, requests that count against it are refused with a clear error until the period resets. Applications are attributed by the key they use, which no request header can change. See Budgets, Applications and How savings are measured.