How ACOG saves money

ACOG lowers cost in two ways: it avoids paying for work your provider has already done, and it moves work to cheaper routes that give the same result. Of these mechanisms, only the semantic cache can answer a request with another model’s answer (and only an equally or more capable model’s), and only auto mode, which you opt into, chooses the model. Outside the savings mechanisms, the policy-block fallback can retry a refused request once on another model.

The savings mechanisms

Mechanism	What it does	Default	Changes the answer?
Response cache	An identical repeat request is answered from a stored response. No upstream call is made.	On	No. The stored bytes are what the provider returned.
In-flight de-duplication	Identical requests that arrive at the same time share one upstream call.	On	No.
Provider prompt caching	ACOG adds your provider’s own prompt-cache markers, so repeated context is billed at the cheaper cached-input rate.	On	No. The provider computes the answer as usual.
Prompt cleanup	Collapses redundant whitespace and removes invisible characters from message text, and drops repeated retrieval chunks.	On	Your words are never changed. Code blocks are left untouched.
Semantic cache	A prompt that means the same as an earlier one, above a similarity threshold, is answered from the earlier answer.	On for hosted organizations, at a 0.97 threshold	It can. The answer is the earlier, near-identical prompt’s answer, and is never from a weaker model than the one requested.
Batch routing	Requests nobody is waiting for run on your provider’s batch service, at roughly half price.	On	No. The same model answers, later.
Auto mode	model: "auto" sends each request to the cheapest model strong enough to handle it.	Only when a request asks for auto	Yes, by design: you asked ACOG to choose the model.
Cost arbitrage	The same model hosted on several platforms (for example Claude on Anthropic, Bedrock and Vertex) is sent to the cheapest one, including your committed-use discounts and credits.	Off	No. The same model answers.

Each of these can be switched off for your organization under Admin → Gateway settings or Admin → Cost policies, and some can be switched off per request with a header. See each page for the details.

When there is nothing to save

A request that no mechanism applies to costs exactly what it would have cost sent directly. ACOG adds no markup to tokens.

Pass-through mode

The savings mechanisms run only while your organization is active. In pass-through mode every request is forwarded to your provider exactly as sent, at full price, and nothing is billed. Governance (keys, model access, budgets, rate limits, DLP) still applies. The console shows a banner whenever your organization is in pass-through.

Deployment	Why an organization is in pass-through	How to leave it
Hosted	No chargeable payment method is on file (every new organization starts here), or an invoice is more than 30 days overdue	Add a payment method under Admin → Billing, and pay any overdue invoice
Self-hosted	No licence is loaded, or the licence has expired or is invalid	Load a valid licence under Admin → Billing

How savings are measured and billed

For every request ACOG records two prices: what it actually cost through ACOG, and what the same request would have cost sent directly to the provider and model you asked for, at that provider’s published list price. The difference is the saving. A request answered from cache never reaches your provider, so it is missing from your provider invoice, and that absence is the saving you can check.

On the hosted service you pay a share of the measured saving, invoiced monthly. A request that saves nothing costs nothing extra. See How savings are measured and Billing.