# How ACOG saves money

> Every way ACOG lowers the cost of a request, which ones are on by default, and which leave the answer byte-for-byte identical.

Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/how-acog-saves-money. Last updated: 2026-10-16.

ACOG lowers cost in two ways: it avoids paying for work your provider has already done, and it moves work to cheaper routes that give the same result. Of these mechanisms, only the semantic cache can answer a request with another model's answer (and only an equally or more capable model's), and only auto mode, which you opt into, chooses the model. Outside the savings mechanisms, the [policy-block fallback](https://tessarac.com/docs/acog/providers/routing-and-failover.md#policy-block-fallback) can retry a refused request once on another model.

## The savings mechanisms

| Mechanism | What it does | Default | Changes the answer? |
|---|---|---|---|
| [Response cache](https://tessarac.com/docs/acog/optimize/caching.md#response-cache) | An identical repeat request is answered from a stored response. No upstream call is made. | On | No. The stored bytes are what the provider returned. |
| [In-flight de-duplication](https://tessarac.com/docs/acog/optimize/caching.md#in-flight-de-duplication) | Identical requests that arrive at the same time share one upstream call. | On | No. |
| [Provider prompt caching](https://tessarac.com/docs/acog/optimize/caching.md#provider-prompt-caching) | ACOG adds your provider's own prompt-cache markers, so repeated context is billed at the cheaper cached-input rate. | On | No. The provider computes the answer as usual. |
| [Prompt cleanup](https://tessarac.com/docs/acog/optimize/request-cleanup.md) | Collapses redundant whitespace and removes invisible characters from message text, and drops repeated retrieval chunks. | On | Your words are never changed. Code blocks are left untouched. |
| [Semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md) | A prompt that means the same as an earlier one, above a similarity threshold, is answered from the earlier answer. | On for hosted organizations, at a 0.97 threshold | It can. The answer is the earlier, near-identical prompt's answer, and is never from a weaker model than the one requested. |
| [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md) | Requests nobody is waiting for run on your provider's batch service, at roughly half price. | On | No. The same model answers, later. |
| [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md) | `model: "auto"` sends each request to the cheapest model strong enough to handle it. | Only when a request asks for `auto` | Yes, by design: you asked ACOG to choose the model. |
| [Cost arbitrage](https://tessarac.com/docs/acog/optimize/cost-arbitrage.md) | The same model hosted on several platforms (for example Claude on Anthropic, Bedrock and Vertex) is sent to the cheapest one, including your committed-use discounts and credits. | Off | No. The same model answers. |

Each of these can be switched off for your organization under **Admin → Gateway settings** or **Admin → Cost policies**, and some can be switched off per request with a header. See each page for the details.

## When there is nothing to save

A request that no mechanism applies to costs exactly what it would have cost sent directly. ACOG adds no markup to tokens.

## Pass-through mode

The savings mechanisms run only while your organization is **active**. In **pass-through** mode every request is forwarded to your provider exactly as sent, at full price, and nothing is billed. Governance (keys, model access, budgets, rate limits, DLP) still applies. The console shows a banner whenever your organization is in pass-through.

| Deployment | Why an organization is in pass-through | How to leave it |
|---|---|---|
| Hosted | No chargeable payment method is on file (every new organization starts here), or an invoice is more than 30 days overdue | Add a payment method under **Admin → Billing**, and pay any overdue invoice |
| Self-hosted | No licence is loaded, or the licence has expired or is invalid | Load a valid licence under **Admin → Billing** |

## How savings are measured and billed

For every request ACOG records two prices: what it actually cost through ACOG, and what the same request would have cost sent directly to the provider and model you asked for, at that provider's published list price. The difference is the saving. A request answered from cache never reaches your provider, so it is missing from your provider invoice, and that absence is the saving you can check.

On the hosted service you pay a share of the measured saving, invoiced monthly. A request that saves nothing costs nothing extra. See [How savings are measured](https://tessarac.com/docs/acog/optimize/savings-measurement.md) and [Billing](https://tessarac.com/docs/acog/administer/billing.md).