Cut your AI spend. Same models, same answers.
A drop-in gateway that sits in front of every large-language-model provider you already use. Bring your own keys, point your SDK at one endpoint, and semantic caching, accuracy-first routing, same-model arbitrage, and batch orchestration shave the bill on every call. A model you name is never swapped to save money, and you only pay a share of what you actually save.
Bring your own keys · Every major provider · Billed only on proven savings
Drop-in, bring-your-own-keys
One endpoint in front of every provider.
Keep your own provider accounts and your own keys. Change a single base URL and your existing OpenAI- or Anthropic-shaped calls flow through the optimizer to the model you asked for, with your own provider account. Your prompts, your keys, and your data stay under your control.
- Bring your own keys (BYOK) — your provider accounts, your negotiated rates, your data-processing terms
- OpenAI- and Anthropic-compatible endpoints — usually a one-line base-URL change, no rewrites
- Works across every major provider — OpenAI, Anthropic, Google Vertex / Gemini, AWS Bedrock, Azure
- Your model, not ours — a named model is never swapped to save money; only model: auto lets the optimizer choose (a safety-classifier refusal is retried once on a fallback model; switch that off per request with a header)
Where the savings come from
Four independent levers, stacked on every request they apply to. Turn them on individually or all at once.
-
Drop-in BYOK gateway
Point your SDK at one endpoint and keep your own provider keys. No rewrites, no proxy lock-in, no data detour — requests pass straight through with your identity and your rates, and you can route around the gateway any time.
-
Semantic caching
A prompt that means the same as an earlier one is answered from the earlier answer instead of paying for the same generation twice. It is deliberately strict: a 0.97 similarity bar by default, never an answer written by a weaker model than the one requested, never across API formats, and on the compatible endpoints a header names the model that wrote it.
-
Accuracy-first model routing
Send model: auto and a lightweight judge routes each request to the cheapest model that will still get it right; if the answer comes back empty, refused or malformed, it retries on the next candidate model. Cost arbitrage sends a model you named to the platform where it costs you least. Accuracy comes first; cost comes second — never the other way around.
-
Batch orchestration + live savings
Latency-tolerant work is folded into provider batch lanes for up to 50% off, and a real-time dashboard shows exactly what you saved — per model, per provider, per route — with the naive-versus-actual cost on every single request.
The request path
Five gates, before a cent is spent.
This is the real order the gateway evaluates. Every decision is recorded with the reason it was made and a decision id you can look up.
Cached — never reached a model Routed — sent to a cheaper model Sent as asked — full price Held — stopped by your rules
Held is red because something was refused — by you. A held request stopped at one of two gates: a budget or rate ceiling you set, or a governance rule — DLP, a policy, a model you have not approved. Either way it was never sent and never billed. These are your rules, not ours: Tessarac enforces the limits you configure and never blocks a request on its own judgement.
Budget & limits
Your ceilings, enforced in the request path, not reconciled at month end. Nothing overspends quietly.
Policy & DLP
Your governance rules, applied before a provider sees a token: redacted, refused, or stopped outright.
Optimise & route
Graded, then routed. Easy work goes to a small model, hard work does not.
Cache
Two near-identical questions are one question. The second is answered from the first, for nothing.
Meter
Priced against the real rate card twice over: what it actually cost, and what the same work would have cost untouched.
Versus LiteLLM
The same models. Different levers.
LiteLLM is a capable router: it load-balances across the deployments you list. It doesn't grade how hard a prompt is, route it to the cheapest model that can answer it, or check the answer before it goes back. Those are the levers that move the bill, and they are the ones the optimizer adds.
<250ms
to serve a cached answer — versus seconds of regeneration
$0
upstream cost on every request served from cache
Cost levers, side by side
| Difficulty-based model routing | — | |
| Answer check with escalation | — | |
| Full-response cache ($0, no upstream call) | ~ | |
| Same-model cheapest-deployment routing | ~ | |
| Prompt / RAG token trimming | — | |
| In-flight request coalescing | — |
✓ Tessarac · — not offered by LiteLLM · ~ offered by LiteLLM with setup or in a different form
How we test it
Identical prompts go through both gateways. LiteLLM is pinned to the model a customer would otherwise
choose; the optimizer receives model: auto and
routes each prompt by difficulty. The harness prices every call itself, from one shared price book,
and never takes either gateway's word for what a call cost.
A control run with every saving lever switched off puts the two gateways within noise of each other: the optimizer adds no measurable cost of its own.
Gainshare pricing
You only pay a share of what you save.
No seat licenses, no per-token markup, no upfront commitment. The optimizer measures the difference between what each request would have cost as you sent it and what it actually cost after optimization — then bills a 25% share of the proven saving. If it doesn't save you money, it doesn't cost you anything.
- Measured, auditable savings — every request carries its naive-versus-actual cost line
- No savings, no charge — pricing is a share of proven savings and nothing else
- Self-host or managed — run it inside your own cluster with offline licensing, or use the hosted gateway
- Secure by construction — cache encrypted at rest, authenticated ingest, and network isolation throughout
How LLM cost reduction works.
Every technique the optimizer applies, explained on its own, including how to do it yourself.
- How to reduce LLM API costs: a practical checklist
- What is an LLM gateway, and when do you need one?
- LLM model routing: sending each request to the least costly model that can answer it
- Prompt caching explained: Anthropic, OpenAI and Gemini compared
- LLM batch APIs: the discount for work nobody is waiting on
- Semantic caching for LLMs: answering repeat questions without a model call
- AI data loss prevention: keeping sensitive data out of LLM prompts
- AI costs out of control? Budgets, attribution and visibility for LLM spend
Stop overpaying for AI.
A drop-in gateway that cuts your LLM bill — your models, your keys, and billed only when you save.