# AI Cost Optimizer Gateway (ACOG) — complete documentation > An OpenAI- and Anthropic-compatible gateway in front of the AI providers you already use. It routes each request to the cheapest model that can do the job, caches what can be reused, and enforces your budgets, model access and data loss prevention rules. This file holds every page of the ACOG documentation (60 pages), in reading order, separated by horizontal rules. The index is https://tessarac.com/docs/acog/llms.txt. --- # What is ACOG? > ACOG is a gateway in front of the AI providers you already use. It lowers what you pay for every request without changing the model or the answer. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/overview. Last updated: 2026-10-16. The AI Cost Optimizer Gateway (ACOG) sits between your applications and the AI providers you already pay for: Anthropic, OpenAI, Google Vertex AI, AWS Bedrock, Azure OpenAI and any OpenAI-compatible server you run. You point your SDK, coding assistant or script at one URL. ACOG forwards each request to your provider using your own provider account, and lowers the cost of the request on the way through. ACOG is **bring your own key (BYOK)**. You keep your provider accounts, your contracts and your model access. ACOG never resells provider capacity and never holds a shared provider key. Every request is sent with credentials your organization owns, or, for Google Vertex AI and Anthropic, with no stored credential at all (see [Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md)). ## What ACOG does for each request | Step | What happens | |---|---| | Authenticate | ACOG reads your gateway key (`acog_…`), finds your organization and checks the key's permissions and IP allowlist. | | Govern | Your model access rules, budgets, rate limits, data loss prevention (DLP) rules and prompt injection screening are applied before anything leaves. | | Route | The `model` in the request selects which of your provider connections serves it. `model: "auto"` lets ACOG pick the cheapest model strong enough for the request. | | Save | Repeated requests are answered from cache, identical concurrent requests share one upstream call, your provider's own prompt caching is switched on, and requests nobody is waiting for can run at batch prices. | | Measure | Every request records what it cost and what the same request would have cost sent directly, so the saving is visible per request, model, user and application. | ## What ACOG never does - **It never swaps a model you asked for to save money.** A request for `claude-sonnet-4-5` is answered by `claude-sonnet-4-5`. Only `model: "auto"`, which you opt into per request, lets ACOG choose. The one exception is a safety refusal: when a provider's classifier refuses a harmless request, ACOG retries it once on a fallback model and says so in the answer (see [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md#policy-block-fallback)); send `x-acog-policy-fallback: off` to opt out. - **It never rewrites your words.** Prompt cleanup touches whitespace and invisible characters only. Cached answers are served only for identical requests, or, with the semantic cache, for prompts that are near-identical and answered by an equally capable model. - **It never sees your provider bill.** Savings are measured from the token counts your provider reports and published list prices, and can be checked request by request against your provider invoices. ## Two ways to run it | | Hosted | Self-hosted | |---|---|---| | Where it runs | Tessarac operates it at `https://acog.tessarac.com` | Your Kubernetes cluster (or virtual machines) | | Sign-in | Tessarac's sign-in, plus your own single sign-on | Passwords, two-factor authentication, passkeys and your own single sign-on | | How you pay | A share of measured savings, invoiced monthly | An annual licence | | Data | Requests pass through Tessarac's hosted service, which runs on AWS | Nothing leaves your environment except calls to your providers and the services you configure | The product, the API and these docs are the same for both. Pages that apply to only one say so. Self-hosting starts at [Self-hosted overview](https://tessarac.com/docs/acog/self-hosted/overview.md). ## Where to go next - [Quickstart](https://tessarac.com/docs/acog/get-started/quickstart.md): connect a provider, create a key and make your first request. - [How ACOG saves money](https://tessarac.com/docs/acog/get-started/how-acog-saves-money.md): every saving mechanism, what it costs you and when it applies. - [Key concepts](https://tessarac.com/docs/acog/get-started/key-concepts.md): organizations, connections, models, keys and applications. - [Use ACOG with coding tools](https://tessarac.com/docs/acog/get-started/coding-tools.md): Claude Code, Cursor, VS Code and Zed. --- # How ACOG saves money > Every way ACOG lowers the cost of a request, which ones are on by default, and which leave the answer byte-for-byte identical. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/how-acog-saves-money. Last updated: 2026-10-16. ACOG lowers cost in two ways: it avoids paying for work your provider has already done, and it moves work to cheaper routes that give the same result. Of these mechanisms, only the semantic cache can answer a request with another model's answer (and only an equally or more capable model's), and only auto mode, which you opt into, chooses the model. Outside the savings mechanisms, the [policy-block fallback](https://tessarac.com/docs/acog/providers/routing-and-failover.md#policy-block-fallback) can retry a refused request once on another model. ## The savings mechanisms | Mechanism | What it does | Default | Changes the answer? | |---|---|---|---| | [Response cache](https://tessarac.com/docs/acog/optimize/caching.md#response-cache) | An identical repeat request is answered from a stored response. No upstream call is made. | On | No. The stored bytes are what the provider returned. | | [In-flight de-duplication](https://tessarac.com/docs/acog/optimize/caching.md#in-flight-de-duplication) | Identical requests that arrive at the same time share one upstream call. | On | No. | | [Provider prompt caching](https://tessarac.com/docs/acog/optimize/caching.md#provider-prompt-caching) | ACOG adds your provider's own prompt-cache markers, so repeated context is billed at the cheaper cached-input rate. | On | No. The provider computes the answer as usual. | | [Prompt cleanup](https://tessarac.com/docs/acog/optimize/request-cleanup.md) | Collapses redundant whitespace and removes invisible characters from message text, and drops repeated retrieval chunks. | On | Your words are never changed. Code blocks are left untouched. | | [Semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md) | A prompt that means the same as an earlier one, above a similarity threshold, is answered from the earlier answer. | On for hosted organizations, at a 0.97 threshold | It can. The answer is the earlier, near-identical prompt's answer, and is never from a weaker model than the one requested. | | [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md) | Requests nobody is waiting for run on your provider's batch service, at roughly half price. | On | No. The same model answers, later. | | [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md) | `model: "auto"` sends each request to the cheapest model strong enough to handle it. | Only when a request asks for `auto` | Yes, by design: you asked ACOG to choose the model. | | [Cost arbitrage](https://tessarac.com/docs/acog/optimize/cost-arbitrage.md) | The same model hosted on several platforms (for example Claude on Anthropic, Bedrock and Vertex) is sent to the cheapest one, including your committed-use discounts and credits. | Off | No. The same model answers. | Each of these can be switched off for your organization under **Admin → Gateway settings** or **Admin → Cost policies**, and some can be switched off per request with a header. See each page for the details. ## When there is nothing to save A request that no mechanism applies to costs exactly what it would have cost sent directly. ACOG adds no markup to tokens. ## Pass-through mode The savings mechanisms run only while your organization is **active**. In **pass-through** mode every request is forwarded to your provider exactly as sent, at full price, and nothing is billed. Governance (keys, model access, budgets, rate limits, DLP) still applies. The console shows a banner whenever your organization is in pass-through. | Deployment | Why an organization is in pass-through | How to leave it | |---|---|---| | Hosted | No chargeable payment method is on file (every new organization starts here), or an invoice is more than 30 days overdue | Add a payment method under **Admin → Billing**, and pay any overdue invoice | | Self-hosted | No licence is loaded, or the licence has expired or is invalid | Load a valid licence under **Admin → Billing** | ## How savings are measured and billed For every request ACOG records two prices: what it actually cost through ACOG, and what the same request would have cost sent directly to the provider and model you asked for, at that provider's published list price. The difference is the saving. A request answered from cache never reaches your provider, so it is missing from your provider invoice, and that absence is the saving you can check. On the hosted service you pay a share of the measured saving, invoiced monthly. A request that saves nothing costs nothing extra. See [How savings are measured](https://tessarac.com/docs/acog/optimize/savings-measurement.md) and [Billing](https://tessarac.com/docs/acog/administer/billing.md). --- # Quickstart > Sign in, connect a provider, create a gateway key and send your first request through ACOG in about ten minutes. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/quickstart. Last updated: 2026-10-16. This guide takes you from nothing to a working request through the hosted service. On a self-hosted installation the steps are the same; replace `https://acog.tessarac.com` with your installation's address. You need an account with at least one AI provider (for example an Anthropic API key, or a Google Cloud project with Vertex AI enabled). ## 1. Sign in and create your organization Go to [acog.tessarac.com](https://acog.tessarac.com) and choose **Sign in or create your account**. The first time you sign in you are asked to name your organization. You become its admin. Everything in ACOG belongs to an organization: provider connections, keys, usage, savings and invoices. Nothing is shared between organizations. ## 2. Connect a provider 1. Open **Admin → Providers** and choose **Connect provider**. 2. Pick the platform (Direct, Bedrock, Vertex, Azure or Custom) and the model family. 3. Enter the credentials. For Google Vertex AI and Anthropic you can choose **Keyless (workload identity)** instead of storing a key; see [Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md). 4. Review the models ACOG found in your account and choose a **default model**. ACOG checks the credentials when you save. The secret is encrypted and is never shown again. The provider pages explain each platform's fields: [Anthropic](https://tessarac.com/docs/acog/providers/anthropic.md), [OpenAI](https://tessarac.com/docs/acog/providers/openai.md), [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md), [AWS Bedrock](https://tessarac.com/docs/acog/providers/aws-bedrock.md), [Azure OpenAI](https://tessarac.com/docs/acog/providers/azure-openai.md) and [custom endpoints](https://tessarac.com/docs/acog/providers/custom-endpoints.md). ## 3. Switch on savings A new hosted organization starts in **pass-through** mode: requests work, but they are forwarded unchanged at full price and nothing is saved. To switch the savings on, add a payment method under **Admin → Billing**. You are billed only a share of what ACOG measurably saves, monthly. See [Billing](https://tessarac.com/docs/acog/administer/billing.md). You can do the rest of this guide first and add the payment method later. ## 4. Create a gateway key 1. Open **Developer** and create a key. Give it a name that says where it will be used, for example `laptop` or `billing-service`. 2. Copy the key. It starts with `acog_` and **is shown only once**: ACOG stores only a hash of it. 3. Store it as an environment variable: ```bash export ACOG_API_KEY="acog_..." ``` A key you create for yourself carries your permissions and your usage is attributed to you. Admins can also create service keys for applications. See [API keys](https://tessarac.com/docs/acog/administer/api-keys.md). ## 5. Check the key This call reaches no provider and costs nothing. It lists the models your key can use: ```bash curl -s https://gateway.acog.tessarac.com/v1/models \ -H "Authorization: Bearer $ACOG_API_KEY" ``` | Status | Meaning | |---|---| | `200` | The key works. The list is what it can route to; an empty list means no provider is connected yet. | | `401` | The key was not read: it is missing, revoked, or in a header ACOG does not read. | | `403` | The key is valid but lacks the gateway access permission (`proxy`). | | `503` | ACOG could not reach its key store. This is not your key; retry. | ## 6. Send a request Use any model you declared on a connection. ACOG speaks both the OpenAI and the Anthropic API, and translates between them, so either SDK can reach any of your models. **curl (OpenAI-compatible)** ```bash curl -s https://gateway.acog.tessarac.com/v1/chat/completions \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "claude-sonnet-4-5", "messages": [{"role": "user", "content": "Hello"}]}' ``` **Python, OpenAI SDK** ```python import os from openai import OpenAI client = OpenAI( base_url="https://gateway.acog.tessarac.com/v1", api_key=os.environ["ACOG_API_KEY"], ) completion = client.chat.completions.create( model="gemini-2.5-flash", messages=[{"role": "user", "content": "Hello"}], ) print(completion.choices[0].message.content) ``` **Python, Anthropic SDK** ```python import os from anthropic import Anthropic client = Anthropic( base_url="https://gateway.acog.tessarac.com", # Pass the gateway key as auth_token (sent as Authorization: Bearer), not api_key. auth_token=os.environ["ACOG_API_KEY"], ) message = client.messages.create( model="claude-sonnet-4-5", max_tokens=256, messages=[{"role": "user", "content": "Hello"}], ) print(message.content[0].text) ``` **Node.js, OpenAI SDK** ```javascript import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://gateway.acog.tessarac.com/v1", apiKey: process.env.ACOG_API_KEY, }); const completion = await client.chat.completions.create({ model: "gpt-4.1-mini", messages: [{ role: "user", content: "Hello" }], }); console.log(completion.choices[0].message.content); ``` The two base URLs differ only because of the path each SDK appends: the OpenAI SDK expects `/v1` in the base URL and the Anthropic SDK adds its own. See [Base URLs and authentication](https://tessarac.com/docs/acog/api/base-urls-and-authentication.md). To let ACOG pick the model, send `"model": "auto"`. See [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md). ## 7. See the result Open **Overview** in the console. Within a few seconds the request appears with its cost and what it would have cost sent directly. **Reports → Recent requests** shows each request's model, tokens, cost and saving. ## Next steps - Point your coding assistant at ACOG: [Use ACOG with coding tools](https://tessarac.com/docs/acog/get-started/coding-tools.md). - Set spending limits: [Budgets](https://tessarac.com/docs/acog/govern/budgets.md). - Invite your team: [Users and invitations](https://tessarac.com/docs/acog/administer/users-and-invitations.md). --- # Key concepts > Organizations, provider connections, declared models, the default connection, gateway keys, applications and teams: the vocabulary used in all ACOG docs. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/key-concepts. Last updated: 2026-10-16. ## Organization An organization is a tenant. Every connection, key, budget, cache entry, usage record and invoice belongs to exactly one organization, and nothing is shared between organizations. You can belong to more than one; the organization switcher at the top of the sidebar moves between them. ## Provider connection A connection is one set of credentials for one platform and one model family, for example *Anthropic models on Google Vertex AI in project `my-project`* or *GPT models on Azure OpenAI resource `my-resource`*. Platforms are called **channels** in the API: | Channel | Model families | |---|---| | `direct` | `anthropic` (the Anthropic API), `openai` (the OpenAI API) | | `bedrock` | `anthropic` (models on AWS Bedrock) | | `vertex` | `anthropic` (Claude on Vertex AI), `gemini` | | `azure` | `openai` (Azure OpenAI) | | `custom` | `compatible` (any OpenAI-compatible server) | An organization has at most one connection per channel and family. Credentials are encrypted at rest and can be replaced but never read back. ## Declared models Each connection declares the models it serves. **Routing is by model**: the `model` field of a request selects the connection that declared it. You never choose a provider in a request; you choose a model. - A declared id ending in `*` matches any model id with that prefix, and `*` alone matches every model. - A version suffix is matched loosely: a request for `claude-haiku-4-5` reaches a connection that declared `claude-haiku-4-5@20251001`, and the other way round. A match never crosses model families or major versions. - A model that no connection declares is refused with `409 model_not_configured`. If several connections declare the same model, ACOG spreads requests across them and fails over between them. See [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md). ## Default connection and default model One connection is the organization's **default**, and each connection has a **default model**. A request for `model: "auto"` goes to the default connection's default model when auto mode is off. A model that the default connection declares is always served by the default connection. ## Gateway key A gateway key (`acog_…`) authenticates requests to the gateway and to the management API. There are two kinds: | Key type | Who uses it | Usage is attributed to | Limits and model rules | |---|---|---|---| | User key | A person | That person | Their per-person limits apply | | Service key | An application, script or bot | The key | Set on the key itself | A key carries **scopes**, which are permissions. A key needs the `proxy` scope to call the gateway. A key never has more permissions than the person who created it. See [API keys](https://tessarac.com/docs/acog/administer/api-keys.md). ## Application An application is a name for traffic, such as `support-bot` or `nightly-etl`. Assign service keys to it and every request made with those keys is reported under that application, in reports and in traces. Attribution comes only from the key: no request header can change it. Requests from keys with no application are reported as `unattributed`. See [Applications](https://tessarac.com/docs/acog/administer/applications.md). ## Team A team groups members below the organization. Budgets, rate limits and model access can be set per team. In the budgets API a team is called a **group**. See [Teams](https://tessarac.com/docs/acog/administer/teams.md). ## Scopes of control Governance rules attach to scopes, from broadest to narrowest: | Rule | Scopes | |---|---| | Model access | Organization, team, user, service key | | Budgets | Organization, team (group), user | | Rate limits | Organization, application, team, user, service key | | Data loss prevention rules | Organization, team, user, application | A narrower scope can only tighten what a broader one allows. ## Request outcome (disposition) Every request is recorded with how it was served: | Disposition | Meaning | |---|---| | Exact cache | Answered from the response cache; no upstream call | | Semantic cache | Answered from the semantic cache; no upstream call | | Dedup | Shared an identical in-flight request's upstream call | | Optimized | Forwarded with savings applied (prompt caching, cleanup, routing) | | Batch | Served by the provider's batch service | | Pass-through | Forwarded unchanged | --- # Use ACOG with coding tools > Point Claude Code, Cursor, VS Code (Continue and Cline) and Zed at ACOG, and avoid the leftover settings that silently bypass it. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/get-started/coding-tools. Last updated: 2026-10-16. Coding assistants send a lot of repeated context, which is exactly what ACOG's caching saves on. Each tool below needs a base URL, your gateway key and a model. The console's **Documents → Quickstart** page shows the same settings filled in with your own gateway address and a model you have connected. In the examples, the gateway is `https://gateway.acog.tessarac.com`, the hosted gateway. Tools that speak the Anthropic API use that base. Tools that speak the OpenAI API use it with `/v1` added. ## Claude Code Set these in your shell, or in the `env` block of Claude Code's `settings.json`: ```bash export ANTHROPIC_BASE_URL=https://gateway.acog.tessarac.com # The gateway key goes in ANTHROPIC_AUTH_TOKEN (sent as Authorization: Bearer), not ANTHROPIC_API_KEY. export ANTHROPIC_AUTH_TOKEN=acog_... export ANTHROPIC_MODEL=claude-sonnet-4-5 # Background work (summaries, compaction) uses the small model. Pin it too. export ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-haiku-4-5 claude ``` For the VS Code or Cursor extension, put the same variables under `claudeCode.environmentVariables`. To use a model that is not a Claude model (for example Gemini through ACOG's translation), also set `ANTHROPIC_CUSTOM_MODEL_OPTION` to the model id and `ANTHROPIC_CUSTOM_MODEL_OPTION_NAME` to a display name. Check these before you start: - Unset `CLAUDE_CODE_USE_BEDROCK`, `CLAUDE_CODE_USE_VERTEX` and `CLAUDE_CODE_USE_FOUNDRY`. Any of them sends traffic straight to that cloud and bypasses ACOG, with no error. - Unset `ANTHROPIC_API_KEY`. If it is set, Claude Code sends it as `x-api-key`, ACOG does not recognize it as a gateway key, and requests fail with `401`. - A stale `ANTHROPIC_MODEL` or `ANTHROPIC_SMALL_FAST_MODEL` naming a model none of your connections declare fails with `409 model_not_configured`. When a model does not support extended thinking or an `anthropic-beta` feature Claude Code asks for, ACOG retries without it and marks the response with `x-acog-thinking-stripped` or `x-acog-beta-stripped`, so the session keeps working. ## Cursor In **Settings → Models → OpenAI**: 1. Turn on **Override OpenAI Base URL** and enter `https://gateway.acog.tessarac.com/v1`. 2. Enter your gateway key as the **OpenAI API Key**. 3. Add a custom model with the exact model id, for example `claude-sonnet-4-5`. Remove any other OpenAI base URL override. Cursor sends requests through its own servers first, and a custom base URL applies to chat and inline edits rather than to every Cursor feature. ## VS Code: Continue Add the model both ways in Continue's `config.json`. The same model id works on both bases; which provider serves it is decided by ACOG's routing, not by the entry: ```json { "models": [ { "title": "ACOG (Anthropic-compatible)", "provider": "anthropic", "model": "claude-sonnet-4-5", "apiBase": "https://gateway.acog.tessarac.com", "apiKey": "acog_..." }, { "title": "ACOG (OpenAI-compatible)", "provider": "openai", "model": "claude-sonnet-4-5", "apiBase": "https://gateway.acog.tessarac.com/v1", "apiKey": "acog_..." } ] } ``` Continue reads the key from its configuration, not from your shell. Remove other entries for the same models and unset `OPENAI_BASE_URL`: when several entries match, the last one wins. The same configuration works in JetBrains IDEs. ## VS Code: Cline In Cline's settings, choose the **OpenAI Compatible** API provider, set the base URL to `https://gateway.acog.tessarac.com/v1`, paste your gateway key as the API key and enter the model id. ## Zed Add a `language_models` block to `~/.config/zed/settings.json`. Zed picks the change up immediately: ```json { "language_models": { "anthropic": { "api_url": "https://gateway.acog.tessarac.com" }, "openai_compatible": { "acog": { "api_url": "https://gateway.acog.tessarac.com/v1", "available_models": [{ "name": "gemini-2.5-flash", "max_tokens": 128000 }] } } } } ``` Zed does not read keys from `settings.json`. Enter the gateway key in Zed's provider settings when it asks. Remove any other `api_url` override for the same provider. ## Which model to use Any model declared on one of your connections works with any of these tools, because ACOG translates between the OpenAI and Anthropic APIs. Use `auto` to let ACOG choose per request; see [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md). If the same model is available on more than one platform and you want one in particular, add the header `x-acog-channel` (for example `ANTHROPIC_CUSTOM_HEADERS="x-acog-channel: vertex"` in Claude Code). See [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md). --- # Connecting providers > How provider connections work: platforms and model families, declared models, the default connection, and how ACOG stores your credentials. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/overview. Last updated: 2026-10-16. ACOG calls AI providers only with credentials your organization owns. A **connection** holds those credentials for one platform and one model family. Open **Admin → Providers** to see your connections and **Connect provider** to add one. You need the **Connect and disconnect AI provider accounts** permission (`admin:providers`). ## Platforms and model families | Platform | Model families | Credentials | Page | |---|---|---|---| | Direct | Anthropic | API key, or keyless (workload identity federation) | [Anthropic](https://tessarac.com/docs/acog/providers/anthropic.md) | | Direct | OpenAI | API key | [OpenAI](https://tessarac.com/docs/acog/providers/openai.md) | | Google Vertex AI | Anthropic (Claude), Gemini | Service account key, or keyless (workload identity federation) | [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md) | | AWS Bedrock | Anthropic and other Bedrock models | Access keys, or a role ACOG assumes | [AWS Bedrock](https://tessarac.com/docs/acog/providers/aws-bedrock.md) | | Azure OpenAI | OpenAI | API key | [Azure OpenAI](https://tessarac.com/docs/acog/providers/azure-openai.md) | | Custom | OpenAI-compatible | Optional API key | [Custom endpoints](https://tessarac.com/docs/acog/providers/custom-endpoints.md) | A platform that hosts several families needs one connection per family. Claude and Gemini in the same Google Cloud project are two connections. An organization has at most one connection per platform and family. Saving a new connection for a platform and family that already has one replaces its credentials. ## The connect wizard 1. **Provider.** Choose the platform and model family. 2. **Credentials.** Enter the credentials and the platform's settings (region, project, endpoint). ACOG checks them before saving. 3. **Review models.** ACOG lists the models your account can call and you choose which ones this connection serves, and its **default model**. If the listing is not available you can enter model ids by hand. Models must be enabled in your own provider account (Bedrock model access, Vertex AI Model Garden, an Azure deployment). If a request is refused because a model is not enabled, ACOG answers `422 model_access_required` with instructions for that platform. ## Declared models The models a connection serves are its **declared models**. A request's `model` selects the connection that declared it. The list is open: you can declare any model id, including one released after you connected, and a declared id ending in `*` matches every id with that prefix. Declaring a model only tells ACOG it may send that model to this connection. Your provider decides whether your account may call it. ACOG checks each connection once a day and tells your admins when the provider offers models the connection does not declare, or no longer serves one it does. It never changes a connection for you. To turn the check off, clear **Daily model check** under **Admin → Webhooks**. ## The default connection One connection is your organization's **default**. Your first connection becomes the default automatically. To move it, choose **Set as default** on another connection. There is always exactly one default while any connection exists. The default matters in two ways: - A request for `model: "auto"` (with auto mode off) goes to the default connection's default model. - A model the default connection declares is served only by the default connection, with no spreading or failover to other connections. See [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md). ## How credentials are stored - Secrets (API keys, service account keys, AWS secret keys) are encrypted with AES-256-GCM before they are stored, and are bound to your organization and the connection's platform and family. - Secrets are write-only. The console shows a masked hint such as `••••a1b2`, never the secret. - Keyless connections store no secret at all. ## Changing or removing a connection You can edit a connection's settings, models and default model at any time; changes apply from the next request. To rotate a credential, enter the new one and save. **Disconnect** removes the connection. Models it declared stop resolving until another connection declares them, and requests for them fail with `409 model_not_configured`. ## Hosted service: what ACOG can reach The hosted service calls providers from the public internet. A connection's base URL, endpoint or token issuer must be a public address on port 443 with HTTPS. Private, LAN and loopback addresses, other ports and `http://` URLs are refused when you save, and a name that later resolves to a private address is not called (`409 connection_not_usable`). To use a model server on your own network, put it behind a public HTTPS address you control, or run ACOG self-hosted, which calls any address you configure. --- # Keyless connections > Connect Google Vertex AI and Anthropic without storing any key: ACOG signs a short-lived identity token that your cloud account trusts. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/keyless-connections. Last updated: 2026-10-16. A keyless connection stores **no credential at all**. Instead of handing ACOG a service-account key file or an API key, you tell your Google Cloud project or your Anthropic organization to trust an identity that ACOG issues for your organization. For every request, ACOG signs a short-lived token as that identity, exchanges it with Google or Anthropic for an access token, and uses that. There is nothing to rotate, nothing to leak, and access ends the moment you remove the trust on your side. Keyless connections are available for: | Provider | How you set it up | Page | |---|---|---| | Google Vertex AI (Gemini and Claude on Vertex) | A workload identity pool and OIDC provider in your Google Cloud project | [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md) | | Anthropic (direct API) | An issuer, a service account and a federation rule in the Claude Console | [Anthropic](https://tessarac.com/docs/acog/providers/anthropic.md) | ## How it works 1. **Your organization has its own identity.** ACOG runs an OpenID Connect issuer for each organization. Yours has an issuer URL and a subject, both shown in the console when you choose **Keyless** for a connection: | Value | Example | |---|---| | Issuer URL | `https://api.acog.tessarac.com/v1/identity/q4ztn6xo2bk7wmy3hfr5lc2pda` | | Subject | `acog:org:q4ztn6xo2bk7wmy3hfr5lc2pda` | Both are fixed for your organization and cannot be chosen. No other organization's tokens carry your issuer or your subject, so trusting them trusts your organization and nothing else. 2. **You configure your provider to trust that identity.** In Google Cloud that is a workload identity pool provider whose attribute condition admits only your subject. In Anthropic it is a federation rule that matches only your subject. 3. **ACOG exchanges a token on each use.** When the gateway needs to call your provider it asks ACOG's control plane to sign a token (valid for 30 minutes) for your organization, exchanges it at Google's Security Token Service or Anthropic's token endpoint, and caches the resulting access token until shortly before it expires. 4. **Your provider verifies the token** against your issuer's public keys at `/.well-known/openid-configuration` and `/jwks.json`. Google and Anthropic fetch these themselves; you do not upload anything. The signing keys are held in AWS KMS on the hosted service and never leave it. ACOG rotates them automatically every 90 days, publishing each new key a day before it signs, so your provider already trusts it when it starts being used. You do not need to do anything when keys rotate. ## What a keyless connection stores | Connection type | Stored settings | Stored secret | |---|---|---| | Google Vertex AI | Project, region, workload identity provider name, optional service account to impersonate | None | | Anthropic | Organization ID, federation rule ID, service account ID, optional workspace ID | None | When you switch an existing key-based connection to keyless, ACOG deletes its stored key. Switching back requires entering a key again. ## Self-hosted installations On a self-hosted installation the same two options exist, plus a third: **install identity** (`ambient`), where the connection uses the identity your installation already runs as (for example GKE Workload Identity, a Google credentials file, or a projected Kubernetes service-account token trusted by Anthropic). See [Install identity for keyless connections](https://tessarac.com/docs/acog/self-hosted/install-identity.md). A self-hosted installation offers keyless connections only when its operator has configured an issuer URL that Google and Anthropic can reach, or when the provider is given the key set directly. The console says so when keyless is not available. ## Troubleshooting When an exchange is refused, the console shows the provider's short error code, and the gateway answers requests with `502 upstream_auth_error`. After a refusal the gateway waits 30 seconds before trying that connection again, so a misconfiguration costs one exchange every 30 seconds rather than one per request. | What you see | Usual cause | Fix | |---|---|---| | `google refused the credentials (invalid_target)` | The workload identity provider name is wrong, or the provider or pool is disabled | Copy the provider's full resource name from Google Cloud and check it is enabled | | `google refused the credentials (invalid_grant)` | The provider's issuer URL does not match your organization's, or its attribute condition does not match your subject | Compare the provider's issuer and condition with the values shown in the console | | Google `403` on model calls after a successful exchange | The identity has no Vertex AI permission, or the model is not enabled for your project | Grant `roles/aiplatform.user` (shown in Google Cloud as **Agent Platform User**) and enable the model in Model Garden | | `anthropic refused the credentials (authentication_error)` | The rule, service account or organization ID does not match, the issuer is not registered, or the rule's subject differs | Check the IDs, and the rule's issuer and subject, in the Claude Console. Its **Authentication events** list shows the reason | | Model list falls back to "ACOG's current models" on Vertex | The identity may not list models (`serviceusage.services.use`) | Optional: grant `roles/serviceusage.serviceUsageConsumer`; requests work either way | --- # Anthropic > Connect the Anthropic API to ACOG with an API key, or keyless with Anthropic workload identity federation so no key is stored anywhere. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/anthropic. Last updated: 2026-10-16. A Direct Anthropic connection sends requests to the Anthropic API (`https://api.anthropic.com`) with your organization's own Anthropic account. It can authenticate in two ways: | Method | What ACOG stores | Best for | |---|---|---| | **API key** | Your Anthropic API key, encrypted | Getting started quickly | | **Keyless (workload identity)** | Four identifiers, no secret | Production: nothing to rotate or leak | Claude models on Google Vertex AI or AWS Bedrock are separate connections; see [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md) and [AWS Bedrock](https://tessarac.com/docs/acog/providers/aws-bedrock.md). ## Connect with an API key 1. In the [Claude Console](https://platform.claude.com/settings/keys), create an API key in the workspace whose limits and billing you want ACOG's traffic to use. 2. In ACOG, open **Admin → Providers → Connect provider**, choose **Direct** and **Anthropic**, and select **API key**. 3. Paste the key and save. ACOG lists the models the key can call; choose the ones to declare and a default model. To rotate the key, create a new one in the Claude Console, paste it into the connection and save, then delete the old key. ## Connect keyless A keyless connection uses [Anthropic workload identity federation](https://platform.claude.com/docs/en/manage-claude/workload-identity-federation). Your Anthropic organization trusts an identity issuer that ACOG runs for your ACOG organization. For each exchange, ACOG signs a short-lived token for your organization and trades it at Anthropic for a short-lived access token that acts as a service account in your workspace. Read [Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md) for how this works. You need the admin, owner or primary owner role in your Anthropic organization. ### 1. Get your organization's identity from ACOG In ACOG, open **Admin → Providers → Connect provider**, choose **Direct** and **Anthropic**, and select **Keyless (workload identity)**. The console shows three values with copy buttons: | Value | Example | |---|---| | Issuer URL | `https://api.acog.tessarac.com/v1/identity/q4ztn6xo2bk7wmy3hfr5lc2pda` | | Subject | `acog:org:q4ztn6xo2bk7wmy3hfr5lc2pda` | | Audience | `https://api.anthropic.com` | Keep this page open; you will come back to it. ### 2. Set up federation in the Claude Console Go to **Settings → Workload identity** in the [Claude Console](https://platform.claude.com/settings/workload-identity-federation) and choose **Connect workload**, then **Custom OIDC**. Create: 1. **An issuer.** - Issuer URL: your ACOG issuer URL, exactly as shown. - Key source: **discovery**. Anthropic fetches ACOG's public keys from the issuer URL. - The issuer's maximum token lifetime must be at least 30 minutes (the default, one hour, is fine). ACOG's tokens are valid for 30 minutes. 2. **A service account** in the workspace whose limits and billing ACOG's traffic should use. 3. **A federation rule** for that issuer: - Subject: your ACOG subject, exactly, with no trailing `*`. This is what makes the rule accept your organization and nothing else. - Audience (optional): `https://api.anthropic.com`. - Target: the service account, in that workspace. - Scope: `workspace:developer`. Note the **rule ID** (`fdrl_…`) and the **service account ID** (`svac_…`). Your **organization ID** (a UUID) is under **Settings → Organization**. If the rule covers more than one workspace, note the **workspace ID** (`wrkspc_…`) too. ### 3. Finish the connection in ACOG Back in the ACOG console, enter the organization ID, federation rule ID, service account ID and, if needed, the workspace ID. Save. ACOG performs a token exchange and lists your models to confirm the setup. Anthropic's **Authentication history** tab on the Workload identity page shows each exchange ACOG makes, and the reason for any that fail. ### What ACOG sends For each exchange ACOG calls `POST https://api.anthropic.com/v1/oauth/token` with a JWT-bearer grant naming your rule, service account, organization and workspace. It then calls the Messages API with `Authorization: Bearer ` and the `oauth-2025-04-20` beta flag, and never sends an `x-api-key`. Each token ACOG signs carries a fresh `jti`, so Anthropic's single-use check stays on. ### Switching an existing connection to keyless Edit the connection, select **Keyless (workload identity)** and fill in the identifiers. When you save, ACOG deletes the stored API key. Delete the key in the Claude Console afterwards. Switching back to an API key requires entering a key again. ### Removing access To cut ACOG off immediately, delete or disable the federation rule (or the issuer) in the Claude Console. Access tokens ACOG already holds expire within the rule's token lifetime. ## Settings | Setting | Notes | |---|---| | Base URL | Defaults to `https://api.anthropic.com`. Change it only for an Anthropic-compatible endpoint. On the hosted service it must be a public HTTPS address on port 443. | | Models | The Claude models this connection serves. See [Connecting providers](https://tessarac.com/docs/acog/providers/overview.md#declared-models). | | Default model | Used for `model: "auto"` when this is your default connection. | | Cache lifetime | How long Anthropic keeps prompt-cache entries ACOG creates: 5 minutes or 1 hour. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-lifetime). | ## Calling Anthropic through ACOG Any of your models can be called through the [Messages API](https://tessarac.com/docs/acog/api/messages.md) or the [Chat Completions API](https://tessarac.com/docs/acog/api/chat-completions.md). To use the Anthropic API exactly as Anthropic defines it, including endpoints ACOG does not translate (token counting, Message Batches, Files), use the native route `https://gateway.acog.tessarac.com/anthropic`. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Troubleshooting | What you see | Cause | Fix | |---|---|---| | `422 upstream_auth_failed` on an API-key connection | Anthropic rejected the stored key | Paste a valid key into the connection | | `502 upstream_auth_error` on a keyless connection | Anthropic refused the token exchange | Open **Authentication history** in the Claude Console for the reason. Check the rule's issuer URL and subject match ACOG's values exactly, and the IDs entered in ACOG | | `authentication_error` with reason `jti_reused` | The same token was presented twice | Only possible on a self-hosted install using its own identity for several connections; see [Install identity](https://tessarac.com/docs/acog/self-hosted/install-identity.md) | | `422 model_access_required` | Your Anthropic workspace cannot use the model | Check the model is available to the workspace | After a refused exchange ACOG waits 30 seconds before trying that connection again, so a misconfiguration does not flood your authentication history. --- # Google Vertex AI > Connect Claude and Gemini on Vertex AI with a service account key or keyless with workload identity federation, and control where requests run. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/google-vertex-ai. Last updated: 2026-10-16. A Vertex AI connection calls models in your own Google Cloud project. Claude models (the `anthropic` family, called with `rawPredict`) and Gemini models (the `gemini` family, called with `generateContent`) are separate connections on the same project. | Method | What ACOG stores | Best for | |---|---|---| | **Service account key** | A service account key file (JSON), encrypted | Getting started quickly | | **Keyless (workload identity)** | The workload identity provider's name, and optionally a service account to impersonate. No secret. | Production. Many organizations forbid service account keys entirely. | Before you start, enable the Vertex AI API in the project and enable the models you want in **Model Garden**. ## Connect with a service account key 1. In Google Cloud, create a service account and grant it **Vertex AI User** (`roles/aiplatform.user`) on the project. Google now labels this role **Agent Platform User** in some views. 2. Create a JSON key for it. 3. In ACOG, open **Admin → Providers → Connect provider**, choose **Vertex** and the model family, select **Service account key**, and upload or paste the key file. 4. Enter the project ID and region, then review the models and choose a default. To let ACOG list the project's models, also grant **Service Usage Consumer** (`roles/serviceusage.serviceUsageConsumer`). Without it, requests still work, and the console offers ACOG's own model catalog instead and says so. ## Connect keyless A keyless connection uses [Google workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation). Your project trusts an identity issuer that ACOG runs for your ACOG organization. For each exchange, ACOG signs a short-lived token for your organization, exchanges it at Google's Security Token Service, and calls Vertex AI with the resulting access token. Read [Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md) for how this works. ### 1. Get your organization's identity from ACOG In ACOG, open **Admin → Providers → Connect provider**, choose **Vertex** and the model family, and select **Keyless (workload identity)**. The console shows your issuer URL and subject, and the exact commands below with your values filled in. ### 2. Create the pool and provider in Google Cloud Replace `PROJECT` with your project ID, `NUMBER` with your project number, and `ISSUER` and `SUBJECT` with the values from the console: ```bash gcloud iam workload-identity-pools create acog \ --location=global --project=PROJECT gcloud iam workload-identity-pools providers create-oidc acog-gateway \ --location=global --workload-identity-pool=acog --project=PROJECT \ --issuer-uri="ISSUER" \ --attribute-mapping="google.subject=assertion.sub" \ --attribute-condition="assertion.sub == 'SUBJECT'" ``` The attribute condition is what limits the provider to your ACOG organization. Every ACOG organization has its own issuer as well, so another organization's tokens cannot match your provider even without the condition. ### 3. Grant access Grant the federated identity Vertex AI access directly: ```bash gcloud projects add-iam-policy-binding PROJECT \ --role=roles/aiplatform.user \ --member="principal://iam.googleapis.com/projects/NUMBER/locations/global/workloadIdentityPools/acog/subject/SUBJECT" ``` Or, if your policies require a service account, grant the service account `roles/aiplatform.user`, grant the federated identity **Workload Identity User** (`roles/iam.workloadIdentityUser`) on that service account, and enter the service account's email in the connection. ACOG then impersonates it. Optionally grant `roles/serviceusage.serviceUsageConsumer` to the same identity so ACOG can list the project's models. ### 4. Finish the connection in ACOG Enter the workload identity provider's full name: ```text projects/NUMBER/locations/global/workloadIdentityPools/acog/providers/acog-gateway ``` Enter the project ID, region and (optionally) the service account email, then save. ACOG performs an exchange and lists your models to confirm the setup. ### Removing access Disable or delete the workload identity provider (or the pool), or remove the IAM binding. Access tokens ACOG already holds expire within an hour. ## Region: where requests are processed The connection's **region** decides where Google processes your requests: | Region | Meaning | |---|---| | `global` | Google may serve the request from any region. No data residency. | | `us` | Google keeps processing inside the United States (its jurisdictional endpoint). | | `eu` | Google keeps processing inside the European Union. | | A single region, such as `us-east5` | Processing in that region. | Some models are served only in particular locations. **Model regions** pins a model to its own location, which must be inside the connection's region: `global` contains every location; `us` contains every `us-*` region; `eu` contains the regions in EU member states (not London `europe-west2` or Zurich `europe-west6`); a region in an EU member state contains the other regions in the same country; any other region contains only itself. Saving a model region outside the connection's region is refused. When Vertex AI answers that a model is not served at its location, ACOG retries it elsewhere, but only where the pin allows: | Pinned to | Retried in | |---|---| | `us` or `eu` | Nowhere else | | A `us-*` region | `us-east5`, `us-central1`, `us` | | A region in an EU member state | Only regions in the same country | | `global` | `us-east5`, `us-central1`, `europe-west1`, `europe-west4`, `asia-southeast1`, `us`, `eu` | | Any other region | Nowhere else | A location found to serve a model is remembered for 30 days. A model served nowhere it may go returns Vertex's own error, and ACOG answers that error immediately for 15 minutes before trying again. ### Keeping traffic in one country A connection's region applies to that connection only. Another connection that declares the same model can serve it elsewhere through failover, cost arbitrage, an `x-acog-channel` header or a DLP route restriction. To keep all of an organization's traffic in one country or in the EU: - make a connection pinned there your default connection; - pin every other connection that can serve the same models inside the same boundary, including the one auto mode's routing judge uses; - leave cost arbitrage off; - do not declare those models on connections that cannot be pinned (Direct, Azure OpenAI, Custom). ## Other settings | Setting | Notes | |---|---| | Project ID | The project that is billed and whose quota is used | | Endpoint override | Optional. On the hosted service, only Google's own Vertex AI hosts: `https://aiplatform.googleapis.com`, `https://aiplatform.us.rep.googleapis.com`, `https://aiplatform.eu.rep.googleapis.com` or `https://-aiplatform.googleapis.com`. Self-hosted, any HTTPS URL, such as a Private Service Connect endpoint. | | Batch storage | Optional `gs://bucket/path` where batch jobs read and write. With it, ACOG can run requests nobody is waiting for as Vertex batch jobs at about half price. The connection's identity needs read and write access to the bucket. See [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md). | | Cache lifetime | How long cached context is kept, where the model supports a choice. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-lifetime). | ## Calling Vertex AI through ACOG Call your models through the [Chat Completions](https://tessarac.com/docs/acog/api/chat-completions.md) or [Messages](https://tessarac.com/docs/acog/api/messages.md) APIs, or use Vertex AI's own API through the native route `https://gateway.acog.tessarac.com/vertex`. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Troubleshooting | What you see | Usual cause | Fix | |---|---|---| | `google refused the credentials (invalid_target)` | The provider name is wrong, or the pool or provider is disabled | Copy the provider's full resource name from Google Cloud and check that it is enabled | | `google refused the credentials (invalid_grant)` | The provider's issuer URL or attribute condition does not match your ACOG values | Compare them with the console | | `403` from Vertex AI after a successful exchange | The identity lacks `roles/aiplatform.user`, or the model is not enabled in Model Garden | Grant the role, enable the model | | `422 model_access_required` | The model is not enabled for the project | Enable it in Model Garden | | `409 connection_not_usable` | A saved region or endpoint is not allowed on the hosted service | Edit the connection to use a Vertex location and a Google host | | The model list says it is showing ACOG's catalog | The identity cannot list models | Optional: grant `roles/serviceusage.serviceUsageConsumer` | --- # AWS Bedrock > Connect AWS Bedrock to ACOG with access keys or a role ACOG assumes, including GovCloud, batch processing and native boto3 access. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/aws-bedrock. Last updated: 2026-10-16. A Bedrock connection calls models in your own AWS account: Claude and the other models Bedrock hosts. Requests are signed with AWS Signature Version 4 for the connection's region. ## Before you start - Enable the models you want under **Model access** in the Bedrock console, in the region you will use. - Choose how ACOG authenticates: | Method | What ACOG stores | Notes | |---|---|---| | **Access keys** | An access key ID and secret access key (and optionally a session token), encrypted | Simple. Rotate the keys yourself. | | **Assume role** | The role's ARN; optionally access keys used only to assume it | ACOG assumes the role with an external ID issued for your organization. No long-lived key is needed. | The identity needs `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` on the models you declare, and `bedrock:ListFoundationModels` so ACOG can list them. ## Connect with access keys 1. Create an IAM user (or use an existing one) with the permissions above and create an access key. 2. In ACOG, open **Admin → Providers → Connect provider**, choose **Bedrock**, select **Access keys**, and enter the region, access key ID and secret access key. 3. Review the models and choose a default. ## Connect with a role (assume role) When ACOG assumes a role in your account, it passes an **external ID** that ACOG issues for your organization. This is AWS's protection against the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html): someone who learns your role's ARN still cannot use it through ACOG, because only your organization's requests carry your external ID. 1. In ACOG, start a Bedrock connection and select **Assume role**. The console shows your organization's external ID and the AWS principal ACOG uses. 2. In AWS, create a role with the Bedrock permissions above and this trust policy: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "" } } } ] } ``` 3. Enter the role ARN in ACOG and save. When ACOG's own identity assumes the role, ACOG checks the trust policy before saving: it must be able to assume the role with your external ID, and it must be **refused** with no external ID and with several IDs that differ from yours. A trust policy without the `sts:ExternalId` condition, or one that matches by pattern (`StringLike`), is refused with the same message as a role ACOG cannot assume. Use `StringEquals` with exactly your ID. If you supply your own access keys on an assume-role connection, your keys assume the role, and ACOG still passes your organization's external ID. You can read your external ID at any time with `GET /v1/orgs/{org}/providers/bedrock-external-id`. It never changes. ### Connections created before external IDs were issued A role connection saved before ACOG issued external IDs keeps its old external ID (or none) until a switch date, 90 days after your organization's ID was issued. Your admins are told the date. On that date the connection switches to your organization's ID, and if the role does not require exactly that ID, the connection is suspended until you fix the trust policy and edit the connection. To switch early, allow both IDs in the trust policy (`"sts:ExternalId": ["", ""]`), then edit the connection to use the new one. ## Regions and GovCloud Enter any Bedrock region, including the AWS GovCloud (US) regions `us-gov-west-1` and `us-gov-east-1`. Signing follows the region's partition, and role and bucket ARNs in GovCloud use the `arn:aws-us-gov:` prefix. Cross-region inference profiles work: declare the profile id (for example `us.anthropic.claude-sonnet-4-5-20250929-v1:0`) as a model. ## Batch processing (optional) With a storage location, ACOG can run requests nobody is waiting for as Bedrock batch inference jobs, at about half the on-demand price. See [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md). | Setting | Value | |---|---| | Batch storage | `s3://bucket[/path]`. The bucket name may not contain dots. | | Batch role ARN | The IAM role Bedrock's `CreateModelInvocationJob` runs as, with read and write access to the bucket | The connection's identity also needs `bedrock:CreateModelInvocationJob`, `bedrock:GetModelInvocationJob` and `iam:PassRole` on the batch role, plus read and write access to the bucket. Leave all batch settings empty to turn batch processing off for the connection. ## Native Bedrock access with boto3 Code that already uses boto3 can call Bedrock through ACOG with only two changes: the endpoint URL and the credentials. ACOG verifies your request's signature, then re-signs it to Bedrock with the connection's own credentials. 1. Issue an ACOG Bedrock credential (requires the **Create and revoke API keys** permission, `admin:keys`): ```bash curl -s -X POST https://acog.tessarac.com/api/v1/orgs/$ORG_ID/bedrock-credentials \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "batch-jobs"}' ``` The response contains an `access_key_id` that begins with `ACOG` and a `secret_access_key` that is **shown only once**. 2. Point boto3 at ACOG: ```python import boto3 bedrock = boto3.client( "bedrock-runtime", region_name="us-east-1", endpoint_url="https://gateway.acog.tessarac.com/bedrock", aws_access_key_id="ACOG...", aws_secret_access_key="...", ) response = bedrock.converse( modelId="anthropic.claude-haiku-4-5-20251001-v1:0", messages=[{"role": "user", "content": [{"text": "Hello"}]}], ) print(response["output"]["message"]["content"][0]["text"]) ``` Signatures are accepted within five minutes of ACOG's clock. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Troubleshooting | What you see | Cause | Fix | |---|---|---| | `422 model_access_required` | The model is not enabled in Bedrock **Model access** for the region | Request access in the Bedrock console | | Save refused: role cannot be assumed | The trust policy does not allow ACOG's principal with exactly your external ID, or allows other IDs too | Use the trust policy above, with `StringEquals` | | The connection card says **suspended** | The role does not require your organization's external ID after the switch date | Update the trust policy, then edit and save the connection | | `403` with "request signature verification failed" on the `/bedrock` route | The signature does not match an issued ACOG credential, or the clock is more than five minutes off | Use the `ACOG…` credential and check the client's clock | --- # OpenAI > Connect the OpenAI API to ACOG with your own API key and call GPT models through either the OpenAI or the Anthropic API shape. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/openai. Last updated: 2026-10-16. A Direct OpenAI connection sends requests to the OpenAI API with your organization's own OpenAI API key. ## Connect 1. In the [OpenAI platform](https://platform.openai.com/api-keys), create an API key in the project whose limits and billing ACOG's traffic should use. A restricted key needs the Model capabilities (chat completions, embeddings) and read access to models. 2. In ACOG, open **Admin → Providers → Connect provider**, choose **Direct** and **OpenAI**, and paste the key. 3. Review the models and choose a default. ## Settings | Setting | Notes | |---|---| | Base URL | Defaults to `https://api.openai.com`. On the hosted service it must be a public HTTPS address on port 443. | | Models | The GPT models this connection serves | | Cache lifetime | How long OpenAI keeps prompt-cache entries for requests ACOG marks: in memory (the provider default) or 24 hours, where the model supports it. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-lifetime). | ## What ACOG adds to your requests ACOG sets a `prompt_cache_key` derived from each request's system prompt and tools, so requests that share the same prefix land on the same OpenAI prompt cache and are billed at the cached-input rate. It never changes your prompt. ## Calling OpenAI through ACOG - The [Chat Completions API](https://tessarac.com/docs/acog/api/chat-completions.md) at `https://gateway.acog.tessarac.com/v1`. - The [Embeddings API](https://tessarac.com/docs/acog/api/embeddings.md). - The [Messages API](https://tessarac.com/docs/acog/api/messages.md): ACOG translates Anthropic-shaped requests to OpenAI and back, so an Anthropic SDK can call a GPT model. - The OpenAI API exactly as OpenAI defines it, including the Responses API, at the native route `https://gateway.acog.tessarac.com/openai/v1`. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Troubleshooting | What you see | Cause | Fix | |---|---|---| | `422 upstream_auth_failed` | OpenAI rejected the stored key | Paste a valid key into the connection | | `422 model_access_required` | The OpenAI project cannot use the model | Check the project's model access | | `429` from OpenAI | Your OpenAI project's rate limit | Raise the limit with OpenAI, or declare the model on a second connection so ACOG can spread load | --- # Azure OpenAI > Connect an Azure OpenAI resource to ACOG, including Azure Government, and name deployments so ACOG can route to them by model. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/azure-openai. Last updated: 2026-10-16. An Azure OpenAI connection calls the deployments in your own Azure OpenAI resource. ## Name deployments after their models ACOG routes by model, and calls Azure at `{endpoint}/openai/deployments//chat/completions`. **Each deployment must be named exactly like the model it serves**, for example a deployment named `gpt-4o-mini` for the `gpt-4o-mini` model. A request for `gpt-4o-mini` then reaches that deployment. ## Connect 1. In the Azure portal, open your Azure OpenAI resource and copy its endpoint and one of its keys (**Keys and Endpoint**). 2. In ACOG, open **Admin → Providers → Connect provider**, choose **Azure** and **OpenAI**, and enter: | Field | Example | |---|---| | Endpoint | `https://my-resource.openai.azure.com` (Azure Government: `https://my-resource.openai.azure.us`) | | API version | `2024-10-21` | | API key | One of the resource's keys | 3. Declare the deployment names as the connection's models and choose a default. ACOG sends the key in the `api-key` header. ### Optional: let ACOG read which model each deployment serves A deployment's name does not prove which model it runs. To bill and report on the model a deployment actually serves, give ACOG read access to the resource through Azure Resource Manager: the tenant ID, client ID and client secret of an app registration with the **Reader** role on the resource, and the subscription, resource group and account name. ACOG only reads deployment details with it. ## Azure Government Azure Government (GCC High and DoD) resources use the `.openai.azure.us` endpoint. Nothing else changes. ## Calling Azure OpenAI through ACOG - The [Chat Completions](https://tessarac.com/docs/acog/api/chat-completions.md) and [Embeddings](https://tessarac.com/docs/acog/api/embeddings.md) APIs, with the deployment name as the model. - The [Messages API](https://tessarac.com/docs/acog/api/messages.md), translated. - The Azure OpenAI API exactly as Azure defines it at the native route `https://gateway.acog.tessarac.com/azure`, with the gateway key in the `api-key` header. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Troubleshooting | What you see | Cause | Fix | |---|---|---| | `404 DeploymentNotFound` from Azure | No deployment is named after the requested model | Create or rename the deployment to the model id | | `422 model_access_required` | The model is not deployed in the resource | Deploy it, named after the model | | `422 upstream_auth_failed` | Azure rejected the stored key | Paste a current key into the connection | --- # Custom endpoints > Connect any OpenAI-compatible server to ACOG: Ollama, vLLM, llama.cpp, DeepSeek, or a gateway of your own. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/custom-endpoints. Last updated: 2026-10-16. A custom connection sends requests to any server that implements the OpenAI Chat Completions API: a model server you run (Ollama, vLLM, llama.cpp), an inference vendor (DeepSeek and others) or another gateway. ## Connect 1. In ACOG, open **Admin → Providers → Connect provider** and choose **Custom**. 2. Enter: | Field | Notes | |---|---| | Base URL | The server's base, without `/v1`. ACOG calls `{base URL}/v1/chat/completions`. | | API key | Optional. Sent as `Authorization: Bearer`. Leave it empty for a server with no authentication. | | Server type | `generic`, `deepseek`, `ollama`, `qwen` or `vllm`. It tells ACOG how the server handles prompt caching. | | Label | Optional display name | | API version | Optional. Appended as `api-version` for servers that need it. | 3. Declare the models the server serves, exactly as it names them (for example `llama3.3:70b`), and choose a default. ## Where the server can be | Deployment | Allowed addresses | |---|---| | Hosted | A public HTTPS address on port 443. Private, LAN and loopback addresses, other ports and `http://` are refused. | | Self-hosted | Any address you configure, including `http://` and private networks | To use a server on your own network with the hosted service, publish it at a public HTTPS address you control, protected by its API key. ## Calling a custom endpoint through ACOG Use the [Chat Completions API](https://tessarac.com/docs/acog/api/chat-completions.md), or the [Messages API](https://tessarac.com/docs/acog/api/messages.md) which ACOG translates. The OpenAI-compatible [Embeddings API](https://tessarac.com/docs/acog/api/embeddings.md) works if your server implements `/v1/embeddings`. The native route is `https://gateway.acog.tessarac.com/custom`. The savings mechanisms apply to custom connections as to any other: the response cache, de-duplication and the semantic cache work regardless of the server. --- # Routing and failover > How ACOG chooses the connection that serves a request, when it spreads load and fails over, and how to pin a request to one platform. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/routing-and-failover. Last updated: 2026-10-16. ACOG routes **by model**. The `model` in a request selects the connections that declared it; you never choose a provider in the request body. ## Which connection serves a request 1. **No connection declares the model.** The request is refused with `409 model_not_configured`. Declare the model on a connection under **Admin → Providers**. 2. **Your default connection declares the model.** The default connection serves it, and only the default connection: there is no spreading and no failover to other connections. If the default answers `429` or `5xx`, that is the answer you get. While its circuit breaker is open (see below) you get `503 upstream_unavailable` with `Retry-After: 5`. 3. **Other connections declare the model.** ACOG spreads requests across them and fails over between them. When it must order them, it prefers `direct`, then `bedrock`, `vertex`, `azure` and `custom`. Model ids match loosely on version: a request for `claude-haiku-4-5` reaches a connection that declared `claude-haiku-4-5@20251001` and the other way round, and is sent with the id the connection declared. A match never crosses model families or major versions. ## Failover For a model that is not pinned to the default connection, ACOG retries on another connection that declares the same model when the first attempt: - fails with a `5xx` or a network error, - is rate-limited with `429` (retried only on a different connection), or - is refused before sending because its settings are no longer allowed (`connection_not_usable`). A request gets up to three attempts in total, with a short randomized backoff between them. Errors that are about the request itself (`400`, `401`, `403`, `404`, `422`) are returned at once. ## Circuit breaker After five consecutive failures on a platform, ACOG stops sending to it for 30 seconds and then lets one request through to test it. While the breaker is open, requests that cannot go elsewhere receive `503 upstream_unavailable` with `Retry-After: 5`. ## What can move a request off the default connection Four things can send a model your default connection declares to another connection, which then spreads and fails over normally: | Cause | When | |---|---| | [Cost arbitrage](https://tessarac.com/docs/acog/optimize/cost-arbitrage.md) | You turned it on, and another platform serves the same model for less | | The `x-acog-channel` header | The request pins a platform (below) | | A [DLP route restriction](https://tessarac.com/docs/acog/govern/data-loss-prevention.md) | A rule allows the request only on certain platforms | | The policy-block fallback | The provider's safety system refused the request (below) | ## Pin a request to one platform: `x-acog-channel` When the same model is declared on several platforms (Claude on Anthropic, Bedrock and Vertex, say), send `x-acog-channel` to choose one: ```bash curl -s https://gateway.acog.tessarac.com/v1/messages \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "x-acog-channel: vertex" \ -H "anthropic-version: 2023-06-01" \ -H "Content-Type: application/json" \ -d '{"model": "claude-sonnet-4-5", "max_tokens": 256, "messages": [{"role": "user", "content": "Hello"}]}' ``` Values are `direct`, `bedrock`, `vertex`, `azure` and `custom`. The pin overrides everything else, including cost arbitrage. An unknown value, or a platform with no connection that declares the model, is refused with `409 model_not_configured`. ## Policy-block fallback Occasionally a provider's safety classifier refuses a harmless request with an empty answer and `stop_reason: "refusal"`. When that happens, ACOG sends the request once more to a fallback model (by default `claude-opus-4-8`) on the same connection, adds a short routing note to the answer saying so, and sets the response header `x-acog-policy-fallback`: | Value | Meaning | |---|---| | `applied` | The fallback model answered | | `skipped` | The request was already for the fallback model, so there was nothing to fall back to | | `unavailable` | No fallback answer: the request opted out, the fallback was not allowed (by model access rules, say), or the fallback model also declined or failed. You get the original refusal. | The header is present only when a refusal was seen. To opt out for a request, send `x-acog-policy-fallback: off`. ## Which model answered Unified routes set `x-acog-resolved-model` to the model that was actually sent, which differs from the request when you asked for `auto` or a version alias. See [Response headers](https://tessarac.com/docs/acog/reference/response-headers.md). --- # Base URLs and authentication > The gateway's base URLs for each SDK, the headers ACOG reads your gateway key from, request limits and request IDs. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/base-urls-and-authentication. Last updated: 2026-10-16. ## Base URLs | Deployment | Gateway | Management API | |---|---|---| | Hosted | `https://gateway.acog.tessarac.com` | `https://acog.tessarac.com/api` | | Self-hosted | `https:///gateway` | `https:///api` | On the hosted service the gateway has its own address, with nothing between your application and ACOG but a load balancer. The console's address also serves the gateway, at `https://acog.tessarac.com/gateway`, and integrations made before October 2026 may use it. That path goes through the edge network in front of the console: a request that sends nothing back for 60 seconds is closed with `504`, and some errors arrive as an HTML page. Move to `gateway.acog.tessarac.com`; nothing else changes, and the same keys work on both. What you configure depends on the path your SDK appends: | Client | Base URL to configure | Why | |---|---|---| | OpenAI SDKs and OpenAI-compatible tools | `https://gateway.acog.tessarac.com/v1` | The OpenAI client appends `/chat/completions` to a base that already includes `/v1` | | Anthropic SDKs and Anthropic-compatible tools | `https://gateway.acog.tessarac.com` | The Anthropic client appends `/v1/messages` itself | | Native provider SDKs | `https://gateway.acog.tessarac.com/` | See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md) | The two shapes are not tied to providers: ACOG translates between them, so an OpenAI SDK can call a Claude model and an Anthropic SDK can call a GPT or Gemini model. ## Endpoints | Method | Path (after the gateway base) | Page | |---|---|---| | `POST` | `/v1/chat/completions` | [Chat Completions](https://tessarac.com/docs/acog/api/chat-completions.md) | | `POST` | `/v1/messages` | [Messages](https://tessarac.com/docs/acog/api/messages.md) | | `POST` | `/responses` | [Responses](https://tessarac.com/docs/acog/api/responses.md) | | `POST` | `/v1/embeddings` | [Embeddings](https://tessarac.com/docs/acog/api/embeddings.md) | | `GET` | `/v1/models` | [Models](https://tessarac.com/docs/acog/api/models.md) | | `GET` | `/v1/acog/batches/{id}` | [Batches](https://tessarac.com/docs/acog/api/batches.md) | | any | `/anthropic/…`, `/openai/…`, `/vertex/…`, `/bedrock/…`, `/azure/…`, `/custom/…` | [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md) | ## Authentication Every request carries a gateway key (`acog_…`). Create one on the **Developer** page; see [API keys](https://tessarac.com/docs/acog/administer/api-keys.md). On the compatible endpoints (`/v1/…` and `/responses`), ACOG reads the key from the first of these that is present: 1. `Authorization: Bearer acog_…` 2. `x-acog-key: acog_…` 3. `x-api-key` or `api-key`, but only when the value starts with `acog_` So with the Anthropic SDK, pass the gateway key as `auth_token` (Python) or `authToken` (TypeScript), which is sent as a bearer token, rather than as `api_key`. A provider key in `x-api-key` is never treated as a gateway key. Native routes read the key from the header their provider's SDK uses. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md#authentication). ### Your provider credentials are never forwarded ACOG removes `x-api-key`, `api-key` and any `Authorization` header that is not a gateway key before forwarding a request, and authenticates to your provider with your connection's own credentials. ### What a failed authentication looks like | Status | `error.type` | Meaning | |---|---|---| | `401` | `unauthorized` | No key was found, or the key is unknown, disabled, expired, or used from an address outside its IP allowlist. The message does not say which. | | `403` | `forbidden` | The key is valid but lacks the gateway access scope (`proxy`). | | `503` | `service_unavailable` | ACOG could not reach its key store. Retry. | A disabled key stops working within about five seconds. ## Request limits | Limit | Value | |---|---| | Request body | 64 MiB. Larger requests are refused with `413 request_too_large`. | | Waiting for a non-streaming reply | Up to 10 minutes without data on `gateway.acog.tessarac.com`; 60 seconds on `acog.tessarac.com/gateway`. Self-hosted, 2 minutes unless your operator sets `ACOG_UPSTREAM_READ_TIMEOUT_SECS`, and no longer than your load balancer allows. | | Waiting between streamed chunks | The same limit applies to silence from the provider. ACOG sends a keep-alive comment every 15 seconds, so a stream stays open for as long as the model keeps working. | Rate limits and budgets are your organization's own. See [Rate limits](https://tessarac.com/docs/acog/govern/rate-limits.md) and [Budgets](https://tessarac.com/docs/acog/govern/budgets.md). ## Request IDs Every response carries `x-request-id`. Send your own `x-request-id` and ACOG uses it; otherwise it generates one. Quote it when you contact support. ## Useful request headers | Header | Effect | Page | |---|---|---| | `x-acog-channel` | Pin the request to one platform | [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md#pin-a-request-to-one-platform-x-acog-channel) | | `x-acog-batch` | Run the request at batch prices | [Batches](https://tessarac.com/docs/acog/api/batches.md) | | `x-acog-cache-control` | Control the response cache | [Caching](https://tessarac.com/docs/acog/optimize/caching.md#controlling-the-response-cache-per-request) | | `x-acog-label` | Tag the request for reports (a ticket number, a cost center) | [Request headers](https://tessarac.com/docs/acog/reference/request-headers.md) | The full list is in [Request headers](https://tessarac.com/docs/acog/reference/request-headers.md). --- # Chat Completions (OpenAI-compatible) > POST /v1/chat/completions: the OpenAI Chat Completions API, served by any model you have connected, with tools, images and streaming. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/chat-completions. Last updated: 2026-10-16. ```http POST https://gateway.acog.tessarac.com/v1/chat/completions ``` The OpenAI [Chat Completions API](https://platform.openai.com/docs/api-reference/chat). Send exactly what you would send to OpenAI; ACOG routes by `model` to whichever of your connections declares it, translating to and from the Anthropic and Gemini APIs when the model is a Claude or Gemini model. ## Example ```python import os from openai import OpenAI client = OpenAI( base_url="https://gateway.acog.tessarac.com/v1", api_key=os.environ["ACOG_API_KEY"], ) completion = client.chat.completions.create( model="claude-haiku-4-5", messages=[ {"role": "system", "content": "You are terse."}, {"role": "user", "content": "Name three prime numbers."}, ], ) print(completion.choices[0].message.content) ``` ```bash curl -s https://gateway.acog.tessarac.com/v1/chat/completions \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "gemini-3.5-flash", "messages": [{"role": "user", "content": "Name three prime numbers."}]}' ``` ## What is supported | Feature | Supported | |---|---| | System, user, assistant and tool messages | Yes | | Tools and function calling, tool results | Yes | | Images in user messages (`image_url`, URL or base64) | Yes. For Bedrock and Vertex, ACOG fetches image URLs and sends them inline (up to 5 MB, 10-second fetch). A failed fetch is a `400`, never a silent drop. | | Streaming (`stream: true`) | Yes. See [Streaming](https://tessarac.com/docs/acog/api/streaming.md). | | `model: "auto"` | Yes. See [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md). | | `:batch` | Run at batch prices. See [Batches](https://tessarac.com/docs/acog/api/batches.md). | | Audio output | Forwarded to OpenAI-compatible connections; never cached | ## Translation to other providers When the model is served by a connection that speaks a different API (Claude on Anthropic, Bedrock or Vertex; Gemini on Vertex), ACOG translates the request and the response, including streamed responses. Translation is strict by default. A field that has no faithful equivalent in the target API is refused with `422 translation_unsupported` rather than silently dropped, so you never get an answer to a different question than the one you asked. An admin can list fields a connection may drop instead. Translated requests are not stored in the response cache, because the stored reply would be in the target's format. ## The response The response is an OpenAI chat completion object. `model` names the model that answered. The header `x-acog-resolved-model` carries it too, which is useful when you sent `auto`. ACOG adds a few response headers of its own, for example `x-acog-cache: hit` when the reply came from a cache. See [Response headers](https://tessarac.com/docs/acog/reference/response-headers.md). ## Errors Errors use the envelope `{"error": {"type": "…", "message": "…"}}`. An error your provider returned is passed through unchanged. See [Errors](https://tessarac.com/docs/acog/reference/errors.md). --- # Messages (Anthropic-compatible) > POST /v1/messages: the Anthropic Messages API, served by any model you have connected, with tools, images, extended thinking and streaming. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/messages. Last updated: 2026-10-16. ```http POST https://gateway.acog.tessarac.com/v1/messages ``` The Anthropic [Messages API](https://docs.claude.com/en/api/messages). Send exactly what you would send to Anthropic, including the `anthropic-version` header. ACOG routes by `model`: a Claude model goes to whichever of your Anthropic, Bedrock or Vertex connections declares it, and a GPT or Gemini model is translated. ## Example ```python import os from anthropic import Anthropic client = Anthropic( base_url="https://gateway.acog.tessarac.com", # The gateway key is sent as Authorization: Bearer. Use auth_token, not api_key. auth_token=os.environ["ACOG_API_KEY"], ) message = client.messages.create( model="claude-sonnet-4-5", max_tokens=512, messages=[{"role": "user", "content": "Explain prompt caching in one paragraph."}], ) print(message.content[0].text) ``` ```typescript import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic({ baseURL: "https://gateway.acog.tessarac.com", authToken: process.env.ACOG_API_KEY, }); const message = await client.messages.create({ model: "claude-sonnet-4-5", max_tokens: 512, messages: [{ role: "user", content: "Explain prompt caching in one paragraph." }], }); ``` ```bash curl -s https://gateway.acog.tessarac.com/v1/messages \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "anthropic-version: 2023-06-01" \ -H "Content-Type: application/json" \ -d '{"model": "claude-sonnet-4-5", "max_tokens": 512, "messages": [{"role": "user", "content": "Hello"}]}' ``` If your code sets `ANTHROPIC_API_KEY` in the environment, the Anthropic SDK sends it as `x-api-key`. ACOG ignores a provider key there, so the request fails with `401`. Unset it, or pass the gateway key explicitly as shown. ## What is supported | Feature | Supported | |---|---| | System prompts, multi-turn conversations | Yes | | Tools, tool use and tool results | Yes | | Images and documents (base64 or URL) | Yes. For Bedrock and Vertex, ACOG fetches URL sources and sends them inline (up to 5 MB, 10-second fetch). | | Prompt caching (`cache_control`) | Yes. ACOG also adds cache breakpoints you did not set; see [Caching](https://tessarac.com/docs/acog/optimize/caching.md#provider-prompt-caching). | | Extended thinking | Yes. If the serving model rejects it, ACOG retries without it and sets `x-acog-thinking-stripped: true`. | | `anthropic-beta` | Forwarded. If the serving platform rejects a beta value, ACOG retries without it and names it in `x-acog-beta-stripped`. | | Streaming | Yes. See [Streaming](https://tessarac.com/docs/acog/api/streaming.md). | | `model: "auto"` and `:batch` | Yes. See [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md) and [Batches](https://tessarac.com/docs/acog/api/batches.md). | Token counting (`/v1/messages/count_tokens`), Message Batches (`/v1/messages/batches`) and the Files API are not part of the compatible route. Call them on the native route, `https://gateway.acog.tessarac.com/anthropic`. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). ## Translation to other providers A GPT model (OpenAI, Azure OpenAI or a custom endpoint) or a Gemini model is translated from and to the Messages format, including streamed responses. Translation is strict: a field with no faithful equivalent is refused with `422 translation_unsupported`. ## The response The response is an Anthropic message object. The header `x-acog-resolved-model` names the model that answered. When a budget passes 50, 80 or 90% of its limit, ACOG appends a one-time notice after the model's reply and sets `x-acog-budget-notice`. See [Budgets](https://tessarac.com/docs/acog/govern/budgets.md#warnings-in-responses). --- # Responses > POST /responses: a minimal OpenAI Responses API bridge for simple prompts on any model. Use the native OpenAI route for the full Responses API. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/responses. Last updated: 2026-10-16. ```http POST https://gateway.acog.tessarac.com/responses ``` A minimal bridge for code that uses the OpenAI Responses API for simple, single-turn prompts. It runs on any model you have connected: ACOG turns the request into a chat completion, routes it like any other, and returns a Responses-style result. Note the path: `/responses`, not `/v1/responses`. With the OpenAI SDK, set the base URL to `https://gateway.acog.tessarac.com` (without `/v1`) for this endpoint. ## Request | Field | Notes | |---|---| | `model` | Required. Any declared model, or `auto`. | | `instructions` | Optional. Sent as a system message. | | `input` | A string, or an array of input parts. Text and image parts are supported. | Other Responses API features (tools, structured outputs, streaming, stored conversation state) are not part of the bridge. ```bash curl -s https://gateway.acog.tessarac.com/responses \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "gemini-3.5-flash", "input": "Name three prime numbers."}' ``` ## Response ```json { "id": "5THSauPYOZSB0ekPo4SkwAk", "object": "response", "model": "gemini-3.5-flash", "output_text": "2, 3 and 5." } ``` An error from the underlying chat completion is returned as is. ## The full Responses API To use the Responses API exactly as OpenAI defines it, with tools, streaming and state, call OpenAI models through the native route: ```python import os from openai import OpenAI client = OpenAI( base_url="https://gateway.acog.tessarac.com/openai/v1", api_key=os.environ["ACOG_API_KEY"], ) response = client.responses.create(model="gpt-4.1-mini", input="Hello") ``` See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md). --- # Embeddings > POST /v1/embeddings: the OpenAI Embeddings API through ACOG, for embedding models on OpenAI, Azure OpenAI and OpenAI-compatible connections. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/embeddings. Last updated: 2026-10-16. ```http POST https://gateway.acog.tessarac.com/v1/embeddings ``` The OpenAI [Embeddings API](https://platform.openai.com/docs/api-reference/embeddings). ACOG routes by `model` like any other request. ```python import os from openai import OpenAI client = OpenAI( base_url="https://gateway.acog.tessarac.com/v1", api_key=os.environ["ACOG_API_KEY"], ) result = client.embeddings.create(model="text-embedding-3-small", input=["first text", "second text"]) print(len(result.data[0].embedding)) ``` ## Which connections can serve embeddings Embeddings are served only by connections that speak the OpenAI API: Direct OpenAI, Azure OpenAI (with a deployment named after the embedding model) and custom OpenAI-compatible endpoints that implement `/v1/embeddings`. If your default connection cannot embed (for example it is an Anthropic connection) and an OpenAI-compatible connection declares the model, that connection serves the request. If no connection that declares the model can embed, the request is refused with `422 embeddings_unsupported_shape`. ## Governance API keys, model access rules, budgets and rate limits apply to embeddings as to any other request. Data loss prevention and prompt injection screening do not run on embeddings requests. --- # Models > GET /v1/models: list the models your gateway key can use. It makes no provider call and costs nothing, so it is also the quickest way to test a key. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/models. Last updated: 2026-10-16. ```http GET https://gateway.acog.tessarac.com/v1/models ``` Lists every model declared on your organization's connections, in the OpenAI list format. ACOG answers from your configuration without calling any provider, so the call is free and fast. It is the recommended way to check a new key. ```bash curl -s https://gateway.acog.tessarac.com/v1/models \ -H "Authorization: Bearer $ACOG_API_KEY" ``` ```json { "object": "list", "data": [ { "id": "claude-haiku-4-5-20251001", "object": "model", "created": 0, "owned_by": "anthropic" }, { "id": "gemini-3.5-flash", "object": "model", "created": 0, "owned_by": "gemini" } ] } ``` | Field | Meaning | |---|---| | `id` | A model id you can send in `model` | | `owned_by` | The model family of the first connection that declares it | | `created` | Always `0`; ACOG does not track release dates | An empty `data` list means your organization has no connections yet. The list shows declared models: whether your provider account can actually call each one is decided by the provider. The list does not apply model access rules, so it can include models a [model access](https://tessarac.com/docs/acog/govern/model-access.md) rule would refuse for this key. | Status | Meaning | |---|---| | `200` | The key works | | `401` | The key was not read: missing, revoked, or in a header ACOG does not read | | `403` | The key lacks the gateway access scope (`proxy`) | | `503` | ACOG could not reach its key store; retry | To list what a provider itself offers, use the native route, for example `GET https://gateway.acog.tessarac.com/anthropic/v1/models`. --- # Native provider routes > Use each provider's own API and SDK through ACOG, byte for byte: Anthropic, OpenAI, Vertex AI, Bedrock, Azure OpenAI and custom endpoints. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/native-routes. Last updated: 2026-10-16. The compatible endpoints translate between API shapes. The native routes do not: each forwards your request to one provider in that provider's own format, with your connection's credentials in place of the gateway key. Use them when you need a provider feature the compatible endpoints do not cover (token counting, Message Batches, the OpenAI Responses API, Bedrock's Converse API), or to keep existing provider-SDK code unchanged apart from its base URL. ## Routes | Route | Forwards to | Base URL for the provider's SDK | |---|---|---| | `/anthropic/…` | Your Direct Anthropic connection | `https://gateway.acog.tessarac.com/anthropic` | | `/openai/…` | Your Direct OpenAI connection | `https://gateway.acog.tessarac.com/openai/v1` | | `/vertex/…` | Your Vertex AI connection for the model in the path | `https://gateway.acog.tessarac.com/vertex` | | `/bedrock/…` | Your Bedrock connection | `https://gateway.acog.tessarac.com/bedrock` | | `/azure/…` | Your Azure OpenAI connection | `https://gateway.acog.tessarac.com/azure` | | `/custom/…` | Your custom connection | `https://gateway.acog.tessarac.com/custom` | The path after the prefix is the provider's own path. A request to a route with no matching connection is refused with `409 connection_not_configured`. ## Authentication Each route reads the gateway key from the header that provider's SDK already sends, so you only swap the key: | Route | Header | |---|---| | `/anthropic` | `x-api-key: acog_…` (a bearer token is not read on this route) | | `/openai`, `/vertex`, `/custom` | `Authorization: Bearer acog_…` | | `/azure` | `api-key: acog_…` | | `/bedrock` | An AWS Signature Version 4 signature made with an issued ACOG Bedrock credential. See [AWS Bedrock](https://tessarac.com/docs/acog/providers/aws-bedrock.md#native-bedrock-access-with-boto3). | Every route also accepts `x-acog-key: acog_…`. ## Examples **Anthropic SDK, including token counting and Message Batches** ```python import os from anthropic import Anthropic client = Anthropic( base_url="https://gateway.acog.tessarac.com/anthropic", api_key=os.environ["ACOG_API_KEY"], # sent as x-api-key, which this route reads ) count = client.messages.count_tokens( model="claude-haiku-4-5", messages=[{"role": "user", "content": "Hello"}], ) batch = client.messages.batches.create( requests=[{ "custom_id": "first", "params": {"model": "claude-haiku-4-5", "max_tokens": 64, "messages": [{"role": "user", "content": "Hello"}]}, }] ) ``` **OpenAI SDK, Responses API** ```python import os from openai import OpenAI client = OpenAI(base_url="https://gateway.acog.tessarac.com/openai/v1", api_key=os.environ["ACOG_API_KEY"]) response = client.responses.create(model="gpt-4.1-mini", input="Hello") ``` **Vertex AI, Gemini `generateContent`** ```bash curl -s "https://gateway.acog.tessarac.com/vertex/v1/projects/my-project/locations/us-central1/publishers/google/models/gemini-3.1-flash-lite:generateContent" \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"contents": [{"role": "user", "parts": [{"text": "Hello"}]}]}' ``` **Azure OpenAI SDK** ```python import os from openai import AzureOpenAI client = AzureOpenAI( base_url="https://gateway.acog.tessarac.com/azure/openai", api_key=os.environ["ACOG_API_KEY"], api_version="2024-10-21", ) completion = client.chat.completions.create( model="gpt-4o-mini", # the deployment name messages=[{"role": "user", "content": "Hello"}], ) ``` ## What ACOG does on a native route | | Native routes | |---|---| | Request body | Forwarded as sent, including provider-only fields. Only provider prompt-cache markers, [prompt cleanup](https://tessarac.com/docs/acog/optimize/request-cleanup.md) and data loss prevention masking can change it. | | Provider headers (`anthropic-version`, `anthropic-beta`, your own tracing headers) | Forwarded | | Your gateway key and any provider credential you sent | Removed; the connection's credentials are used | | Response | Returned as the provider sent it, including errors | | Model access rules, budgets, rate limits | Applied | | Data loss prevention and prompt injection screening | Applied to request bodies ACOG can read | | Response cache | Identical non-streaming requests are answered from cache | | Provider prompt caching | Added on Direct Anthropic and OpenAI requests | | Translation, auto mode, failover, `x-acog-batch` | Not available. Use the compatible endpoints for these. | | `x-acog-resolved-model` header | Not set; the provider's response names the model | ### Model access on native routes ACOG needs to know the model to apply model access rules. When a rule restricts models and ACOG cannot tell the model from the request (an opaque or multipart body), the request is refused with `403 model_unverifiable`. ### Vertex AI locations A native Vertex request goes to the location configured for its model on your connection (its model region, or the connection's region), whatever location the URL names. It is not retried in other locations. See [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md#region-where-requests-are-processed). --- # Streaming > Stream responses through ACOG with server-sent events, on every route and across API shapes, and what to expect from timeouts and keep-alives. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/streaming. Last updated: 2026-10-16. Set `stream: true` (or use your SDK's streaming method) on any route, compatible or native. ACOG relays the stream as it arrives, as server-sent events (`text/event-stream`). ```python import os from anthropic import Anthropic client = Anthropic(base_url="https://gateway.acog.tessarac.com", auth_token=os.environ["ACOG_API_KEY"]) with client.messages.stream( model="claude-sonnet-4-5", max_tokens=1024, messages=[{"role": "user", "content": "Write a haiku about caching."}], ) as stream: for text in stream.text_stream: print(text, end="", flush=True) ``` ## Streams across API shapes A streamed request is translated like any other. An OpenAI-shaped streaming request for a Claude or Gemini model receives OpenAI `chat.completion.chunk` events ending with `data: [DONE]`. An Anthropic-shaped streaming request for a GPT or Gemini model receives Anthropic events ending with `message_stop`. Bedrock's binary event stream is converted to server-sent events on the compatible routes. Token usage is read from the stream itself, so streamed requests are measured and billed exactly like non-streamed ones. ## Keep-alives and timeouts | | | |---|---| | Keep-alive | Every 15 seconds without data, ACOG sends an SSE comment line (`: keep-alive`). SDKs ignore it. It stops proxies and load balancers from closing a stream while the model is thinking. | | Provider silence | If the provider sends nothing for 10 minutes (hosted) or 2 minutes (self-hosted, unless your operator sets `ACOG_UPSTREAM_READ_TIMEOUT_SECS`), ACOG ends the request. There is no limit on a stream's total length. | **Prefer streaming for long generations.** A non-streaming request sends nothing until the whole answer is ready. On `gateway.acog.tessarac.com` it may wait up to 10 minutes; on the console's `acog.tessarac.com/gateway` it is closed after 60 seconds without data (`504`). A streamed request shows its first tokens at once and is kept open by its keep-alives for as long as it takes. ## What changes when you stream - Streamed responses are never stored in the response cache or the semantic cache, and identical streamed requests are not de-duplicated, because a stream cannot be replayed. - Batch routing never applies to a streamed request. - A budget warning is appended as a final text event before the stream ends; see [Budgets](https://tessarac.com/docs/acog/govern/budgets.md#warnings-in-responses). - Data loss prevention scans the request before it is sent. Streamed replies are not scanned. --- # Batches > Run requests on your provider's batch tier at about half price with one header or a model suffix, and collect the result inline or by polling a job. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/batches. Last updated: 2026-10-16. Every major provider runs a batch tier: the same model and the same answer, at about half the price, in exchange for waiting. ACOG lets you send any single request to that tier without learning the provider's batch API. You opt in per request; ACOG submits the batch job, waits for it, and gives you an ordinary response. ACOG can also decide this for you on requests nobody is waiting for. See [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md). ## Opt in Either add a header: ```http x-acog-batch: defer; max-wait=2h ``` or add `:batch` to the model, for clients that cannot set headers: ```json { "model": "claude-haiku-4-5:batch", "max_tokens": 256, "messages": [ … ] } ``` | Setting | Meaning | |---|---| | `defer` | Required in the header: run this request on the batch tier | | `max-wait=` | How long you can wait for the answer, for example `45s`, `30m` or `2h`. Default `1h`. A value ACOG cannot read falls back to the default. | The `:batch` suffix uses the default wait. If you send both, the header wins. ## How you get the answer | Wait | Delivery | |---|---| | Within the hold window (50 seconds on the hosted service; 10 minutes by default on a self-hosted installation) | ACOG keeps your connection open and returns the answer on it, as a normal response | | Longer | ACOG returns `202 Accepted` at once with a job handle, and you poll for the answer | The job handle: ```json { "id": "acogbatch_18df08b9e2364f83f2825dbe0c564531926d21a9046fcf58", "object": "acog.batch", "status": "in_progress", "model": "claude-haiku-4-5-20251001", "created_at_ms": 1792160771173, "deadline_secs": 7200 } ``` ## Poll for the result ```http GET https://gateway.acog.tessarac.com/v1/acog/batches/{id} ``` ```bash curl -s https://gateway.acog.tessarac.com/v1/acog/batches/acogbatch_18df08b9e2364f83f2825dbe0c564531926d21a9046fcf58 \ -H "Authorization: Bearer $ACOG_API_KEY" ``` | Response | Meaning | |---|---| | `200` with the job handle (`status: "in_progress"`) | Still running. Poll again later; once a minute is plenty. | | `200` with the model's response | Done. The body is the response you would have received synchronously, in the shape you asked in. | | `404 batch_not_found` | No such job for your organization | | `502 batch_failed` | The provider's batch job failed | A job is visible only to keys of the organization that created it. ## Which connections support it | Connection | Batch tier | |---|---| | Direct Anthropic | Message Batches. Works with API-key and keyless connections. | | Direct OpenAI | Batch API | | AWS Bedrock | Batch inference jobs, when the connection has batch storage configured. Always delivered as a job handle. | | Google Vertex AI | Batch prediction jobs, when the connection has batch storage configured. Always delivered as a job handle. | | Azure OpenAI, custom endpoints | None. The request runs synchronously at the normal price. | A request ACOG cannot run on the batch tier (a streaming request, a connection with no batch tier, a request using a feature the batch API does not accept) is answered synchronously, at the normal price, rather than refused. ## Pricing A batched request is priced at the provider's batch rate, roughly half the synchronous rate, and is reported with the disposition **Batch**. ACOG checks first that the batch tier really is cheaper for your request: a request whose prompt would mostly be read from the provider's prompt cache can be cheaper synchronously, and is then sent synchronously. ## Provider batch APIs directly To use a provider's own batch API (for example to submit thousands of requests in one Anthropic Message Batch), call it on the [native route](https://tessarac.com/docs/acog/api/native-routes.md), such as `POST https://gateway.acog.tessarac.com/anthropic/v1/messages/batches`. --- # Management API > Automate ACOG administration with the REST API behind the console: connections, keys, budgets, webhooks, usage and savings. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/api/management-api. Last updated: 2026-10-16. Everything the console does, it does through the management API, and a gateway key with the right scopes can do the same. ## Base URL and authentication | Deployment | Base URL | |---|---| | Hosted | `https://acog.tessarac.com/api` | | Self-hosted | `https:///api` | Authenticate with a gateway key whose scopes include the permission each endpoint needs: ```bash curl -s https://acog.tessarac.com/api/v1/orgs/$ORG_ID/providers \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" ``` `x-acog-key` works too. A key's scopes are permissions (see [Roles and permissions](https://tessarac.com/docs/acog/reference/roles-and-permissions.md)); a key can never hold more than the person who created it, and seeing the Activity page's conversations (`audit:conversations`) is never available to a key. Your organization ID (`org_…`) is on **Admin → Organization**. Routes under `/v1/orgs/{org}` act only on your own organization: another organization's ID answers `404`, as if it did not exist. Requests and responses are JSON. Creating returns the created object; deleting returns `204 No Content`. Errors use standard status codes: `400` for an invalid body, `401` without a valid principal, `403` without the permission (the message names the permission), `404` for not found, `409` for a conflict. ## Endpoints ### Provider connections | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/providers` | `admin:providers` | | `POST` | `/v1/orgs/{org}/providers` | `admin:providers` | | `DELETE` | `/v1/orgs/{org}/providers/{id}` | `admin:providers` | | `GET` | `/v1/orgs/{org}/providers/bedrock-external-id` | `admin:providers` | | `GET` | `/v1/orgs/{org}/providers/workload-identity` | `admin:providers` | | `GET` | `/v1/orgs/{org}/channels` | `proxy` | A connection is created with: ```json { "channel": "vertex", "provider": "gemini", "display_name": "Vertex Gemini", "public": { "project_id": "my-project", "region": "us-central1", "auth_mode": "workload_identity", "workload_identity_provider": "projects/123456789/locations/global/workloadIdentityPools/acog/providers/acog-gateway", "models": ["gemini-3.5-flash", "gemini-3.1-flash-lite"], "default_model": "gemini-3.5-flash" }, "secret": {} } ``` `public` is stored and readable; `secret` is encrypted, write-only and never returned. Each provider page lists its fields. Posting a channel and family that already has a connection replaces it. `GET …/providers/workload-identity` returns your organization's issuer URL, subject and key set for [keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md). ### API keys and Bedrock credentials | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/api-keys` | any principal (your own keys; all keys with `admin:keys`) | | `POST` | `/v1/orgs/{org}/api-keys` | any principal | | `DELETE` | `/v1/orgs/{org}/api-keys/{id}` | any principal (your own keys; any key with `admin:keys`) | | `GET`, `POST` | `/v1/orgs/{org}/bedrock-credentials` | `admin:keys` | | `DELETE` | `/v1/orgs/{org}/bedrock-credentials/{id}` | `admin:keys` | ```bash curl -s -X POST https://acog.tessarac.com/api/v1/orgs/$ORG_ID/api-keys \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "billing-service", "scopes": ["proxy"], "allowed_cidrs": ["203.0.113.0/24"]}' ``` The response's `key` field is the only time the key is shown. `DELETE` disables an active key; `DELETE …/api-keys/{id}?purge=true` then removes the disabled key permanently. See [API keys](https://tessarac.com/docs/acog/administer/api-keys.md). ### Users, roles and invitations | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/users` | `read:metrics` | | `PUT`, `DELETE` | `/v1/orgs/{org}/users/{id}` | `admin:users` | | `GET`, `POST` | `/v1/orgs/{org}/roles` | `admin:users` | | `PUT`, `DELETE` | `/v1/orgs/{org}/roles/{id}` | `admin:users` | | `GET`, `POST` | `/v1/orgs/{org}/invites` | `admin:users` (hosted) | | `DELETE` | `/v1/orgs/{org}/invites/{id}` | `admin:users` (hosted) | | `POST` | `/v1/orgs/{org}/users` | `admin:users` (self-hosted: returns a one-time initial password) | ### Budgets | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/budgets` | `read:billing` | | `PUT` | `/v1/orgs/{org}/budgets/{scope}/{entityID}` | `admin:budgets` | | `DELETE` | `/v1/orgs/{org}/budgets/{scope}/{entityID}` | `admin:budgets` | | `GET` | `/v1/budgets/usage` | `read:billing` | ```bash curl -s -X PUT https://acog.tessarac.com/api/v1/orgs/$ORG_ID/budgets/org/$ORG_ID \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" \ -H "Content-Type: application/json" \ -d '{"limit_usd": 5000, "window": "month"}' ``` `scope` is `org`, `group` (a team) or `user`, and `entityID` is the organization, team or user ID. See [Budgets](https://tessarac.com/docs/acog/govern/budgets.md). ### Webhooks | Method | Path | Permission | |---|---|---| | `GET` | `/v1/webhook-events` | any principal | | `GET`, `POST` | `/v1/orgs/{org}/webhooks` | `admin:org` | | `PUT`, `DELETE` | `/v1/orgs/{org}/webhooks/{id}` | `admin:org` | | `POST` | `/v1/orgs/{org}/webhooks/{id}/rotate-secret` | `admin:org` | | `POST` | `/v1/orgs/{org}/webhooks/{id}/ping` | `admin:org` | | `GET` | `/v1/orgs/{org}/webhooks/{id}/deliveries` | `admin:org` | See [Webhooks](https://tessarac.com/docs/acog/administer/webhooks.md) for events, payloads and signature verification. ### Usage and savings | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/dashboard/summary?window=30d` | `read:metrics` | | `GET` | `/v1/orgs/{org}/dashboard/timeseries?window=30d&bucket=day` | `read:metrics` | | `GET` | `/v1/orgs/{org}/dashboard/breakdown?dimension=model&window=30d` | `read:metrics` | | Field | Meaning | |---|---| | `request_count` | Requests in the window | | `spent_usd` | What the traffic cost through ACOG | | `would_have_spent_usd` | The same traffic at your providers' list prices with no optimization | | `saved_usd` | The sum of each request's saving | | `gainshare_usd`, `net_saved_usd` | The share billed on the saving (hosted), and the saving after it | | `savings_pct` | The saving as a share of `would_have_spent_usd` | | `by_disposition` | Requests and saving per [disposition](https://tessarac.com/docs/acog/get-started/key-concepts.md#request-outcome-disposition) | `breakdown` takes `dimension` = `model`, `channel`, `endpoint`, `key_id` or `user`. ### Billing (read only) | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}/billing` | `read:billing` | | `GET` | `/v1/orgs/{org}/billing/invoices` | `read:billing` | ### Organization and settings | Method | Path | Permission | |---|---|---| | `GET` | `/v1/orgs/{org}` | `read:metrics` | | `PUT` | `/v1/orgs/{org}` | `admin:org` | | `GET`, `PUT` | `/v1/orgs/{org}/settings` | `admin:org` | ### Licence (self-hosted) | Method | Path | Permission | |---|---|---| | `GET` | `/v1/license` | `read:billing` | | `POST` | `/v1/license/load` | `admin:org` | | `DELETE` | `/v1/license` | `admin:org` | | `GET` | `/v1/license/usage-export` | `admin:org` | See [Licensing](https://tessarac.com/docs/acog/self-hosted/licensing.md). --- # Auto mode > Send model: auto and ACOG grades each request and routes it to the cheapest model strong enough to answer it, across every provider you have connected. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/auto-mode. Last updated: 2026-10-16. Most requests do not need your most capable model. Auto mode grades how hard each request is and sends it to the cheapest model that can handle it. It applies only to requests that ask for it with `"model": "auto"`; a request that names a model is answered by that model, except in two documented cases you can switch off: the [semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md) can reuse an equally or more capable model's stored answer, and the [policy-block fallback](https://tessarac.com/docs/acog/providers/routing-and-failover.md#policy-block-fallback) retries a refused request once on another model (switched off per request). ```bash curl -s https://gateway.acog.tessarac.com/v1/chat/completions \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}' ``` The response header `x-acog-resolved-model` names the model that answered. ## With auto mode off `auto` always works. With auto mode off, it resolves to your default connection's **default model**. That is a convenient way to change the model all your `auto` callers use from one place, under **Admin → Providers**. ## Turning auto mode on Open **Admin → Cost policies → Auto mode** (requires the **Connect and disconnect AI provider accounts** permission) and switch it on. Changes apply from the next request. ### Tiers Every model in your pool is placed on one of four tiers by what it can do, never by its price: | Tier | Name | For | |---|---|---| | 1 | Lightweight | Simple requests: greetings, lookups, short answers | | 2 | Balanced | Everyday work with a clear, bounded scope | | 3 | Advanced | Multi-step reasoning and hard problems | | 4 | Frontier | Open-ended design and the hardest problems | Each `auto` request is graded and sent to the cheapest model on the lightest tier that can handle it, or on a higher tier if that is cheaper. ACOG places known models on its reference ladder. You can move any model to a different tier; your choice sticks, even after a re-evaluation. **Re-evaluate** re-grades models ACOG has no reference placement for; you are notified when it starts and finishes. ### The pool The pool is built from the models declared on your connections. You can narrow it to some platforms or model families, and add specific models. You never enter prices: ACOG knows each provider's list prices. ### Your default model as a ceiling | Choice | Effect | |---|---| | **Use my default model as the ceiling** | Auto mode never routes to a model priced above your default model | | **My default is where most traffic should go** | Auto mode may route above your default for hard requests. Those requests save nothing: you pay the stronger model's price, and no saving is counted or billed. | ## How a request is graded 1. A free structural check settles requests whose difficulty is clear from their shape alone. 2. Otherwise a small, cheap judge model reads the request and grades it. The judge is the cheapest model in your own pool that answers, on your own connection, so it is billed by your provider and appears in reports as `acog-internal-judge`. 3. On any doubt or error, the request goes to the strongest tier. A wrongly cheap answer is worse than a missed saving. Requests that need images or tools are sent only to models that support them. ### Checking the answer After a model chosen by auto mode answers, ACOG checks the answer before returning it, including an answer replayed from cache. The check runs inside the gateway and makes no model call. An answer fails it when it is empty, when its opening reads as a refusal, or when the request asked for structured output (a JSON response format, tools or functions) and the answer is not JSON. A failed answer is sent again to the next model in auto mode's ranked list for that request, within the same limit of five attempts as below; if no candidate is left, the last answer is returned. Streamed responses are not checked, because a stream has already reached you as it is produced. ### Conversations stay on one model Auto mode keeps a conversation on a stable choice so it does not switch models mid-task. It recognizes a conversation by these request headers, if your client sends one: `x-acog-session-id`, `x-session-id`, `x-claude-code-session-id`. Agent and sub-agent identity is read from `x-acog-agent-id`, `x-agent-id` or `x-claude-code-agent-id`. ### When a model is not available If the chosen model is refused because your provider account cannot use it (`422 model_access_required`), auto mode tries the next suitable model, up to five attempts. ## How auto mode savings are measured A request auto mode sends to a cheaper model saves the difference between the price of your **default model** and the price of the model that answered. The saving is never more than what the default model would have cost. See [How savings are measured](https://tessarac.com/docs/acog/optimize/savings-measurement.md). ## Data residency The judge reads the request's text on the connection its model is configured on. If you keep traffic inside a region, make sure the judge's connection is pinned there too. See [Google Vertex AI](https://tessarac.com/docs/acog/providers/google-vertex-ai.md#keeping-traffic-in-one-country). --- # Caching > The response cache, in-flight de-duplication and your provider's prompt caching: what each one saves, when it applies and how to control it. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/caching. Last updated: 2026-10-16. ACOG uses three kinds of caching, all on by default and none of which changes an answer. | | What is reused | Saves | Applies to | |---|---|---|---| | Response cache | A complete earlier response to an identical request | The whole request | Identical, non-streaming requests | | In-flight de-duplication | One upstream call shared by identical requests arriving together | Every request but the first | Identical, non-streaming requests in flight at the same time | | Provider prompt caching | The provider's processing of a repeated prompt prefix | Most of the input cost of the repeated part | Every request to providers that support it | A fourth, the [semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md), answers near-identical prompts and is described on its own page. ## Response cache When a request is identical to an earlier one (same model, same normalized body), ACOG returns the stored response without calling your provider. The reply carries `x-acog-cache: hit`, costs nothing, and is recorded as an **Exact cache** hit. - Requests are compared after normalization: key order, whitespace in the JSON and `cache_control` markers do not make two requests different. - Entries are kept for one hour by default, encrypted, and only for your organization. - Only successful responses are stored. Errors, and responses a provider's safety system refused, are never stored. The response cache does not apply to: - streaming requests; - requests translated between API shapes (for example an OpenAI-shaped request served by a Claude model); - requests for audio output, and media generation that is not provably deterministic; - organizations that have turned the cache off. ### Turning it off Under **Admin → Gateway settings**, turn off **Response cache** to stop ACOG storing or serving cached responses for your organization, for example for privacy-sensitive workloads. The same page can flush the cache. Turning it off also turns off de-duplication and the semantic cache. ### Controlling the response cache per request Send `x-acog-cache-control` with one or more comma-separated directives: | Directive | Effect | |---|---| | `no-cache` | Do not look up a cached response, but store this one | | `no-store` | Neither look up nor store | | `ttl=` | Keep this response for this long (1 to 86400) | | `namespace=` | Keep this request's cache separate: entries in one namespace never answer another. Letters, digits, `.`, `_` and `-`, up to 64 characters. | ```bash curl -s https://gateway.acog.tessarac.com/v1/messages \ -H "Authorization: Bearer $ACOG_API_KEY" \ -H "x-acog-cache-control: no-cache" \ -H "anthropic-version: 2023-06-01" \ -H "Content-Type: application/json" \ -d '{"model": "claude-haiku-4-5", "max_tokens": 64, "messages": [{"role": "user", "content": "Hello"}]}' ``` Unknown directives are ignored. ## In-flight de-duplication When identical requests arrive while the first is still waiting for the provider, ACOG makes one upstream call and gives every caller the same response. The extra callers' responses carry `x-acog-cache: hit` and are recorded as **Dedup**. This commonly saves money on retries and on fan-out from several workers. It applies under the same conditions as the response cache. ## Provider prompt caching Providers bill repeated prompt prefixes at a much lower cached-input rate, but only when the request asks for caching. ACOG adds what each provider needs: | Provider | What ACOG does | |---|---| | Anthropic (Direct, Bedrock and Vertex) | Adds `cache_control` breakpoints on the end of the tool definitions, the system prompt and the conversation so far. It adds at most three, never more than Anthropic allows, and never changes breakpoints you set. | | OpenAI and Azure OpenAI | Sets `prompt_cache_key` from your system prompt and tools, so requests sharing them land on the same cache | | Gemini on Vertex | References a cached context when one exists for the request's prefix | | Custom endpoints | Left unchanged (servers declared as Qwen excepted) | This changes how the request is billed, not what it asks: the provider produces the same answer. ### Cache lifetime Anthropic and OpenAI let a request choose how long its prompt-cache entry lives. Longer lifetimes cost more to write and save more on reuse. ACOG decides the lifetime per request, from the first of: 1. the request header `x-acog-cache-ttl`: `auto`, `in_memory`, `5m`, `30m`, `1h`, `24h` or `forever`; 2. the API key's cache lifetime setting; 3. the connection's **Cache lifetime** setting (**Admin → Providers**); 4. the default: 5 minutes. A lifetime the provider cannot honor is dropped (the provider's default applies) rather than causing an error. Anthropic offers 5 minutes and 1 hour; OpenAI offers in-memory and 24 hours. ### Cache refresh pings An optional setting under **Admin → Gateway settings** keeps your provider's prompt caches warm between requests by sending small refresh requests before entries expire. It is off by default because it spends money to save money, and it may be unavailable on some deployments. Refresh requests are billed by your provider and appear in reports as `acog-cache-refresh`. --- # Semantic cache > Answer a prompt that means the same as an earlier one from the earlier answer, above a similarity threshold, and never from a weaker model. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/semantic-cache. Last updated: 2026-10-16. The response cache only helps when a request is byte-for-byte identical. The semantic cache also answers a prompt that means the same thing as an earlier one, for example "What's our refund policy?" and "what is the refund policy", from the earlier answer, without calling your provider. Because a similar prompt is not an identical one, the semantic cache is more careful than the response cache: - **A similarity threshold.** A stored answer is used only when the prompts are at least this similar, on a scale from 0 to 1. The recommended and default threshold is **0.97**, a strict bar that admits only very close matches. - **Never a weaker model.** A stored answer is used only if the model that wrote it is at least as capable as the model the new request asked for, by your auto mode tiers. Without tiers it is used only for the same model. An Opus answer can serve a Haiku request; a Haiku answer never serves an Opus request. - **Never a different API format.** A stored answer is used only for a request expecting the same response format: an Anthropic Messages reply answers only Messages requests, and an OpenAI chat completion only Chat Completions requests. ## Where it applies | Applies | Does not apply | |---|---| | Non-streaming requests to `/v1/messages`, `/v1/chat/completions`, and the native `/anthropic/v1/messages` and `/openai/v1/chat/completions` | Streaming requests | | | Requests translated between API shapes | | | Token counting, embeddings, Responses, Gemini `generateContent`, Bedrock and Azure native routes | | | Prompts longer than the embedding model can read in full. Two long prompts that share an opening would look identical, so ACOG does not guess. | | | Requests sent with `x-acog-cache-control: no-cache` or `no-store`, and organizations with the response cache turned off | ## Settings Under **Admin → Gateway settings → Semantic cache**: | Setting | Default | |---|---| | Semantic cache | On for hosted organizations | | Similarity threshold | 0.97 | Lowering the threshold increases hits, and also the chance of serving an answer to a question that only looks similar. The recommended threshold is 0.97. ## What you see A semantic hit returns `x-acog-cache: hit`, makes no provider call, costs nothing, and is recorded as **Semantic cache** in reports. On the compatible endpoints, `x-acog-cache-model` names the model that generated the stored answer. A semantic hit is billed like any other saving: the saving is what the request would have cost at the model you asked for. ## Privacy Each organization's semantic cache is separate; nothing is shared between organizations. Stored answers are encrypted at rest. ## Self-hosted installations The semantic cache needs an embedding service and a vector database (Milvus). The hosted service has both. A self-hosted installation runs it only after the operator enables them; see [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md#semantic-cache). --- # Batch routing > ACOG can send requests nobody is waiting for to your provider's half-price batch tier automatically, and only when it is safe and actually cheaper. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/batch-routing. Last updated: 2026-10-16. Provider batch tiers cost about half as much, for the same model and the same answer, but take longer. You can opt any request in yourself (see [Batches](https://tessarac.com/docs/acog/api/batches.md)). Batch routing goes further: ACOG decides on its own to batch a request when that costs the caller nothing. ## When ACOG batches a request on its own All of these must hold: 1. **Batching is cheaper for this request.** ACOG compares the batch price with what the request would cost synchronously, including your provider prompt-cache discounts. A request whose prompt would mostly be read from cache can be cheaper synchronously. 2. **Waiting costs nothing.** Either the provider's batch tier is currently answering about as fast as its synchronous API (measured), or ACOG has good evidence nobody is waiting for this answer, on a batch tier measured as healthy. 3. **The answer can arrive on the same connection.** The batch job must be expected to finish within the hold window (50 seconds on the hosted service), because a caller who did not opt in must receive an ordinary response, never a job handle. 4. **It is not a streaming request.** When anything is uncertain (no recent measurement, an unclear intent) the request is sent synchronously. A wrong guess here would cost someone minutes; a missed saving costs half of one request. If the caller sends a follow-up while a request is held in the batch tier, ACOG pulls the held request back and answers it synchronously, and does not batch that conversation again for a day. ## Which providers Batch routing applies to Direct Anthropic and Direct OpenAI connections. Bedrock and Vertex AI batch jobs take hours and return job handles, so ACOG never chooses them on its own; opt in explicitly to use them. ## Turning it off Under **Admin → Gateway settings**, turn off **Batch auto-routing**. Requests that opt in explicitly are unaffected. ## In reports Batched requests are recorded with the disposition **Batch** and priced at the batch rate. ACOG's own measurement requests, which keep the batch tier's latency known, appear as `acog-batch-probe` and are billed by your provider. --- # Cost arbitrage > Send a model that runs on several platforms to the cheapest one for you, counting your committed-use discounts and credits. The model never changes. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/cost-arbitrage. Last updated: 2026-10-16. The same model is often available on several platforms: Claude on the Anthropic API, AWS Bedrock and Google Vertex AI, for example. Their prices differ, and your own discounts and credits make the difference larger. Cost arbitrage sends each request for such a model to whichever platform is cheapest for you. It never changes which model runs; that is [auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md)'s job. Cost arbitrage is **off by default**. ## Setting it up Open **Admin → Cost policies → Cost arbitrage** (requires the **Connect and disconnect AI provider accounts** permission). For each model and platform you can enter: | Field | Meaning | |---|---| | List price | Supplied by ACOG from the provider's published prices | | Reserved rate | Your committed-use price, if you have one | | Reserved left | How much committed capacity remains | | Credits left | Provider credits you want used first | Fill in only what you have. ACOG uses the cheapest platform for each request, given what you entered. Changes apply from the next request. ## Interaction with routing Cost arbitrage can move a model your default connection declares to another platform (see [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md)). An `x-acog-channel` header on a request overrides it. If you need traffic to stay in one region, leave cost arbitrage off or declare the model only on connections pinned to that region. --- # Prompt cleanup > Two lossless reductions ACOG makes to requests: whitespace and invisible-character cleanup, and removal of duplicated retrieval chunks. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/request-cleanup. Last updated: 2026-10-16. Input tokens are billed whether or not they carry meaning. ACOG removes two kinds that do not. ## Prompt simplification Collapses redundant whitespace and removes invisible characters (zero-width spaces and similar) from message text. It never rewrites, paraphrases or removes any of your words, and leaves code blocks untouched, because whitespace there can matter. On by default. Turn it off under **Admin → Gateway settings → Prompt simplification**. ## Duplicate retrieval chunks Retrieval pipelines often pass the same passage more than once. When a later chunk in a message's content array, or in a top-level `documents` or `context` array, repeats an earlier one exactly or up to whitespace, ACOG drops the repeat and keeps the first occurrence in place. What the request asks for does not change. ## When they run Both run on requests from active organizations (not in [pass-through](https://tessarac.com/docs/acog/get-started/how-acog-saves-money.md#pass-through-mode)), on the compatible endpoints and the native routes alike, wherever ACOG can read the request body as a conversation. A body it cannot read that way is forwarded exactly as sent. --- # How savings are measured > How ACOG computes what each request cost, what it would have cost sent directly, and the saving you can check against your provider invoices. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/optimize/savings-measurement. Last updated: 2026-10-16. Every request through ACOG is recorded with two prices. | | Meaning | |---|---| | **Actual cost** | What the request cost through ACOG: the tokens your provider billed, at that provider's price for the model that answered. Zero for a request answered from a cache. | | **Would have cost direct** | What the same request would have cost sent straight to the provider and model you asked for, with no ACOG optimizations, at the provider's published list price. | The **saving** is the difference, counted per request and never below zero for a request. Totals add up the per-request savings, which is why a total saving can be larger than "would have cost" minus "actual" over the same period. ## Where the numbers come from - **Token counts** are the ones your provider reports in each response, including cached-input and cache-write tokens. - **Prices** come from ACOG's price book of the providers' published list prices, which follows the providers' price changes. Each recorded price notes its source. - Your own discounts with a provider are not known to ACOG, so both prices are list prices. The saving is therefore the saving at list price. ## How each mechanism is counted | Disposition | Actual cost | Would have cost direct | |---|---|---| | Exact cache, Semantic cache, Dedup | Zero: no provider call | The recorded usage of the answer that was reused, priced at the model you asked for | | Optimized (prompt caching, cleanup) | The billed tokens, including the cheaper cached-input tokens | The same request without ACOG's cache markers and cleanup | | Batch | The batch-tier price | The synchronous price for the same request, including your own prompt-cache discounts | | Auto mode | The model that answered | Your default model, for the same request. The saving is never more than the default model would have cost. | | Pass-through | The billed tokens | The same: no saving | ACOG's own calls on your connections (the auto mode judge, batch tier probes and cache refresh pings) are recorded as separate rows named `acog-internal-judge`, `acog-batch-probe` and `acog-cache-refresh`, so their cost is visible. ## Checking the numbers A request answered from cache never reached your provider, so it is absent from your provider invoice. That absence is the saving, and it can be checked. **Reports → Export → Savings proof** downloads a per-request reconciliation (CSV, Excel or JSON) with each request's tokens, prices, price sources and saving, which you can match against your provider invoices. The **How we measure savings** link on the Reports export menu explains each column. ## What you pay On the hosted service you pay a share of the measured saving, invoiced monthly; requests that save nothing cost nothing extra. See [Billing](https://tessarac.com/docs/acog/administer/billing.md) and the [pricing page](/pricing). ## Where to see it | Place | What it shows | |---|---| | **Overview** | Spend, savings and what you would have spent directly, over a window you choose | | **Reports** | Usage and savings by user, model, provider, API key or label; recent requests with each one's cost and saving; an executive summary | | **Activity** | Each request with the model asked for, the model that answered, tokens used and avoided, and its cost (requires the Activity permission) | | Management API | `GET /v1/orgs/{org}/dashboard/summary` and related endpoints. See [Management API](https://tessarac.com/docs/acog/api/management-api.md#usage-and-savings). | --- # Model access > Decide which models your organization, each team, each person and each service key may use, with allow and deny lists that only ever narrow. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/model-access. Last updated: 2026-10-16. Your connections decide which models exist for your organization. Model access decides who may use them. ## How rules stack A rule can be set at four levels: | Level | Applies to | |---|---| | Organization | Every request | | Team | Requests attributed to members of the team | | Person | Requests made with that person's keys | | Service key | Requests made with that one service key | The organization sets the ceiling, a team can only narrow it, a person narrows it further, and a single service key can be pinned tighter still. If any level blocks a model, the request is blocked: a narrower level can never re-allow what a broader one denied. Each rule is one of: | Mode | Effect | |---|---| | **Allow only these models** | Every model not listed is refused at this level | | **Deny these models** | Listed models are refused at this level | ## Setting rules Model access lists are on **Admin → Providers**. Viewing them is open to provider admins; changing them requires the **Set spend budgets, rate limits, model access, and guardrails** permission (`admin:budgets`). Changes apply from the next request. ## What a refused request gets ```json { "error": { "type": "model_not_allowed", "message": "…", "scope": "team" } } ``` The status is `403`. `scope` names the level that refused (`org`, `team`, `user` or `service-key`), and the message names the model; the allowed list itself is not disclosed to the caller. Refusals appear on the **Activity** page and can be delivered as the `governance.model_access_denied` [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event. On a [native route](https://tessarac.com/docs/acog/api/native-routes.md), if a rule restricts models and ACOG cannot tell which model the request is for (an opaque or multipart body), the request is refused with `403 model_unverifiable` rather than let through. ## With auto mode and fallbacks Model access applies to the model that will actually answer, including the model auto mode resolves `auto` to. The policy-block fallback is not used when its model is not allowed. `GET /v1/models` lists every declared model and does not apply model access, so it can list models a rule would refuse for that key. --- # Budgets > Set daily, weekly or monthly spending limits for the organization, a team or a person, with warnings in responses before anything is blocked. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/budgets. Last updated: 2026-10-16. A budget is a dollar limit on spend through ACOG for a period. When a budget is used up, ACOG pauses new requests that count against it until the period resets. ## Scopes and windows | Scope | Pool | Counts | |---|---|---| | Organization | One pool for the whole organization | All spend | | Team | One pool shared by the team's members | Spend by the team's members | | Person | That person's own pool | Spend with the person's keys | | Window | Resets | |---|---| | Daily | Midnight UTC | | Weekly | Every seven days, at midnight UTC on Thursday | | Monthly | The first of the month, midnight UTC | Every budget that applies to a request is checked, and the tightest one decides. A person on several teams with budgets counts against exactly one of them: the team with the highest limit. Spend is the actual cost of each request (see [How savings are measured](https://tessarac.com/docs/acog/optimize/savings-measurement.md)), so savings stretch a budget. ## Setting budgets Open **Admin → Governance → Spending & limits → Budgets**. Viewing budgets requires the **View invoices and billing status** permission; creating and changing them requires **Set spend budgets, rate limits, model access, and guardrails** (`admin:budgets`). Each row expands to show live usage, warnings already sent, the reset time and, for a team, spend per member. With the API: ```bash curl -s -X PUT https://acog.tessarac.com/api/v1/orgs/$ORG_ID/budgets/group/$TEAM_ID \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" \ -H "Content-Type: application/json" \ -d '{"limit_usd": 500, "window": "week"}' ``` The scope in the path is `org`, `group` (a team) or `user`. There is no "unlimited" value: delete the budget instead. ## Warnings in responses When a budget that applies to a request passes 50, 80 or 90% of its limit, ACOG warns in the response itself, once per threshold per period: - A one-line notice is appended after the model's reply, in the form `⚠ Budget notice: this {organization|group|personal} {daily|weekly|monthly} budget is {pct}% consumed (${used} of ${limit}).` In a stream, it arrives as a final text event. Token usage figures are not changed. - The response header `x-acog-budget-notice` carries the same information for programs, for example `group/week=80`. Warnings apply to chat and message responses, not to embeddings. ## When a budget is used up When a budget reaches 100% of its limit, requests that count against it are refused until the window resets: ```json { "error": { "type": "budget_exhausted", "message": "…", "scope": "group", "window": "week", "decision_id": "…" } } ``` The status is `402`. `scope` is `org`, `group` or `user`: the most specific budget that ran out. ## Alerts The [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) events `budget.threshold_crossed` (when a budget passes 50, 80 or 90% of its limit) and `budget.exhausted` (when it reaches 100% of its limit) carry the scope, window, percentage, amount used and limit. ## Your organization's maximum On the hosted service your plan may also set a ceiling over all of your traffic, shown at the top of **Spending & limits**. No budget you set can exceed it. Contact us to raise it. ## If budgets cannot be checked If ACOG cannot read budget usage for a moment (a storage outage), requests are allowed rather than blocked. --- # Rate limits > Cap requests and tokens per minute for the organization, an application, a team, a person or a service key. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/rate-limits. Last updated: 2026-10-16. Rate limits cap how fast requests and tokens can flow through ACOG, independently of your providers' own limits. ## Scopes | Scope | Default (requests per minute) | |---|---| | Organization | 600 | | Application | No limit | | Team | No limit | | Person | No limit | | Service key | 300 | Each scope has two limits: **requests per minute (RPM)** and **tokens per minute (TPM)**. Token limits have no default. Every limit that applies to a request is enforced, and the first one reached refuses it. Limits for people are set on the person, never on one of their keys; limits for a single key are set only on service keys. ## Setting limits Open **Admin → Governance → Spending & limits → Rate limits**. Changing limits requires **Set spend budgets, rate limits, model access, and guardrails** (`admin:budgets`). Leave a field at 0 to keep the default. Changes apply from the next request. ## When a limit is reached ```json { "error": { "type": "rate_limited", "message": "request rate limit exceeded for this org; retry after 12s" } } ``` The status is `429`, with a `Retry-After` header in seconds. A refused request does not count toward the limit. The [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event `governance.rate_limit_exceeded` is sent at most once a minute per caller and rule. A `429` can also come from your provider, when your provider account's own limit is reached. Its body is the provider's own error. Declaring the model on a second connection lets ACOG spread load and fail over (see [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md)). ## If limits cannot be checked If ACOG cannot read its rate counters for a moment, requests are allowed rather than blocked. --- # Data loss prevention > Detect card numbers, secrets, personal data and your own code in prompts before they reach a model, and record, mask, re-route or block them. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/data-loss-prevention. Last updated: 2026-10-16. Data loss prevention (DLP) scans prompts on their way to a model and acts on sensitive data it finds. **Nothing is scanned until you turn a data type on**, and every detector runs inside the ACOG deployment. Detection never sends the values it finds anywhere else, and its records never contain them (but see [Stored conversations and DLP](#stored-conversations-and-dlp)). Open **Admin → Data loss prevention**. Viewing it requires **View invoices and billing status**; changing it requires **Manage data-loss-prevention rules and connect source code** (`admin:security`). ## What it detects Detection runs in three layers. | Layer | Detects | Default action when turned on | |---|---|---| | 1. Exact data | Credit card numbers, US Social Security numbers, bank routing numbers, IBANs, live secrets and API keys. Checked by format and checksum, so it is the most precise layer. | Block | | 2. Free text | People's names, email addresses, phone numbers, mailing addresses, IP addresses, medical and health terms, and your own **custom dictionary** of terms | Audit | | 3. Protected sources | Text that matches code or documents in repositories you connect | Audit | ### Custom dictionary Add words or phrases specific to your organization, such as project code names. ACOG stores a one-way hash of each term, not the text. Terms are up to 128 characters, with options for whole-word and case-sensitive matching. ### Protected sources Connect a GitHub or GitLab repository and ACOG indexes it as fingerprints. A prompt that contains text from the repository is detected even when it has been edited slightly. The repository content itself is not kept. ## Actions | Action | Effect | |---|---| | **Audit** | The prompt goes through unchanged. ACOG records that it saw the data type, never the value. | | **Tokenize** | (Exact data and free text, set through the API or a policy rule) The value is replaced with a deterministic token before the prompt leaves, and the real value is restored in the reply, so the workflow still works and the model never sees it. | | **Mask** | The value is replaced with a placeholder before the prompt leaves. The model never sees it. | | **Route** | (Protected sources and policy rules) The request may only be served by platforms you approve, for example your own cloud account. | | **Block** | The request is refused and never reaches the model. | Each detector has a confidence threshold from 50 to 100% (60 to 100% for protected sources). The recommended starting points are 85 for exact data, 50 for free text and 85 for protected sources. ## Policy rules Policy rules apply different actions for one team, one person or one application, on top of the data types you turned on. Precedence is organization, then team, then the person or application. A rule marked **Off** is not applied. ## What a blocked request gets ```json { "error": { "type": "dlp_blocked", "message": "…", "category": "credit_card", "reason": "layer1_baseline", "decision_id": "…" } } ``` The status is `403`. Give the `decision_id` to an admin to find the event. `reason` is one of `layer1_baseline`, `layer2_detection`, `corpus_match`, `no_compliant_route`, `dlp_unavailable` or `input_too_large_to_scan`. ## If the scanner is unavailable Exact-data checks run inside the gateway and are always available. Free-text and protected-source detection run in a separate service. If it cannot be reached, the setting **If the scanner is unavailable** decides: | Choice | Effect | |---|---| | **Block the request** (default, recommended) | The request is refused with `403 dlp_blocked`, reason `dlp_unavailable` | | **Let the request through** | The request continues without those layers | ## What is scanned | | Scanned | |---|---| | Prompts on the compatible endpoints and native routes | Yes | | Files sent inline in a prompt | As text | | Requests whose body is not JSON | Exact-data types only | | Embeddings requests | No | | Model replies | In a successful non-streamed reply, tokenized values are restored, and any exact-data value (card, Social Security number, routing number, IBAN, secret) whose type is set to Mask or Block is masked before the reply reaches you. Free-text types are not masked in replies. In a streamed reply, tokenized values are restored; masking is not applied. | ## Seeing what was found The **Overview** tab shows findings by data type and outcome. Blocks and masks are delivered as the `governance.dlp_blocked` [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event. A detection record holds the category, its confidence, a reference ID and the position in the text, never the value itself. ## Stored conversations and DLP **Store prompts for review** (on by default, under **Admin → Gateway settings**) keeps each conversation as it was sent to the model, which is after DLP has acted: | Action for the data type | In the stored conversation | |---|---| | Block | Nothing: the request never reached the model | | Mask | The placeholder, not the value | | Tokenize | The token, not the value | | Audit | The value, because Audit lets the prompt through unchanged | The same applies to prompt and completion text sent to your own Langfuse project. In the stored and exported reply, only exact-data types set to Mask or Block are masked; a free-text value the model writes into its reply is kept as written. If a data type must never be kept, set it to **Mask** or **Block** and, for free-text types, turn off prompt storage and Langfuse prompt and completion capture (see [Observability](https://tessarac.com/docs/acog/administer/observability.md)). See [Activity and audit log](https://tessarac.com/docs/acog/govern/activity-and-audit-log.md#stored-conversations). ## Self-hosted installations Names and medical terms need a named-entity model. The chart deploys one by default; the DLP service reports `ner_ready` when it is loaded. See [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md). --- # Prompt injection protection > Screen prompts for injection and jailbreak attempts, log them or block them, and review every attempt caught. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/prompt-injection. Last updated: 2026-10-16. ACOG screens prompts for attempts to override a model's instructions: instruction overrides, role and persona hijacks, attempts to extract the system prompt, guardrail evasion and delimiter injection. **Screening is on by default, in log-only mode.** Attempts are recorded and requests go through unchanged until you choose to block. ## Settings Open **Admin → Governance → Prompt security**. Changing the setting requires **Manage data-loss-prevention rules and connect source code** (`admin:security`). | Setting | Effect | |---|---| | **Screen prompts for injection attempts** off | Prompts are sent with no screening, and nothing is recorded | | On, **Log only** | Every prompt is screened; attempts are recorded and the request continues | | On, **Block the request** | Every prompt is screened; an attempt is refused before it reaches the model | The mode can also be set per team and per API key. The most specific setting wins: key, then team, then organization. ## What a blocked request gets ```json { "error": { "type": "prompt_injection_blocked", "message": "…", "category": "instruction_override" } } ``` The status is `400`. Blocks are delivered as the `governance.guardrail_blocked` [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event. ## Reviewing attempts The **Injection attempts** log on the same page lists every attempt caught, with its category and who sent it. It is visible to holders of `admin:security`. ## Where it applies Prompt screening runs on the compatible endpoints and the native routes. It does not run on embeddings requests. --- # Activity and audit log > See every request made through the gateway and who made it, read stored conversations, and review every administrative change. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/govern/activity-and-audit-log. Last updated: 2026-10-16. ACOG keeps two records: the **Activity** page, of requests through the gateway, and the **Audit log**, of changes to your organization's configuration. ## Activity **Activity** lists every request your organization made through the gateway, newest first, and who made it. Filter by application, user, team, outcome, tokens and cost, over a time window. Open a request to see: - **The request:** who, which key and application, when. - **Which model answered:** the model requested, the model that served it, whether auto mode chose it, your default model, and the routing saving. - **Tokens:** used, what would have been used, and tokens avoided. - **Timing and outcome:** total latency, the provider's share and ACOG's overhead, and any provider error. ### Stored conversations When prompt storage is on, the request also shows **the conversation**: the prompt and the reply. Streamed replies are marked as not scanned. | Setting (Admin → Gateway settings → Store prompts for review) | Default | |---|---| | Store prompts for review | On | | Delete after | 30 days (1 to 365) | Turning it off stops storing new conversations. Stored conversations are deleted when they reach the retention period. A change affects only requests made after it. ### Who can see Activity Activity requires the **See every request made through the gateway, and by whom** permission (`audit:conversations`). Only people signed in to the console can hold it: no API key and no custom role can carry it. Every view of a stored conversation is itself recorded in the audit log. ## Audit log **Admin → Audit log** lists every change to your organization's configuration, newest first: who made it, when, the action (create, update or delete) and what it changed, for example `monthly_limit_usd 25000 → 30000`. Expand an entry for the actor's role, the exact time, the API endpoint, the source IP and a before-and-after table. Search covers the whole log, not only the page on screen. Filter by actor and by kind of entity. Impersonation (an admin viewing the console as another member) is recorded at start and end, with both identities. The audit log requires **View invoices and billing status** (`read:billing`). --- # Users and invitations > Invite people to your organization, give them roles, change or remove them, and view the console as another member. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/users-and-invitations. Last updated: 2026-10-16. Membership is invite-only. Open **Admin → Users** to manage members, invitations and roles. You need the **Invite, remove, and manage team members and roles** permission (`admin:users`). ## Adding people ### Hosted service: invitations Choose **Invite a member**, enter the email address and choose a role. The person receives an email invitation, chooses how to sign in, and joins with the role you picked. There is no temporary password to pass on. Pending invitations are listed under **Invitations**, where you can revoke one; the link then stops working immediately. People who sign in through your [single sign-on](https://tessarac.com/docs/acog/administer/single-sign-on.md) are added automatically on their first sign-in, with a role mapped from their identity provider's groups. ### Self-hosted: add a member Choose **Add member**, enter the email address and choose a role. ACOG shows a one-time initial password, **once**; give it to the person securely. They must choose their own password at first sign-in. ## Roles | Role | Can | |---|---| | **Admin** | Everything | | **Auditor** | View dashboards, reports, billing and the audit log. Cannot call the gateway or change anything. | | **Member** | Call the gateway with their own keys and view dashboards | You can also create custom roles from any combination of permissions. See [Roles and permissions](https://tessarac.com/docs/acog/reference/roles-and-permissions.md). You can only give a role whose permissions you hold yourself. A refused change says which permission you lack. ## Changing someone's role Choose the role in the member's row and confirm. If you change your own role to one that cannot manage users, you lose access to this page as soon as you save. The organization always keeps at least one admin: the only admin's role cannot be changed and they cannot be removed until someone else is an admin. ## Removing someone **Remove** ends the person's access to the organization immediately. Their user keys are disabled, and team-scoped keys are unbound. Their past usage stays in reports. ## Viewing the console as another member An admin signed in to the console can impersonate another member to see the console exactly as they do, for example to check what their role allows. The admin's own session is paused until they choose **Exit impersonation**. The start and end are recorded in the [audit log](https://tessarac.com/docs/acog/govern/activity-and-audit-log.md) with both identities. Impersonation is not available to API keys. --- # Teams > Group members below the organization so you can give each team its own budget, rate limits and model access. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/teams. Last updated: 2026-10-16. A team is a group of members inside your organization. Teams let you set [budgets](https://tessarac.com/docs/acog/govern/budgets.md), [rate limits](https://tessarac.com/docs/acog/govern/rate-limits.md) and [model access](https://tessarac.com/docs/acog/govern/model-access.md) for a group of people at once, and report usage by team. In the budgets API a team is called a **group**. Open **Admin → Teams**. You need the **Create teams and manage their membership** permission (`admin:teams`). ## Creating a team Create the team and name it. You are added as its owner. ## Members Open a team to manage its roster. You can add or remove one or several members at once; each member has a team role of owner, admin or member. Teams are made of existing members of the organization. A person can be on several teams. For budgets, a person on several teams with budgets counts against the team with the highest limit. Removing a member from a team unbinds their keys that were bound to the team; the keys keep working. ## Deleting a team Keys bound to the team are unbound, not disabled. --- # Single sign-on > Let your team sign in to ACOG through your identity provider over OIDC or SAML, with roles mapped from groups and sign-in routed by email domain. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/single-sign-on. Last updated: 2026-10-16. Connect your identity provider (Okta, Microsoft Entra ID, Ping, Keycloak, Auth0, ADFS and others) so your team signs in with it. People are added to your organization on their first sign-in, with a role mapped from their identity-provider groups. Open **Admin → Single sign-on**. You need the **Configure single sign-on (OIDC / SAML) identity providers** permission (`admin:sso`). ## 1. Register ACOG with your identity provider Choose **Add connection** and pick OIDC or SAML. The **Register with your IdP** panel shows the values to give your identity provider, with copy buttons: | Protocol | Value | |---|---| | OIDC | Redirect URI: `https://acog.tessarac.com/api/v1/auth/sso/{connection id}/callback` | | SAML | Assertion Consumer Service URL: `https://acog.tessarac.com/api/v1/auth/sso/{connection id}/acs` | | SAML | Service provider metadata: `https://acog.tessarac.com/api/v1/auth/sso/{connection id}/metadata` | On a self-hosted installation the host is your installation's. ## 2. Configure the connection **OIDC** | Field | Notes | |---|---| | Issuer URL | Your identity provider's issuer. On the hosted service it must be a public HTTPS address. | | Client ID and client secret | From the application you registered. The secret is stored encrypted; leave it blank later to keep it. | | Scopes | Default `openid email profile groups`. Must include the email and groups scopes. | | Email claim, groups claim | Which claims carry the email address and groups | **SAML** | Field | Notes | |---|---| | IdP metadata XML | Paste your identity provider's metadata | | Email attribute, groups attribute | Which attributes carry the email address and groups | The protocol cannot be changed after the connection is created. ## 3. Map groups to roles | Setting | Effect | |---|---| | **Group → role mapping** | Map an identity-provider group to an ACOG role. The first matching group wins. | | **Default role** | The role for someone whose groups match no mapping | | **Email domain allow-list** | Optional. When set, only people with an email address at these domains are added. | ## 4. Route sign-in by email domain People sign in at the same address as everyone else. To send everyone with an email at your domain to your identity provider automatically, claim the domain under **Email domain routing**: add the DNS TXT record shown (`acog-site-verification=`) to your domain, then choose **Check DNS**. Until the record is verified, the domain routes no sign-ins. ## Recovery Existing members keep their accounts if you delete a connection, but need another way to sign in. On a self-hosted installation, password sign-in remains available for break-glass recovery if your identity provider is unreachable. Changes to SSO connections are delivered as the `security.sso_connection_changed` [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event and recorded in the [audit log](https://tessarac.com/docs/acog/govern/activity-and-audit-log.md). --- # API keys > Create gateway keys for people and services, limit their permissions and source networks, attribute their traffic, and revoke them. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/api-keys. Last updated: 2026-10-16. Gateway keys (`acog_…`) authenticate requests to the gateway and to the management API. Manage them on the **Developer** page. Anyone with gateway access can create keys for themselves; admins with the **Create and revoke API keys for anyone in the org** permission (`admin:keys`) can create keys for others and service keys. ## User keys and service keys | | User key | Service key | |---|---|---| | Used by | A person, as themselves | An application, script or bot | | Usage attributed to | The person | The key (and its application) | | Limits and model rules | The person's | Set on the key itself | | Who can create | Anyone, for themselves; admins, for anyone | Admins with `admin:keys` | | Can belong to an application | No | Yes | ## Creating a key On **Developer**, create a key and set: | Setting | Notes | |---|---| | Name | Where the key is used, for example `ci-pipeline` | | Type and owner | Admins only: user key for a chosen person, or service key | | Permissions (scopes) | Default: gateway access only (`proxy`). A key can never have more permissions than you. | | Application | Service keys only. See [Applications](https://tessarac.com/docs/acog/administer/applications.md). | | Trace tags | Tags added to this key's traces; see [Observability](https://tessarac.com/docs/acog/administer/observability.md) | | Cache lifetime | Override the provider prompt-cache lifetime for this key's requests | | IP allowlist | IPv4 or IPv6 ranges (CIDR) this key may be used from. Empty means any source. | **The key is shown once.** Copy it into your secret manager before closing the dialog: ACOG stores only a SHA-256 hash and cannot show it again. A key with any admin permission is powerful: anyone holding it can change your organization. ACOG asks you to confirm, and recommends restricting it to specific IP ranges and binding it to a person. ## IP allowlist A request from outside a key's allowlist is refused with `401` and a generic message, on both the gateway and the management API. On a self-hosted installation behind a load balancer, the operator must set `ACOG_TRUSTED_PROXY_CIDRS` so ACOG sees the real client address. ## Revoking a key Deleting an active key **disables** it: requests with it fail with `401` within about five seconds. A disabled key can then be deleted permanently. Keys are also revoked when their person is removed from the organization. Revocations are delivered as the `security.api_key_revoked` [webhook](https://tessarac.com/docs/acog/administer/webhooks.md) event. ## Rotating a key Create a new key, deploy it, then disable the old one. To avoid an outage, keep both until every caller uses the new key. ## Keys for native AWS Bedrock access boto3 and other AWS SDKs sign requests with an access key pair rather than sending a bearer token. For them, issue an ACOG Bedrock credential instead; see [AWS Bedrock](https://tessarac.com/docs/acog/providers/aws-bedrock.md#native-bedrock-access-with-boto3). --- # Applications > Name the applications that use the gateway, attribute their traffic through their keys, and control what they send to Langfuse. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/applications. Last updated: 2026-10-16. An application is a name for the traffic one of your systems sends through ACOG, such as `support-bot` or `nightly-etl`. Reports, rate limits, DLP policy rules and traces can then be broken down or set per application. Open **Admin → Applications**. You need the **Create and revoke API keys for anyone in the org** permission (`admin:keys`). ## How traffic is attributed Every request is attributed to the application that owns the service key it was made with. No request header can change the attribution, so callers cannot report their traffic as another application's. Requests from keys with no application, including all user keys, are reported as `unattributed`. ## Registering an application | Field | Notes | |---|---| | Name | A display name | | Slug | Lowercase letters, digits and hyphens. Permanent: it names the application on every trace. `acog` is reserved. | | Forward traces | On by default. When on, ACOG sends a trace to Langfuse for every request the application makes (if Langfuse is configured). Turn it off for an application that already sends its own traces. Usage, cost and savings reporting are unaffected either way. | | Trace tags, trace filter | Tags added to this application's traces, and sampling. See [Observability](https://tessarac.com/docs/acog/administer/observability.md). | Then assign service keys to it on the **Developer** page. ## Deleting an application Its keys keep working: they are unbound, not disabled, and their traffic goes back to being reported as `unattributed`. --- # Webhooks > Receive signed HTTPS notifications when a budget is crossed, an invoice is issued, a request is refused by policy or a key is revoked. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/webhooks. Last updated: 2026-10-16. Webhooks tell your systems when something happens in your organization. Every delivery is signed, retried on failure and recorded in a delivery log. Open **Admin → Webhooks**. You need the **Edit the organization's profile** permission (`admin:org`). ## Adding an endpoint Choose **Add endpoint** and enter: - an HTTPS URL. On the hosted service it must be a public address on port 443; a self-hosted installation can deliver to any port; - a description; - the events to send. The endpoint's **signing secret is shown once**. Store it with your receiver. An organization can have up to 20 endpoints. ## Events | Event | When | |---|---| | `budget.threshold_crossed` | A budget first passes 50, 80 or 90% of its limit in a period | | `budget.exhausted` | A budget reaches 100% of its limit; requests against it are refused | | `billing.invoice_created` | A monthly invoice is issued | | `billing.invoice_paid` | An invoice is paid | | `billing.payment_failed` | A payment attempt fails | | `billing.state_changed` | Savings switch between active and pass-through | | `governance.guardrail_blocked` | Prompt injection screening blocks a prompt | | `governance.dlp_blocked` | Data loss prevention blocks or masks a request | | `governance.model_access_denied` | A caller asks for a model it may not use | | `governance.rate_limit_exceeded` | A caller exceeds a rate limit | | `security.api_key_created` | A key is created or rotated in | | `security.api_key_revoked` | A key is revoked, deleted, or removed with its member | | `security.member_invited` | Someone is invited (hosted) or added (self-hosted) | | `security.member_removed` | A member is removed | | `security.sso_connection_changed` | A single sign-on connection is created, changed or deleted | | `webhook.ping` | **Send test** on an endpoint | The four `governance.*` events are sent at most once a minute for each caller and rule; every refusal is still on the Activity page. **No event contains prompt text, a detected value or a secret.** ## What your receiver gets One `POST` per event and endpoint: ```http POST /your/webhook HTTP/1.1 Content-Type: application/json User-Agent: acog-webhooks/1 Acog-Event: budget.threshold_crossed Acog-Delivery: whd_8Kd2… Acog-Signature: t=1792161166,v1=5f0c3a… {"id": "whev_3xQ…", "type": "budget.threshold_crossed", "created_at": "2026-10-16T14:32:18Z", "data": {"scope": "org", "pool_id": "…", "window": "month", "pct": 80, "used_usd": 4000, "limit_usd": 5000}} ``` `id` identifies the event and is the same on every attempt and every endpoint. `Acog-Delivery` identifies this event's delivery to this endpoint and is the same on every retry: use either to ignore duplicates. ## Verifying the signature `Acog-Signature` carries a timestamp `t` and `v1`, the hex HMAC-SHA256 of `"."` keyed with the endpoint's secret. ```python import hashlib, hmac, time def verify(secret: str, header: str, raw_body: bytes, tolerance: int = 300) -> bool: parts = dict(item.split("=", 1) for item in header.split(",")) timestamp, signature = parts["t"], parts["v1"] if abs(time.time() - int(timestamp)) > tolerance: return False expected = hmac.new(secret.encode(), f"{timestamp}.".encode() + raw_body, hashlib.sha256).hexdigest() return hmac.compare_digest(expected, signature) ``` Verify against the raw body exactly as received, compare in constant time, and reject a timestamp more than five minutes from now. ## Responding and retries Answer with any `2xx` status within 10 seconds, then do the work. Anything else is a failure, including a redirect (redirects are not followed). - A failed delivery is retried after 1 minute, 5 minutes, 30 minutes, 2 hours, 6 hours, 12 hours and 24 hours. - After 50 consecutive failures the endpoint is switched off; the console says why. - A delivery still undelivered after 3 days is given up. - Delivery is at least once and events are not ordered. Events that happen while an endpoint is switched off are not queued for it. ## Managing an endpoint Open an endpoint to see its delivery log (every attempt and its payload, kept 30 days), change its events, switch it off or on, **rotate its secret** (the old secret stops working immediately) or **send a test**. --- # Observability > Send a trace for every request to Langfuse or any OpenTelemetry collector, with tags, filtering and control over whether prompt text leaves the gateway. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/observability. Last updated: 2026-10-16. ACOG can send a trace for every request to your own observability tools. Traces are sent after the request, in the background: a failure here never delays or fails a request, or changes its billing. Open **Admin → Observability**. You need the **Edit the organization's profile** permission (`admin:org`). ## Destinations Configure either, both or neither. | Destination | Settings | What is sent | |---|---|---| | **OpenTelemetry** | An OTLP/HTTP endpoint | One span per request, using the OpenTelemetry generative-AI semantic conventions | | **Langfuse** | Host, public key and secret key (all three, or none) | A trace per request with its generation: model, tokens, cost, latency and, optionally, the prompt and completion | The Langfuse secret key is stored encrypted and never shown again. ## Prompt and completion text With prompt and completion capture on, each Langfuse trace includes the request's prompt and the model's reply. Off, only the economics leave the gateway: tokens, cost, latency and model. Capture is on by default once Langfuse is configured. If you use [data loss prevention](https://tessarac.com/docs/acog/govern/data-loss-prevention.md), consider turning capture off: text DLP allowed through to the model would otherwise also be sent to Langfuse. ## Tags Tags (for example `env:prod`) help you filter traces. They come from four places and add up, never replacing each other: 1. organization default tags, set here; 2. the application's tags ([Applications](https://tessarac.com/docs/acog/administer/applications.md)); 3. the API key's tags ([API keys](https://tessarac.com/docs/acog/administer/api-keys.md)); 4. the request's own `x-acog-trace-tags` header (comma-separated), plus optional JSON in `x-acog-trace-metadata`. ## Filtering Trace filters (keep only errors, drop cache hits, sample a percentage) limit what Langfuse receives. They never affect usage reporting or billing. ## Per application Each application can turn trace forwarding off, for example when it already sends its own traces. See [Applications](https://tessarac.com/docs/acog/administer/applications.md). --- # Billing > How the hosted service bills a share of measured savings, monthly invoices, payment methods, account credit and pass-through mode. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/billing. Last updated: 2026-10-16. This page describes billing on the hosted service. A self-hosted installation is licensed instead; see [Licensing](https://tessarac.com/docs/acog/self-hosted/licensing.md). ## What you pay for You pay a share of what ACOG measurably saves you, and nothing else: no seats, no per-token markup, no minimum. A request that saves nothing costs nothing extra. The rate is on the [pricing page](/pricing). How savings are measured is described in [How savings are measured](https://tessarac.com/docs/acog/optimize/savings-measurement.md). Your provider bills you directly for the tokens you use, as before. ACOG never resells provider capacity. ## Activating savings A new organization starts in **pass-through** mode: requests work and are forwarded unchanged at full price, nothing is saved and nothing is billed. Add a payment method under **Admin → Billing** and savings switch on. ## Invoices - An invoice is issued for each completed calendar month, for that month's measured savings. - Payment is due 30 days after the invoice is issued. - Your default payment method is charged; if it cannot be charged, backup methods are tried in the order shown. - **Account credit** is applied to invoices before anything is charged. A partly used credit carries its remainder to the next invoice. If an invoice is more than 30 days overdue, your organization moves to pass-through until the balance is paid. Adding a new card does not by itself clear an overdue balance. ## The Billing page **Admin → Billing** shows, for users with the **View invoices and billing status** permission: | | | |---|---| | Status | **Cost savings active** or **Pass-through** | | Saved this month | Updated live | | Accrued fee | This month's share so far, billed at month end | | Amount due | The open invoice, its due date and whether it is overdue | | Account credit, past due | Where applicable | | Payment methods | Card or US bank account (ACH). Card and bank details go directly to the payment processor; ACOG never sees or stores them. Bank account verification by micro-deposits takes one to two business days. | | Invoices | Each month's savings, fee, total, due date and status, with **Pay** for an open invoice | Your organization's name, billing contact and address printed on invoices are on **Admin → Organization**. ## Events `billing.invoice_created`, `billing.invoice_paid`, `billing.payment_failed` and `billing.state_changed` can be delivered to your systems as [webhooks](https://tessarac.com/docs/acog/administer/webhooks.md). --- # Organization and gateway settings > Your organization's profile and ID, and the gateway settings that decide what ACOG may cache, trim or batch for your traffic. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/organization-and-gateway-settings. Last updated: 2026-10-16. Both pages require the **Edit the organization's profile** permission (`admin:org`) to change; others can view them. ## Admin → Organization | Field | Notes | |---|---| | Organization ID | `org_…`, read-only. Used in management API paths. | | Created | Read-only | | Name | Shown in the console and on invoices | | Billing contact | Email and phone printed on every invoice | | Billing address | Street, unit, city, state or province, postal code and country | ## Admin → Gateway settings What the gateway may replay or trim to make a request cost less. Every setting applies to every request your keys make, from the next one on. | Setting | Default | Page | |---|---|---| | Response cache | On | [Caching](https://tessarac.com/docs/acog/optimize/caching.md#response-cache) | | Semantic cache and similarity threshold | On, 0.97 (hosted) | [Semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md) | | Prompt simplification | On | [Prompt cleanup](https://tessarac.com/docs/acog/optimize/request-cleanup.md) | | Batch auto-routing | On | [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md) | | Cache refresh pings | Off | [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-refresh-pings) | | Store prompts for review, and retention | On, 30 days | [Activity and audit log](https://tessarac.com/docs/acog/govern/activity-and-audit-log.md#stored-conversations) | ## Admin → Webhooks: other settings Three more settings sit with the webhooks: | Setting | Effect | |---|---| | Budget alert webhook (legacy) | A single unsigned URL that receives a Slack-compatible alert when a budget passes 50, 80 or 90% of its limit and when it reaches 100% of its limit. Prefer [signed webhooks](https://tessarac.com/docs/acog/administer/webhooks.md). | | Cost recommendations by email | ACOG compares your configuration with how your traffic is actually served and emails your admins when a change would save money. Nothing is changed for you, and no mail is sent when there is nothing to recommend. | | Daily model check | Once a day, ACOG lists the models each connection can call and tells your admins about new or retired models. It never changes a connection. | ## Admin → Cost policies [Auto mode](https://tessarac.com/docs/acog/optimize/auto-mode.md) and [cost arbitrage](https://tessarac.com/docs/acog/optimize/cost-arbitrage.md). These require the **Connect and disconnect AI provider accounts** permission. --- # Notifications and the assistant > What the notification bell tells you, and how the built-in assistant answers questions about your usage, spend and reports. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/notifications-and-assistant. Last updated: 2026-10-16. ## Notifications The bell at the top of the console shows notifications for your organization, newest first, each marked as information, a recommendation or a warning. They come from: - **Cost recommendations:** a configuration change that would save money; - **The daily model check:** models a provider added or retired, or a connection whose settings no longer validate; - **Auto mode:** a re-evaluation starting and finishing; - **Prompt caching:** settings that keep your provider's prompt cache from paying off; - **AWS Bedrock:** an external ID switch date approaching, or a suspended connection. Notifications can carry a link to the page where you act on them. **Mark all read** clears the unread count; **Clear all** removes them. ACOG only notifies you when a change would help: an empty bell means there is nothing to do. ## The assistant The assistant (bottom right of every page) answers questions about your organization's data, for example: - "How much have I spent this month?" - "Cost by model, last 30 days" - "Which users saved the most?" - "Has anyone triggered DLP this week?" It can only read data you are already allowed to see, using read-only tools, and every data-backed answer links to the page that proves it. It is not a general-purpose chat. The assistant runs on your organization's own connected providers, choosing the cheapest capable model, so it is billed to your provider account like any other request. It needs at least one provider connection. Conversations are kept in your browser and cleared when you switch organization or sign out. --- # Your account > Your profile and sign-in security: password, two-factor authentication, recovery codes and passkeys. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/administer/your-account. Last updated: 2026-10-16. Open **Settings** from the menu under your name. Everyone can manage their own account. ## Profile Your display name is editable: it is how your name appears across the console. Your organization, role and email address are managed by an admin. ## Sign-in security **Hosted service.** Sign-in is handled by Tessarac's sign-in service or your organization's [single sign-on](https://tessarac.com/docs/acog/administer/single-sign-on.md). Manage your password, two-factor authentication and passkeys in the security panel on this page. If your organization uses single sign-on, your identity provider manages them. **Self-hosted installation.** When you sign in with a password: | | | |---|---| | Password | Change it at any time; you stay signed in | | Two-factor authentication | A 6-digit code from an authenticator app at every sign-in. You receive recovery codes once: store them safely, each works once. | | Passkeys | Sign in with your device's screen lock or a hardware security key | People who sign in through single sign-on manage these with their identity provider. ## Sessions On a self-hosted installation, a console session lasts 24 hours from your last activity. Signing out on the hosted service also ends your session with Tessarac's sign-in service, so the next sign-in asks who you are. --- # Self-hosted overview > What a self-hosted ACOG installation runs, what it needs from your environment, what it connects to, and how it is licensed. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/overview. Last updated: 2026-10-16. ACOG can run entirely inside your own environment. A self-hosted installation runs the same product as the hosted service, built from the same code, and is installed with one Helm chart. ## What you run | Service | Role | |---|---| | Web | The console, and the single entry point: it serves `/gateway` (the data plane) and `/api` (the control plane) on one address | | Data plane | The gateway your applications call | | Control plane | Organizations, users, keys, connections, settings, licensing | | Metering | Usage, cost and savings, off the request path | | DLP | Free-text and protected-source detection for data loss prevention | | Embedder | Embeddings for DLP protected sources and the semantic cache | | NER | Named-entity detection (names, medical terms) for DLP | And their datastores: **PostgreSQL** (configuration), **Dragonfly** or another Redis-protocol store (hot state, caches), **ClickHouse** (usage and analytics), and optionally **Milvus** (the semantic cache). The chart can run all of them for you or use ones you already operate. All images are public at `docker.io/tessarac` and multi-architecture (amd64 and arm64). The data plane and control plane use their `-selfhost` builds (`tessarac/dataplane-selfhost`, `tessarac/controlplane-selfhost`). ## Requirements | | | |---|---| | Kubernetes | Any conformant distribution, with Helm 3 (OCI support) and a ReadWriteOnce StorageClass. See [Install on Kubernetes](https://tessarac.com/docs/acog/self-hosted/kubernetes.md). | | Size | About 8 vCPU, 16 GiB memory and 60 GiB disk for a single node running the bundled datastores. With the semantic cache, plan for 16 vCPU, 32 GiB and 200 GiB or more. | | Ingress | One HTTPS address for the web service, with a 660-second idle timeout and a 64 MiB body limit. You provide the TLS certificate; ACOG itself needs no certificates, certificate authority or cert-manager. | | Email | An SMTP relay (or Amazon SES) for verification, password reset and invitation mail. Required in production. | | Licence | A licence file and its password, from your Tessarac agreement. See [Licensing](https://tessarac.com/docs/acog/self-hosted/licensing.md). | ## What it connects to | Destination | Why | Required | |---|---|---| | Your AI providers | To serve requests | Yes | | Your SMTP relay | Mail | Yes, in production | | Your identity provider | Single sign-on | If you use it | | Your Git host | DLP protected sources | If you use them | | Your OpenTelemetry collector or Langfuse | Traces | If you use them | | Tessarac | | **Never.** No component contacts Tessarac at runtime, licensing included. | Users' browsers load nothing from outside your installation's address. ## How it differs from the hosted service | | Self-hosted | |---|---| | Sign-in | Passwords, two-factor authentication, passkeys and your own SSO. Members are added by an admin with a one-time password, not by email invitation. | | Billing | No invoices. The licence switches savings on; without one the gateway runs in pass-through. | | Keyless connections | Also **install identity**: use the identity your installation already runs as. See [Install identity](https://tessarac.com/docs/acog/self-hosted/install-identity.md). | | Provider addresses | Any address you configure, including private networks and plain HTTP | | Semantic cache | Off until you enable Milvus and the embedder | | Batch hold window | 10 minutes by default (`ACOG_BATCH_HOLD_SECS`); your load balancer's idle timeout must cover it (660 s) | ## Next 1. [Install on Kubernetes](https://tessarac.com/docs/acog/self-hosted/kubernetes.md) 2. [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md) 3. [Licensing](https://tessarac.com/docs/acog/self-hosted/licensing.md) --- # Install on Kubernetes > Install ACOG with its Helm chart: write your overlay, render and check it, install, sign in as the first administrator and load your licence. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/kubernetes. Last updated: 2026-10-16. The supported installation path is the Helm chart `oci://docker.io/tessarac/ai-cost-optimizer`. Pulls are anonymous. ## Before you start - A Kubernetes cluster, `kubectl` and Helm 3 with OCI support. - A StorageClass that provisions ReadWriteOnce volumes (`kubectl get storageclass`). Without one, the datastore volumes stay `Pending`. - An ingress controller, and a DNS name and TLS certificate for the console, for example `acog.example.com`. - An SMTP relay. - Your licence file and password. ## 1. Choose the chart version Use the newest release unless you have a reason not to. Releases are numbered `1.x.y`; versions ending in `-beta.N` are pre-releases and are never chosen unless you name them. ```bash helm show chart oci://docker.io/tessarac/ai-cost-optimizer export CHART_VERSION= helm pull oci://docker.io/tessarac/ai-cost-optimizer --version "$CHART_VERSION" --untar ``` The pulled chart contains `values-selfhost.yaml`, the vendor overlay for self-hosted installations. It pins the images that go together for this version; read it before every install or upgrade. ## 2. Write your own overlay Keep your settings in a file of your own, layered after the vendor overlay. Generate the secrets first: ```bash openssl rand -base64 32 # ACOG_CREDS_KEY openssl rand -hex 32 # ACOG_OPERATOR_TOKEN openssl rand -hex 32 # ACOG_INGEST_TOKEN ``` `my-install.yaml`: ```yaml # Run the datastores as plain Deployments inside this release. # The vendor overlay enables the high-availability topology, which needs four # cluster-wide operators. Turn it off unless you have installed them. ha: enabled: false infra: bundled: storageClass: "" postgres: password: "" clickhouse: password: "" secret: ACOG_POSTGRES_DSN: "postgres://aco:@aco-postgres:5432/aco?sslmode=disable" ACOG_CLICKHOUSE_PASSWORD: "" ACOG_CREDS_KEY: "" ACOG_OPERATOR_TOKEN: "" ACOG_INGEST_TOKEN: "" services: controlplane: extraEnv: ACOG_ENVIRONMENT: "production" ACOG_SELF_SERVE_SIGNUP: "false" ACOG_WEB_ORIGIN: "https://acog.example.com" ACOG_PUBLIC_BASE_URL: "https://acog.example.com" ACOG_TRUSTED_PROXY_CIDRS: "10.0.0.0/8" ACOG_EMAIL_PROVIDER: "smtp" ACOG_EMAIL_FROM: "no-reply@example.com" ACOG_SMTP_HOST: "smtp.example.com" ACOG_SMTP_PORT: "587" ACOG_BOOTSTRAP_ADMIN_EMAIL: "admin@example.com" metering: extraEnv: ACOG_ENVIRONMENT: "production" ingress: enabled: true className: "" annotations: # for ingress-nginx; see below nginx.ingress.kubernetes.io/proxy-read-timeout: "660" nginx.ingress.kubernetes.io/proxy-send-timeout: "660" nginx.ingress.kubernetes.io/proxy-body-size: "64m" hosts: - { host: acog.example.com, service: web, port: 8080 } tls: - hosts: [acog.example.com] secretName: acog-example-com-tls ``` **Give the ingress a 660-second idle timeout and a 64 MiB body limit.** Common defaults are too tight for a gateway: ingress-nginx stops reading after 60 seconds and refuses bodies over 1 MiB, and an AWS load balancer closes connections idle for 60 seconds. With those defaults, long non-streaming replies and held batch requests are cut off, and large prompts are refused. 660 seconds is above the batch hold window (600 by default) and below 700 seconds, the web tier's keep-alive; a longer idle timeout makes the load balancer reuse connections the web tier has closed, and returns occasional `502`s. With the AWS Load Balancer Controller, use `alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=660`. Every setting is explained in [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md). Keep SMTP credentials and other secrets out of the file: put them in the infra Secret, or manage that Secret yourself with `infra.existingSecret`. **`ACOG_CREDS_KEY` encrypts every stored provider credential.** Back it up. If it is lost, every stored credential must be entered again. ## 3. Render and check Render before you install, and confirm that no operator custom resources appear and that the images are the ones you expect: ```bash helm template aco oci://docker.io/tessarac/ai-cost-optimizer --version "$CHART_VERSION" \ -n aco -f values-selfhost.yaml -f my-install.yaml \ | grep -E '^kind:|image:' | sort -u ``` If you see `kind: Cluster`, `kind: Milvus`, `kind: Dragonfly` or `kind: ClickHouseInstallation`, `ha.enabled` is still on. ## 4. Install Use release name `aco` and namespace `aco` unless you have a reason not to (the database address in the overlay above assumes the release name `aco`). ```bash helm install aco oci://docker.io/tessarac/ai-cost-optimizer \ --version "$CHART_VERSION" \ -n aco --create-namespace \ -f values-selfhost.yaml \ -f my-install.yaml ``` The control plane applies the database schema on start. Wait for it: ```bash kubectl -n aco rollout status deploy/aco-controlplane ``` ## 5. Sign in as the first administrator On a new database, the control plane creates `ACOG_BOOTSTRAP_ADMIN_EMAIL` as an administrator with a random one-time password, and prints it once in its log: ```bash kubectl -n aco logs deploy/aco-controlplane | grep BREAKGLASS ``` Sign in at your console address with it. You are asked to choose your own password immediately. This happens only while no user can sign in; an established installation is never touched. If you are ever locked out, set `ACOG_BOOTSTRAP_FORCE=true` to reset that account's password and print it again, then **set it back to `false` immediately**, or every restart resets the account. ## 6. Load your licence Go to **Admin → Billing**, upload the `.acog-license` file and enter its password. No restart is needed. See [Licensing](https://tessarac.com/docs/acog/self-hosted/licensing.md), including the declarative option for air-gapped and GitOps installs. ## 7. Connect a provider and test Follow the [Quickstart](https://tessarac.com/docs/acog/get-started/quickstart.md) from step 2, with your own address in place of `acog.tessarac.com`. ## Air-gapped installation Nothing at runtime needs the internet, so an air-gapped installation differs only in where images come from: 1. Render the chart with your overlays and collect the image list (`grep image:` as above). 2. Copy each image to your registry with all of its architectures (`crane copy` or `skopeo copy --all`), keeping the same repository names. 3. Point the chart at your registry with `global.registry`, `global.imagePullSecrets` and `infra.bundled..image`, render again and check. Do this again for each new chart version before upgrading. The `helm test` pod uses `curlimages/curl`, which cannot be redirected by a value; mirror it or skip `helm test`. --- # Configuration > The settings every self-hosted ACOG installation needs: secrets, production mode, public address, email, trusted proxies, the semantic cache and DLP models. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/configuration. Last updated: 2026-10-16. ACOG is configured with environment variables, set through the chart's `infra.secret` (secrets) and `services..extraEnv` (everything else). Each service reads its environment once at start, so a change takes effect after a restart (`helm upgrade` restarts what changed). ## Secrets you generate | Variable | Generate with | Notes | |---|---|---| | `ACOG_CREDS_KEY` | `openssl rand -base64 32` | Encrypts every stored provider credential, two-factor secrets, SSO client secrets and the DLP dictionary key. The control plane and data plane must have the same value. **If it is lost or changed, every stored credential must be entered again.** | | `ACOG_OPERATOR_TOKEN` | `openssl rand -hex 32` | Authenticates internal calls between services. If empty, DLP still enforces but records nothing. | | `ACOG_INGEST_TOKEN` | `openssl rand -hex 32` | Authenticates usage records sent to metering. Metering will not start without it in production mode. | | `ACOG_POSTGRES_DSN` | | The PostgreSQL connection string | | `ACOG_CLICKHOUSE_PASSWORD` | | Must match the ClickHouse password | ## Control plane | Variable | Set to | Why | |---|---|---| | `ACOG_ENVIRONMENT` | `production` | The default, `development`, writes account-recovery links to the log instead of mailing them. Production mode requires an email relay and marks the session cookie `Secure`, so the console must be served over HTTPS. Set it on metering too. | | `ACOG_SELF_SERVE_SIGNUP` | `false` | Public sign-up is on by default | | `ACOG_WEB_ORIGIN`, `ACOG_PUBLIC_BASE_URL` | `https://acog.example.com` | Your console's real address. Email links, SSO redirects and passkeys are built from it. | | `ACOG_TRUSTED_PROXY_CIDRS` | Your ingress and load-balancer networks | So per-key IP allowlists see the real client address | | `ACOG_BOOTSTRAP_ADMIN_EMAIL` | The first administrator's email | First install only; see [Install on Kubernetes](https://tessarac.com/docs/acog/self-hosted/kubernetes.md#5-sign-in-as-the-first-administrator) | | `ACOG_CLICKHOUSE_DATABASE` | `acog` (already set) | Leave as is | ### Email | Variable | Notes | |---|---| | `ACOG_EMAIL_PROVIDER` | `smtp` or `ses`. In production the control plane refuses to start without one. | | `ACOG_EMAIL_FROM` | The sender address | | `ACOG_SMTP_HOST`, `ACOG_SMTP_PORT` | Port 587 uses STARTTLS. Set `ACOG_SMTP_TLS` only for implicit TLS on port 465. | | `ACOG_SMTP_USERNAME`, `ACOG_SMTP_PASSWORD` | Put these in the infra Secret, not in a values file | | `ACOG_SES_REGION` | For `ses` | ## Data plane | Variable | Default | Notes | |---|---|---| | `ACOG_TRUSTED_PROXY_CIDRS` | | As for the control plane | | `ACOG_MAX_REQUEST_BYTES` | `67108864` (64 MiB) | Keep in step with the web tier's body limit | | `ACOG_BATCH_HOLD_SECS` | `600` | The longest batch wait served on the caller's held connection; a longer one gets a job handle to poll. The connection is silent while held, so keep this at 600 or below: the web tier waits 660 s, as should your load balancer (see [Install on Kubernetes](https://tessarac.com/docs/acog/self-hosted/kubernetes.md#2-write-your-own-overlay)). Otherwise held batches are cut off. | | `ACOG_RATE_LIMIT_ORG_PER_MIN` | `600` | Default per-organization request limit; `0` disables | | `ACOG_RATE_LIMIT_KEY_PER_MIN` | `300` | Default per-service-key request limit; `0` disables | ## Semantic cache The [semantic cache](https://tessarac.com/docs/acog/optimize/semantic-cache.md) is off on a self-hosted installation until you add a vector store. You need all four of these: ```yaml infra: bundled: milvus: enabled: true minio: # a sibling of milvus, not inside it rootPassword: "" services: dataplane: extraEnv: ACOG_EMBEDDING_ENDPOINT: "http://aco-embedder:8086/embed" # the full path ACOG_MILVUS_URL: "http://aco-milvus:19530" corpusTenancy: semanticNameKeySecret: name: aco-semantic-cache-name-key ``` ```bash kubectl -n aco create secret generic aco-semantic-cache-name-key \ --from-literal=ACOG_SEMANTIC_CACHE_NAME_KEY="$(openssl rand -hex 32)" ``` Milvus brings an etcd and a MinIO with it: three more Deployments and volumes (20, 5 and 20 GiB). Keep the naming key; a different key starts every organization's cache empty again. Each organization then switches the cache on or off under **Admin → Gateway settings**. ## Data loss prevention The chart runs the DLP service and a named-entity model for detecting names and medical terms. The DLP service reports `ner_ready` once the model is loaded; until then those two data types are unavailable and the console says so. ## Keyless provider connections To offer [keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md), set the issuer address. See [Install identity](https://tessarac.com/docs/acog/self-hosted/install-identity.md). | Variable | Notes | |---|---| | `ACOG_WORKLOAD_IDENTITY_ISSUER_BASE` | Your API's public address plus `/v1/identity`. Unset means keyless connections are not offered. | | `ACOG_WORKLOAD_IDENTITY_KEY_SOURCE` | `local` (default): signing keys are generated and stored sealed by `ACOG_CREDS_KEY`. `kms`: AWS KMS. | | `ACOG_WORKLOAD_IDENTITY_KEY_ROTATION` | Default `2160h` (90 days) | | `ACOG_WORKLOAD_IDENTITY_KEY_ACTIVATION_DELAY` | Default `24h`. If you upload key sets to Google or Anthropic by hand, allow days. | --- # Licensing > What the licence switches on, the three offline ways to load it, how to check it, and how renewal works without your installation contacting anyone. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/licensing. Last updated: 2026-10-16. A self-hosted installation always passes traffic. The licence decides whether the savings run: | Licence | State | Requests | |---|---|---| | None, expired, not yet valid, or invalid | **Pass-through** | Forwarded unchanged to your provider and measured. No savings. | | Loaded and within its dates | **Active** | Savings run | The licence covers the whole installation, every organization in it. Licensing is entirely offline. A licence is a signed file, checked against a key built into ACOG; nothing contacts Tessarac, and there is no licence server. ## What you receive A licence file named `.acog-license`, and its password, sent to you separately. Your licence agreement names a renewal contact. ## Loading it Each method checks the file immediately and stores it, replacing any previous licence. No restart is needed: the control plane applies it at once, and the gateway within 30 seconds. 1. **Console.** **Admin → Billing**, upload the file and enter the password. 2. **API.** `POST /v1/license/load` with `{"file_b64": "", "password": ""}` (requires `admin:org`). 3. **Declarative**, for air-gapped and GitOps installations. Create a Secret and reference it in your overlay; the control plane applies it at every start. ```bash kubectl -n aco create secret generic aco-license \ --from-file=license=./.acog-license \ --from-file=password=./licence-password.txt ``` ```yaml license: enabled: true existingSecret: aco-license ``` Without Kubernetes, set `ACOG_LICENSE_FILE` and `ACOG_LICENSE_PASSWORD_FILE` on the control plane. ## Checking it **Admin → Billing** shows the licence's state (Active, Expired, Not yet valid, Invalid or No license), who it is issued to, and when it was issued and expires. `GET /v1/license` (requires `read:billing`) returns the same: ```json { "self_host": true, "state": "active", "effective_billing": "active", "license": { "license_id": "…", "licensee": "…", "issued_at": "…", "expires_at": "…", "days_remaining": 212 } } ``` The gateway status pill in the console turns red with **License expired** or **License invalid** when savings stop for that reason. A licence with no expiry date is perpetual. ### Check the usage export on day one Renewal uses a signed usage report (below). Once, after loading a new licence, confirm it can produce one: ```bash curl -s -o /dev/null -w '%{http_code}\n' https://acog.example.com/api/v1/license/usage-export \ -H "Authorization: Bearer $ACOG_ADMIN_KEY" ``` | Status | Meaning | |---|---| | `200` | The licence can produce a renewal report | | `409` | The licence cannot produce a report and must be re-issued. Contact your renewal contact. | | `501` | The usage store is unreachable; this says nothing about the licence | ## Renewing Renewal is a manual exchange. Nothing leaves your installation on its own. 1. **Admin → Billing → Download usage report** produces a signed JSON file of usage totals and a per-model breakdown. It contains no prompts and no responses. 2. Send it to the renewal contact named in your licence agreement. 3. Load the new licence file you receive. It takes effect at once. With a declarative licence, replace the Secret and restart the control plane (the chart does not restart it when only a Secret changes). ## Removing a licence `DELETE /v1/license` (requires `admin:org`) removes it; the installation returns to pass-through. The console has no control for this. --- # Self-hosted keyless connections > Offer keyless Vertex AI and Anthropic connections from your own installation: its issuer address, uploaded key sets, and the installation's own identity. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/install-identity. Last updated: 2026-10-16. A self-hosted installation offers the same [keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md) as the hosted service, and one more mode, **install identity**. | Mode | Console label | Identity presented to Google or Anthropic | |---|---|---| | `workload_identity` | Keyless (workload identity) | A per-organization issuer that your installation runs | | `ambient` | This install's identity | The identity the installation itself runs as | ## Per-organization keyless connections Your installation becomes an OpenID Connect issuer with one issuer per organization, exactly as on the hosted service. Set its public address on the control plane: ```yaml services: controlplane: extraEnv: ACOG_WORKLOAD_IDENTITY_ISSUER_BASE: "https://acog.example.com/api/v1/identity" ``` Each organization's issuer is then `https://acog.example.com/api/v1/identity/`. Google and Anthropic fetch its public keys from `/.well-known/openid-configuration` and `/jwks.json`, so the address must be reachable from the internet over HTTPS on port 443. Without `ACOG_WORKLOAD_IDENTITY_ISSUER_BASE`, keyless connections are not offered and saving one is refused with that reason. Signing keys are generated by the control plane and stored sealed with `ACOG_CREDS_KEY` (or held in AWS KMS with `ACOG_WORKLOAD_IDENTITY_KEY_SOURCE=kms`). They rotate every 90 days, and each new key is published 24 hours before it starts signing. ### When Google or Anthropic cannot reach your installation If the issuer address is not reachable from the internet, upload the key set instead of letting the provider fetch it. **Admin → Providers → Connect provider → Keyless** shows the key set under **Can't reach the internet? Upload these keys**: - Google: create the workload identity provider with `--jwk-json-path` pointing at the saved key set. - Anthropic: create the federation issuer with an inline JWKS. Then the issuer URL only has to match as text. **You must upload the new key set every time a key rotates**, before it starts signing: the Providers page warns when a new key is pending and shows when it takes effect. Increase `ACOG_WORKLOAD_IDENTITY_KEY_ACTIVATION_DELAY` (default `24h`) to give yourself time. ## Install identity With install identity, a connection uses the identity the installation already runs as, and stores nothing. Any organization on the installation can create such a connection, so **grant the installation's identity only what every organization on it may use.** ### Google (Vertex AI) The data plane and control plane use, in order: 1. `GOOGLE_APPLICATION_CREDENTIALS`, naming a service account key file or an `external_account` (workload identity federation) file whose credential source is a file or URL; 2. otherwise the metadata server: GKE Workload Identity, or the Compute Engine service account. On GKE, bind the chart's Kubernetes service account to a Google service account: ```yaml serviceAccount: annotations: iam.gke.io/gcp-service-account: acog@my-project.iam.gserviceaccount.com ``` Grant that service account `roles/aiplatform.user` on the projects your organizations will use. A connection can still name a service account to impersonate. ### Anthropic The chart can mount a projected Kubernetes service-account token for Anthropic to trust: ```yaml installIdentity: anthropic: enabled: true ``` This mounts a token with audience `https://api.anthropic.com` at `/var/run/secrets/anthropic.com/token`, rotated by Kubernetes. In the Claude Console, register your cluster's service-account issuer as a federation issuer (see Anthropic's [Kubernetes guide](https://platform.claude.com/docs/en/manage-claude/wif-providers/kubernetes)) and create a rule matching the chart's service account. The connection takes the same four identifiers as a keyless connection. ACOG refuses to send any token whose audience is not `https://api.anthropic.com`, so a misconfigured token file never sends the cluster's own credentials. **More than one install-identity Anthropic connection:** each presents the same token, and Anthropic's single-use check (`check_jti`) refuses a token seen before. Turn `check_jti` off on that issuer in the Claude Console. --- # Upgrades > Upgrade a self-hosted ACOG installation safely: back up, read the new overlay, render, upgrade with every values file, and know what a rollback can undo. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/upgrades. Last updated: 2026-10-16. ## Releases A self-hosted release ships the same builds the hosted service runs. Releases are numbered `1.x.y` and published to `oci://docker.io/tessarac/ai-cost-optimizer`; each image is tagged with its version. Versions ending in `-beta.N` are pre-releases, never chosen unless you name them. ## Procedure 1. **Back up PostgreSQL.** Database migrations cannot be reversed. ```bash kubectl -n aco exec deploy/aco-postgres -- pg_dump -U aco -d aco -Fc > aco-$(date +%F).dump ``` 2. **Read the new chart's `values-selfhost.yaml`.** It pins the images that go together for that version. 3. **Render and compare** with your current release, and confirm no operator custom resources appear (see [Install on Kubernetes](https://tessarac.com/docs/acog/self-hosted/kubernetes.md#3-render-and-check)). 4. **Upgrade, passing every values file every time:** ```bash helm upgrade aco oci://docker.io/tessarac/ai-cost-optimizer \ --version -n aco \ -f values-selfhost.yaml \ -f my-install.yaml ``` Do not use `--reuse-values`: it keeps old defaults and is how settings such as `ha.enabled: false` get lost. Keep `ha.enabled: false` and any datastore modes in your own overlay. 5. **Watch the control plane first.** It applies database migrations at start, one at a time, each in a transaction. ```bash kubectl -n aco rollout status deploy/aco-controlplane ``` 6. **Check the rest**, and confirm the licence is still active under **Admin → Billing** (it is stored in PostgreSQL and survives upgrades). The bundled datastores replace their pod when their specification changes, so a datastore is briefly unavailable during such an upgrade. On an air-gapped installation, mirror the new version's images before step 4. ## Rolling back `helm rollback aco -n aco` restores the previous images and configuration. It does not undo database migrations the newer version applied. If the rollback crosses a schema change, also restore the PostgreSQL backup from step 1. ## Datastore upgrades Major-version upgrades of PostgreSQL, ClickHouse or Dragonfly that you run yourself are your own operation; the chart does not perform them. ## Version notes Notes that need action, or change what you see, when you upgrade past a version: | Upgrading past | Note | |---|---| | 1.26.46 | The semantic cache answers a request only from an answer in the same API format. Entries stored before the upgrade are not served, so the cache refills from new traffic. | | 1.26.17 | Releases come only from plain versions such as `1.26.17`; `-beta.N` versions are pre-releases. | | 1.26.16 | The version line moved from 0.x to 1.x (1.26.16 follows 0.26.15). Images are also tagged with the release version. | | 0.26.11 | The default prompt-cache lifetime is five minutes. Connections and keys set to one hour are unchanged. | | 0.26.10 | An auto mode saving is capped at what your default model would have cost. Upgrade metering before, or together with, the data plane. | | 0.26.9 | Do not run 0.26.8's metering: it overstated auto mode savings. Contact support to correct rows recorded with it. | | 0.26.8 | Auto mode measures its saving against your default model. Upgrade the control plane before, or together with, the web console. | | 0.26.7 | One-hour cache writes and fast mode are priced as Anthropic bills them, so recorded costs rise for that traffic. Upgrade metering first. | | 0.26.6 | The semantic cache no longer answers prompts longer than the embedder can read. | | 0.26.5 | Every service restarts once during the upgrade. | | 0.26.1 | With the semantic cache on, create the collection-naming key Secret before upgrading (see [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md#semantic-cache)); the cache starts empty. | | 0.26.0 | Remove `services.analytics` from your values before upgrading; the chart refuses to render while it is present. | --- # Without Kubernetes > Running ACOG on virtual machines: what is and is not supported, the processes to run, the start order and the constraints that matter. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/self-hosted/without-kubernetes. Last updated: 2026-10-16. Kubernetes with the Helm chart is the supported installation. There is no packaged virtual-machine installation (no system packages, systemd units or Compose file), and none is tested. You can derive one from the chart; this page lists what that involves. Talk to us before choosing this path. ## Processes At least ten processes: the web tier, data plane, control plane, metering, DLP, embedder and named-entity service, and PostgreSQL, Dragonfly (or another Redis-protocol store) and ClickHouse. Add Milvus, etcd and MinIO for the semantic cache. ## Start order 1. The datastores. 2. The control plane (it applies the database schema). 3. Metering. 4. The data plane, DLP, embedder and named-entity service. 5. The web tier. ## Constraints - **Run exactly one metering instance.** - The control plane and data plane must share the same `ACOG_CREDS_KEY`. - The web tier needs `ACOG_API_BASE_URL=/api`, and exits if it is missing. Set `ACOG_GATEWAY_URL` only if your gateway has its own address; the console then shows it to your users instead of `/gateway`. - Terminate TLS in front of the web tier, and serve only the web tier publicly. - The ClickHouse database must be named `acog`. - Configuration is read once at start; restart a process to change it. Every setting is described in [Configuration](https://tessarac.com/docs/acog/self-hosted/configuration.md). The chart's templates are the reference for how the processes are wired together. --- # Errors > Every error ACOG itself returns, with its HTTP status, type, extra fields, cause and what to do. Errors from your provider are passed through unchanged. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/reference/errors. Last updated: 2026-10-16. Errors ACOG returns itself use this envelope: ```json { "error": { "type": "model_not_configured", "message": "…" } } ``` Some types add fields, listed below. An error from your provider is passed through with its own status and body, unless ACOG maps it to one of the types below. On native routes, provider errors are always passed through unchanged. Failed responses are never cached. ## Authentication and authorization | Status | `type` | Cause | What to do | |---|---|---|---| | 401 | `unauthorized` | No gateway key found, or the key is unknown, disabled, expired, or used from outside its IP allowlist | Check the key and the header it is in ([Base URLs and authentication](https://tessarac.com/docs/acog/api/base-urls-and-authentication.md#authentication)) | | 403 | `forbidden` | The key lacks the gateway access scope (`proxy`); or, on `/bedrock`, the request signature does not verify | Use a key with `proxy`; for Bedrock, use an issued ACOG credential and a correct clock | | 503 | `service_unavailable` | ACOG could not reach its key or configuration store | Retry | ## Governance | Status | `type` | Extra fields | Cause | |---|---|---|---| | 402 | `budget_exhausted` | `scope` (`org`, `group`, `user`), `window` (`day`, `week`, `month`), `decision_id` | A budget is used up. See [Budgets](https://tessarac.com/docs/acog/govern/budgets.md). | | 429 | `rate_limited` | `Retry-After` header | A rate limit is reached. See [Rate limits](https://tessarac.com/docs/acog/govern/rate-limits.md). | | 403 | `model_not_allowed` | `scope` (`org`, `team`, `user`, `service-key`) | A model access rule refuses the model. See [Model access](https://tessarac.com/docs/acog/govern/model-access.md). | | 403 | `model_unverifiable` | `channel` | A native-route request whose model cannot be determined, while a model access rule applies | | 403 | `dlp_blocked` | `category`, `reason`, `decision_id` | Data loss prevention blocked the request. See [Data loss prevention](https://tessarac.com/docs/acog/govern/data-loss-prevention.md). | | 400 | `prompt_injection_blocked` | `category` | Prompt injection screening blocked the request. See [Prompt injection protection](https://tessarac.com/docs/acog/govern/prompt-injection.md). | ## Routing and the request | Status | `type` | Cause | What to do | |---|---|---|---| | 409 | `model_not_configured` | No connection declares the model; `auto` with no connections; an `x-acog-channel` value with no matching connection; or no `model` in the body | Declare the model on a connection under **Admin → Providers** | | 409 | `connection_not_configured` | A native route with no connection for that provider, or a connection missing a required setting | Create or complete the connection | | 409 | `connection_not_usable` | The connection's address, port or region is not allowed on the hosted service | Edit the connection ([Connecting providers](https://tessarac.com/docs/acog/providers/overview.md#hosted-service-what-acog-can-reach)) | | 400 | `invalid_request_error` | The body is not valid for the endpoint, a referenced image could not be fetched, or a native path contains `.` or `..` segments | Fix the request | | 413 | `request_too_large` | The body is over 64 MiB | Send less, or reference large files by URL | | 413 | `context_window_exceeded` | The prompt is larger than the model's context window. Extra fields: `model`, `context_window`, `estimated_tokens`. | Shorten the prompt or use a model with a larger window | | 422 | `translation_unsupported` | A field cannot be translated faithfully to the serving model's API | Remove the field, or call a model that speaks the same API | | 422 | `embeddings_unsupported_shape` | No connection that declares the model can serve embeddings | Declare the model on an OpenAI, Azure OpenAI or compatible connection | ## Your provider | Status | `type` | Extra fields | Cause | What to do | |---|---|---|---|---| | 422 | `model_access_required` | `provider_status`, `provider_detail` | Your provider account cannot use the model (Bedrock model access, Vertex Model Garden, Azure deployment) | Enable the model in your provider account; the message says how | | 422 | `upstream_auth_failed` | `provider_status`, `provider_detail` | Your provider rejected the connection's stored key | Update the key under **Admin → Providers** | | 502 | `upstream_auth_error` | | ACOG could not obtain a credential for the provider, for example a keyless exchange was refused | See the provider's troubleshooting section ([Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md#troubleshooting)) | | 500 | `credentials_error` | | A stored credential could not be used | Contact your administrator; re-enter the connection's credentials | | 502 | `upstream_error` | | A network or protocol failure reaching the provider | Retry | | 503 | `upstream_unavailable` | `Retry-After: 5` | The provider's platform is failing repeatedly and ACOG is pausing requests to it | Retry after the delay | ## Batches | Status | `type` | Cause | |---|---|---| | 404 | `batch_not_found` | No such batch job for your organization | | 502 | `batch_failed` | The provider's batch job failed. Extra fields: `id`, `status`. | | 502 | `batch_poll_failed`, `batch_connection_unavailable` | ACOG could not check the job, or the job's connection no longer exists | ## Hosted service: errors from the edge These apply only to the console's address, `acog.tessarac.com/gateway`, which goes through the edge network in front of the console. There, a request that sends nothing back for 60 seconds is closed with `504`, and some `403`, `404` and `5xx` responses arrive as an HTML page from the edge rather than the JSON envelope above. `gateway.acog.tessarac.com` has no edge in front of it; use it, and see [Base URLs](https://tessarac.com/docs/acog/api/base-urls-and-authentication.md#base-urls). --- # Request headers > Every request header ACOG reads: authentication, routing, batching, caching, labels, tracing, sessions and fallback control. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/reference/request-headers. Last updated: 2026-10-16. Headers not listed here, such as `anthropic-version` and `anthropic-beta`, are forwarded to your provider. ## Authentication | Header | Where | Notes | |---|---|---| | `Authorization: Bearer acog_…` | Compatible endpoints, `/openai`, `/vertex`, `/custom`, management API | | | `x-acog-key: acog_…` | Everywhere | | | `x-api-key: acog_…` | `/anthropic`; also compatible endpoints when the value starts with `acog_` | | | `api-key: acog_…` | `/azure`; also compatible endpoints when the value starts with `acog_` | | ## Routing and execution | Header | Values | Effect | |---|---|---| | `x-acog-channel` | `direct`, `bedrock`, `vertex`, `azure`, `custom` | Pin the request to one platform. See [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md#pin-a-request-to-one-platform-x-acog-channel). | | `x-acog-batch` | `defer`, optionally `; max-wait=` | Run at batch prices. See [Batches](https://tessarac.com/docs/acog/api/batches.md). | | `x-acog-policy-fallback` | `off` | Do not retry a safety-classifier refusal on the fallback model | ## Caching | Header | Values | Effect | |---|---|---| | `x-acog-cache-control` | `no-cache`, `no-store`, `ttl=`, `namespace=` (comma-separated) | Control the response cache. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#controlling-the-response-cache-per-request). | | `x-acog-cache-ttl` | `auto`, `in_memory`, `5m`, `30m`, `1h`, `24h`, `forever` | Provider prompt-cache lifetime for this request. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-lifetime). | ## Attribution and tracing | Header | Effect | |---|---| | `x-acog-label` | A free-form label for reports, such as a ticket number or cost center. Up to 128 characters. | | `x-acog-trace-tags` | Comma-separated tags added to this request's trace | | `x-acog-trace-metadata` | A JSON object added to this request's trace | | `x-request-id` | Your own request ID, echoed in the response and recorded | The application a request is attributed to comes from its key, never from a header. ## Conversations (auto mode) | Header | Effect | |---|---| | `x-acog-session-id`, `x-session-id`, `x-claude-code-session-id` | Identify a conversation, so auto mode keeps it on a stable model | | `x-acog-agent-id`, `x-agent-id`, `x-claude-code-agent-id` | Identify the agent or sub-agent within a conversation | --- # Response headers > The headers ACOG adds to responses: the model that answered, cache hits, budget warnings, fallbacks and features removed for compatibility. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/reference/response-headers. Last updated: 2026-10-16. Your provider's own response headers are passed through. ACOG adds: | Header | Value | When | |---|---|---| | `x-request-id` | The request's ID | Always | | `x-acog-resolved-model` | The model that was sent the request | Compatible endpoints, including cache hits. Differs from your request for `auto` or a version alias. | | `x-acog-cache` | `hit` | The response came from the response cache, the semantic cache or a shared in-flight request | | `x-acog-cache-model` | A model id | A semantic-cache hit on a compatible endpoint: the model that generated the stored answer | | `x-acog-budget-notice` | `scope/window=pct`, comma-separated, for example `org/month=80` | A budget passed 50, 80 or 90% of its limit with this response | | `x-acog-policy-fallback` | `applied`, `skipped` or `unavailable` | The provider's safety classifier refused the request. See [Routing and failover](https://tessarac.com/docs/acog/providers/routing-and-failover.md#policy-block-fallback). | | `x-acog-thinking-stripped` | `true` | The serving model rejected extended thinking, and ACOG retried without it | | `x-acog-beta-stripped` | The rejected `anthropic-beta` values, or `all` | The serving platform rejected beta features, and ACOG retried without them | | `x-acog-fields-stripped` | Comma-separated field names | The serving model rejected these request fields, and ACOG retried without them | | `Retry-After` | Seconds | With `429 rate_limited` and `503 upstream_unavailable` | Streaming responses also carry a keep-alive comment line (`: keep-alive`) every 15 seconds without data. It is part of the event stream, not a header. ACOG adds no header with costs or savings. Read them from the console or the [management API](https://tessarac.com/docs/acog/api/management-api.md#usage-and-savings). --- # Roles and permissions > The twelve permissions, the built-in admin, auditor and member roles, custom roles, and how permissions apply to API keys. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/reference/roles-and-permissions. Last updated: 2026-10-16. Everything a person or an API key can do in ACOG is governed by one set of permissions. Roles are named sets of permissions; an API key's scopes are permissions too. ## Permissions | Permission | Console label | Allows | |---|---|---| | `proxy` | Gateway access | Send AI requests through the gateway | | `read:metrics` | Dashboards | View usage and savings dashboards | | `read:billing` | Billing | View invoices and billing status; view governance settings and the audit log | | `admin:users` | Team & roles | Invite, remove and manage members and roles | | `admin:keys` | API keys | Create and revoke API keys for anyone in the organization; manage applications | | `admin:providers` | Provider connections | Connect and disconnect AI provider accounts; cost policies | | `admin:sso` | Single sign-on | Configure single sign-on identity providers | | `admin:teams` | Teams | Create teams and manage their membership | | `admin:budgets` | Budgets & limits | Set budgets, rate limits, model access and guardrails | | `admin:security` | Data loss prevention | Manage data loss prevention rules, connect source code, change prompt security | | `audit:conversations` | Activity | See every request made through the gateway and by whom, including stored conversations | | `admin:org` | Organization profile | Edit the organization's profile, gateway settings, webhooks and observability | ## Built-in roles | Role | Permissions | |---|---| | **Admin** | All twelve | | **Auditor** | `read:metrics`, `read:billing` | | **Member** | `read:metrics`, `proxy` | Built-in roles cannot be edited. An organization always has at least one admin. ## Custom roles Under **Admin → Users → Roles**, create a role from any combination of permissions, except `audit:conversations`. A custom role cannot be deleted while members hold it. ## Granting is bounded You can give a role, or create a key, only with permissions you hold yourself. A refused grant returns `403` naming the permission you lack. ## API keys A key's scopes are permissions, `proxy` by default. A key never has more permissions than its creator, and `audit:conversations` is never available to a key: reading other people's conversations requires a person signed in to the console. ## Single sign-on People who sign in through single sign-on get a role from your identity provider's groups. See [Single sign-on](https://tessarac.com/docs/acog/administer/single-sign-on.md#3-map-groups-to-roles). --- # Glossary > The terms used across the ACOG documentation and console, from active and auto mode to wire shape and workload identity. Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/reference/glossary. Last updated: 2026-10-16. **Active.** An organization whose savings are switched on. Compare *pass-through*. **Application.** A name for the traffic one of your systems sends. Requests are attributed to the application of the service key they use. **Auto mode.** Routing a `model: "auto"` request to the cheapest model strong enough for it. **Batch tier.** A provider's discounted, slower API for requests that can wait. **BYOK (bring your own key).** ACOG calls providers only with your organization's own accounts and credentials. **Channel.** The platform a connection uses: `direct`, `bedrock`, `vertex`, `azure` or `custom`. **Compatible endpoints.** `/v1/chat/completions`, `/v1/messages`, `/v1/embeddings`, `/v1/models` and `/responses`: the OpenAI- and Anthropic-compatible API, which works with any connected model. **Connection.** Credentials and settings for one platform and model family in your organization. **Declared model.** A model a connection says it serves. Requests are routed by declared model. **Default connection, default model.** The connection that serves `auto` (with auto mode off) and the models it declares; the model it uses for `auto`. **Disposition.** How a request was served: exact cache, semantic cache, dedup, optimized, batch or pass-through. **Gainshare.** On the hosted service, the share of measured savings you pay. **Gateway key.** An `acog_…` key that authenticates requests to the gateway and the management API. **Install identity.** On a self-hosted installation, a keyless connection that uses the identity the installation already runs as. **Issuer.** The OpenID Connect identity ACOG runs for each organization, which your Google Cloud project or Anthropic organization trusts for keyless connections. **Keyless connection.** A Vertex AI or Anthropic connection that stores no credential and authenticates with workload identity federation. **Native routes.** `/anthropic`, `/openai`, `/vertex`, `/bedrock`, `/azure` and `/custom`: each provider's own API, forwarded without translation. **Pass-through.** An organization whose requests are forwarded unchanged at full price, with no savings and no charge. **Provider prompt caching.** A provider billing repeated prompt prefixes at a lower rate. ACOG adds the markers that turn it on. **Response cache.** Answering an identical request from a stored response. **Semantic cache.** Answering a near-identical prompt from a stored answer, above a similarity threshold. **Service key.** A gateway key for an application rather than a person. **Translation.** Converting a request and its response between the OpenAI, Anthropic and Gemini APIs. **Wire shape.** The API format of a request or response: OpenAI Chat Completions, Anthropic Messages or Gemini. **Workload identity federation.** Trusting an external identity's short-lived tokens instead of a stored key. Google Cloud and Anthropic both support it.