Batches

Every major provider runs a batch tier: the same model and the same answer, at about half the price, in exchange for waiting. ACOG lets you send any single request to that tier without learning the provider’s batch API. You opt in per request; ACOG submits the batch job, waits for it, and gives you an ordinary response.

ACOG can also decide this for you on requests nobody is waiting for. See Batch routing.

Opt in

Either add a header:

x-acog-batch: defer; max-wait=2h

or add :batch to the model, for clients that cannot set headers:

{ "model": "claude-haiku-4-5:batch", "max_tokens": 256, "messages": [ … ] }

Setting	Meaning
defer	Required in the header: run this request on the batch tier
max-wait=<duration>	How long you can wait for the answer, for example 45s, 30m or 2h. Default 1h. A value ACOG cannot read falls back to the default.

The :batch suffix uses the default wait. If you send both, the header wins.

How you get the answer

Wait	Delivery
Within the hold window (50 seconds on the hosted service; 10 minutes by default on a self-hosted installation)	ACOG keeps your connection open and returns the answer on it, as a normal response
Longer	ACOG returns 202 Accepted at once with a job handle, and you poll for the answer

The job handle:

{
  "id": "acogbatch_18df08b9e2364f83f2825dbe0c564531926d21a9046fcf58",
  "object": "acog.batch",
  "status": "in_progress",
  "model": "claude-haiku-4-5-20251001",
  "created_at_ms": 1792160771173,
  "deadline_secs": 7200
}

Poll for the result

GET https://gateway.acog.tessarac.com/v1/acog/batches/{id}

curl -s https://gateway.acog.tessarac.com/v1/acog/batches/acogbatch_18df08b9e2364f83f2825dbe0c564531926d21a9046fcf58 \
  -H "Authorization: Bearer $ACOG_API_KEY"

Response	Meaning
200 with the job handle (status: "in_progress")	Still running. Poll again later; once a minute is plenty.
200 with the model’s response	Done. The body is the response you would have received synchronously, in the shape you asked in.
404 batch_not_found	No such job for your organization
502 batch_failed	The provider’s batch job failed

A job is visible only to keys of the organization that created it.

Which connections support it

Connection	Batch tier
Direct Anthropic	Message Batches. Works with API-key and keyless connections.
Direct OpenAI	Batch API
AWS Bedrock	Batch inference jobs, when the connection has batch storage configured. Always delivered as a job handle.
Google Vertex AI	Batch prediction jobs, when the connection has batch storage configured. Always delivered as a job handle.
Azure OpenAI, custom endpoints	None. The request runs synchronously at the normal price.

A request ACOG cannot run on the batch tier (a streaming request, a connection with no batch tier, a request using a feature the batch API does not accept) is answered synchronously, at the normal price, rather than refused.

Pricing

A batched request is priced at the provider’s batch rate, roughly half the synchronous rate, and is reported with the disposition Batch. ACOG checks first that the batch tier really is cheaper for your request: a request whose prompt would mostly be read from the provider’s prompt cache can be cheaper synchronously, and is then sent synchronously.

Provider batch APIs directly

To use a provider’s own batch API (for example to submit thousands of requests in one Anthropic Message Batch), call it on the native route, such as POST https://gateway.acog.tessarac.com/anthropic/v1/messages/batches.