Rate limits
Cap requests and tokens per minute for the organization, an application, a team, a person or a service key.
Rate limits cap how fast requests and tokens can flow through ACOG, independently of your providers’ own limits.
Scopes
| Scope | Default (requests per minute) |
|---|---|
| Organization | 600 |
| Application | No limit |
| Team | No limit |
| Person | No limit |
| Service key | 300 |
Each scope has two limits: requests per minute (RPM) and tokens per minute (TPM). Token limits have no default.
Every limit that applies to a request is enforced, and the first one reached refuses it. Limits for people are set on the person, never on one of their keys; limits for a single key are set only on service keys.
Setting limits
Open Admin → Governance → Spending & limits → Rate limits. Changing limits requires Set spend budgets, rate limits, model access, and guardrails (admin:budgets). Leave a field at 0 to keep the default. Changes apply from the next request.
When a limit is reached
{
"error": {
"type": "rate_limited",
"message": "request rate limit exceeded for this org; retry after 12s"
}
}The status is 429, with a Retry-After header in seconds. A refused request does not count toward the limit. The webhook event governance.rate_limit_exceeded is sent at most once a minute per caller and rule.
A 429 can also come from your provider, when your provider account’s own limit is reached. Its body is the provider’s own error. Declaring the model on a second connection lets ACOG spread load and fail over (see Routing and failover).
If limits cannot be checked
If ACOG cannot read its rate counters for a moment, requests are allowed rather than blocked.