Google Vertex AI

A Vertex AI connection calls models in your own Google Cloud project. Claude models (the anthropic family, called with rawPredict) and Gemini models (the gemini family, called with generateContent) are separate connections on the same project.

Method	What ACOG stores	Best for
Service account key	A service account key file (JSON), encrypted	Getting started quickly
Keyless (workload identity)	The workload identity provider’s name, and optionally a service account to impersonate. No secret.	Production. Many organizations forbid service account keys entirely.

Before you start, enable the Vertex AI API in the project and enable the models you want in Model Garden.

Connect with a service account key

1. In Google Cloud, create a service account and grant it Vertex AI User (roles/aiplatform.user) on the project. Google now labels this role Agent Platform User in some views.
2. Create a JSON key for it.
3. In ACOG, open Admin → Providers → Connect provider, choose Vertex and the model family, select Service account key, and upload or paste the key file.
4. Enter the project ID and region, then review the models and choose a default.

To let ACOG list the project’s models, also grant Service Usage Consumer (roles/serviceusage.serviceUsageConsumer). Without it, requests still work, and the console offers ACOG’s own model catalog instead and says so.

Connect keyless

A keyless connection uses Google workload identity federation. Your project trusts an identity issuer that ACOG runs for your ACOG organization. For each exchange, ACOG signs a short-lived token for your organization, exchanges it at Google’s Security Token Service, and calls Vertex AI with the resulting access token. Read Keyless connections for how this works.

1. Get your organization’s identity from ACOG

In ACOG, open Admin → Providers → Connect provider, choose Vertex and the model family, and select Keyless (workload identity). The console shows your issuer URL and subject, and the exact commands below with your values filled in.

2. Create the pool and provider in Google Cloud

Replace PROJECT with your project ID, NUMBER with your project number, and ISSUER and SUBJECT with the values from the console:

gcloud iam workload-identity-pools create acog \
  --location=global --project=PROJECT

gcloud iam workload-identity-pools providers create-oidc acog-gateway \
  --location=global --workload-identity-pool=acog --project=PROJECT \
  --issuer-uri="ISSUER" \
  --attribute-mapping="google.subject=assertion.sub" \
  --attribute-condition="assertion.sub == 'SUBJECT'"

The attribute condition is what limits the provider to your ACOG organization. Every ACOG organization has its own issuer as well, so another organization’s tokens cannot match your provider even without the condition.

3. Grant access

Grant the federated identity Vertex AI access directly:

gcloud projects add-iam-policy-binding PROJECT \
  --role=roles/aiplatform.user \
  --member="principal://iam.googleapis.com/projects/NUMBER/locations/global/workloadIdentityPools/acog/subject/SUBJECT"

Or, if your policies require a service account, grant the service account roles/aiplatform.user, grant the federated identity Workload Identity User (roles/iam.workloadIdentityUser) on that service account, and enter the service account’s email in the connection. ACOG then impersonates it.

Optionally grant roles/serviceusage.serviceUsageConsumer to the same identity so ACOG can list the project’s models.

4. Finish the connection in ACOG

Enter the workload identity provider’s full name:

projects/NUMBER/locations/global/workloadIdentityPools/acog/providers/acog-gateway

Enter the project ID, region and (optionally) the service account email, then save. ACOG performs an exchange and lists your models to confirm the setup.

Removing access

Disable or delete the workload identity provider (or the pool), or remove the IAM binding. Access tokens ACOG already holds expire within an hour.

Region: where requests are processed

The connection’s region decides where Google processes your requests:

Region	Meaning
global	Google may serve the request from any region. No data residency.
us	Google keeps processing inside the United States (its jurisdictional endpoint).
eu	Google keeps processing inside the European Union.
A single region, such as us-east5	Processing in that region.

Some models are served only in particular locations. Model regions pins a model to its own location, which must be inside the connection’s region: global contains every location; us contains every us-* region; eu contains the regions in EU member states (not London europe-west2 or Zurich europe-west6); a region in an EU member state contains the other regions in the same country; any other region contains only itself. Saving a model region outside the connection’s region is refused.

When Vertex AI answers that a model is not served at its location, ACOG retries it elsewhere, but only where the pin allows:

Pinned to	Retried in
us or eu	Nowhere else
A us-* region	us-east5, us-central1, us
A region in an EU member state	Only regions in the same country
global	us-east5, us-central1, europe-west1, europe-west4, asia-southeast1, us, eu
Any other region	Nowhere else

A location found to serve a model is remembered for 30 days. A model served nowhere it may go returns Vertex’s own error, and ACOG answers that error immediately for 15 minutes before trying again.

Keeping traffic in one country

A connection’s region applies to that connection only. Another connection that declares the same model can serve it elsewhere through failover, cost arbitrage, an x-acog-channel header or a DLP route restriction. To keep all of an organization’s traffic in one country or in the EU:

• make a connection pinned there your default connection;
• pin every other connection that can serve the same models inside the same boundary, including the one auto mode’s routing judge uses;
• leave cost arbitrage off;
• do not declare those models on connections that cannot be pinned (Direct, Azure OpenAI, Custom).

Other settings

Setting	Notes
Project ID	The project that is billed and whose quota is used
Endpoint override	Optional. On the hosted service, only Google’s own Vertex AI hosts: https://aiplatform.googleapis.com, https://aiplatform.us.rep.googleapis.com, https://aiplatform.eu.rep.googleapis.com or https://<location>-aiplatform.googleapis.com. Self-hosted, any HTTPS URL, such as a Private Service Connect endpoint.
Batch storage	Optional gs://bucket/path where batch jobs read and write. With it, ACOG can run requests nobody is waiting for as Vertex batch jobs at about half price. The connection’s identity needs read and write access to the bucket. See Batch routing.
Cache lifetime	How long cached context is kept, where the model supports a choice. See Caching.

Calling Vertex AI through ACOG

Call your models through the Chat Completions or Messages APIs, or use Vertex AI’s own API through the native route https://gateway.acog.tessarac.com/vertex. See Native provider routes.

Troubleshooting

What you see	Usual cause	Fix
google refused the credentials (invalid_target)	The provider name is wrong, or the pool or provider is disabled	Copy the provider’s full resource name from Google Cloud and check that it is enabled
google refused the credentials (invalid_grant)	The provider’s issuer URL or attribute condition does not match your ACOG values	Compare them with the console
403 from Vertex AI after a successful exchange	The identity lacks roles/aiplatform.user, or the model is not enabled in Model Garden	Grant the role, enable the model
422 model_access_required	The model is not enabled for the project	Enable it in Model Garden
409 connection_not_usable	A saved region or endpoint is not allowed on the hosted service	Edit the connection to use a Vertex location and a Google host
The model list says it is showing ACOG’s catalog	The identity cannot list models	Optional: grant roles/serviceusage.serviceUsageConsumer