# Google Vertex AI

> Connect Claude and Gemini on Vertex AI with a service account key or keyless with workload identity federation, and control where requests run.

Product: AI Cost Optimizer Gateway (ACOG). Web version: https://tessarac.com/docs/acog/providers/google-vertex-ai. Last updated: 2026-10-16.

A Vertex AI connection calls models in your own Google Cloud project. Claude models (the `anthropic` family, called with `rawPredict`) and Gemini models (the `gemini` family, called with `generateContent`) are separate connections on the same project.

| Method | What ACOG stores | Best for |
|---|---|---|
| **Service account key** | A service account key file (JSON), encrypted | Getting started quickly |
| **Keyless (workload identity)** | The workload identity provider's name, and optionally a service account to impersonate. No secret. | Production. Many organizations forbid service account keys entirely. |

Before you start, enable the Vertex AI API in the project and enable the models you want in **Model Garden**.

## Connect with a service account key

1. In Google Cloud, create a service account and grant it **Vertex AI User** (`roles/aiplatform.user`) on the project. Google now labels this role **Agent Platform User** in some views.
2. Create a JSON key for it.
3. In ACOG, open **Admin → Providers → Connect provider**, choose **Vertex** and the model family, select **Service account key**, and upload or paste the key file.
4. Enter the project ID and region, then review the models and choose a default.

To let ACOG list the project's models, also grant **Service Usage Consumer** (`roles/serviceusage.serviceUsageConsumer`). Without it, requests still work, and the console offers ACOG's own model catalog instead and says so.

## Connect keyless

A keyless connection uses [Google workload identity federation](https://cloud.google.com/iam/docs/workload-identity-federation). Your project trusts an identity issuer that ACOG runs for your ACOG organization. For each exchange, ACOG signs a short-lived token for your organization, exchanges it at Google's Security Token Service, and calls Vertex AI with the resulting access token. Read [Keyless connections](https://tessarac.com/docs/acog/providers/keyless-connections.md) for how this works.

### 1. Get your organization's identity from ACOG

In ACOG, open **Admin → Providers → Connect provider**, choose **Vertex** and the model family, and select **Keyless (workload identity)**. The console shows your issuer URL and subject, and the exact commands below with your values filled in.

### 2. Create the pool and provider in Google Cloud

Replace `PROJECT` with your project ID, `NUMBER` with your project number, and `ISSUER` and `SUBJECT` with the values from the console:

```bash
gcloud iam workload-identity-pools create acog \
  --location=global --project=PROJECT

gcloud iam workload-identity-pools providers create-oidc acog-gateway \
  --location=global --workload-identity-pool=acog --project=PROJECT \
  --issuer-uri="ISSUER" \
  --attribute-mapping="google.subject=assertion.sub" \
  --attribute-condition="assertion.sub == 'SUBJECT'"
```

The attribute condition is what limits the provider to your ACOG organization. Every ACOG organization has its own issuer as well, so another organization's tokens cannot match your provider even without the condition.

### 3. Grant access

Grant the federated identity Vertex AI access directly:

```bash
gcloud projects add-iam-policy-binding PROJECT \
  --role=roles/aiplatform.user \
  --member="principal://iam.googleapis.com/projects/NUMBER/locations/global/workloadIdentityPools/acog/subject/SUBJECT"
```

Or, if your policies require a service account, grant the service account `roles/aiplatform.user`, grant the federated identity **Workload Identity User** (`roles/iam.workloadIdentityUser`) on that service account, and enter the service account's email in the connection. ACOG then impersonates it.

Optionally grant `roles/serviceusage.serviceUsageConsumer` to the same identity so ACOG can list the project's models.

### 4. Finish the connection in ACOG

Enter the workload identity provider's full name:

```text
projects/NUMBER/locations/global/workloadIdentityPools/acog/providers/acog-gateway
```

Enter the project ID, region and (optionally) the service account email, then save. ACOG performs an exchange and lists your models to confirm the setup.

### Removing access

Disable or delete the workload identity provider (or the pool), or remove the IAM binding. Access tokens ACOG already holds expire within an hour.

## Region: where requests are processed

The connection's **region** decides where Google processes your requests:

| Region | Meaning |
|---|---|
| `global` | Google may serve the request from any region. No data residency. |
| `us` | Google keeps processing inside the United States (its jurisdictional endpoint). |
| `eu` | Google keeps processing inside the European Union. |
| A single region, such as `us-east5` | Processing in that region. |

Some models are served only in particular locations. **Model regions** pins a model to its own location, which must be inside the connection's region: `global` contains every location; `us` contains every `us-*` region; `eu` contains the regions in EU member states (not London `europe-west2` or Zurich `europe-west6`); a region in an EU member state contains the other regions in the same country; any other region contains only itself. Saving a model region outside the connection's region is refused.

When Vertex AI answers that a model is not served at its location, ACOG retries it elsewhere, but only where the pin allows:

| Pinned to | Retried in |
|---|---|
| `us` or `eu` | Nowhere else |
| A `us-*` region | `us-east5`, `us-central1`, `us` |
| A region in an EU member state | Only regions in the same country |
| `global` | `us-east5`, `us-central1`, `europe-west1`, `europe-west4`, `asia-southeast1`, `us`, `eu` |
| Any other region | Nowhere else |

A location found to serve a model is remembered for 30 days. A model served nowhere it may go returns Vertex's own error, and ACOG answers that error immediately for 15 minutes before trying again.

### Keeping traffic in one country

A connection's region applies to that connection only. Another connection that declares the same model can serve it elsewhere through failover, cost arbitrage, an `x-acog-channel` header or a DLP route restriction. To keep all of an organization's traffic in one country or in the EU:

- make a connection pinned there your default connection;
- pin every other connection that can serve the same models inside the same boundary, including the one auto mode's routing judge uses;
- leave cost arbitrage off;
- do not declare those models on connections that cannot be pinned (Direct, Azure OpenAI, Custom).

## Other settings

| Setting | Notes |
|---|---|
| Project ID | The project that is billed and whose quota is used |
| Endpoint override | Optional. On the hosted service, only Google's own Vertex AI hosts: `https://aiplatform.googleapis.com`, `https://aiplatform.us.rep.googleapis.com`, `https://aiplatform.eu.rep.googleapis.com` or `https://<location>-aiplatform.googleapis.com`. Self-hosted, any HTTPS URL, such as a Private Service Connect endpoint. |
| Batch storage | Optional `gs://bucket/path` where batch jobs read and write. With it, ACOG can run requests nobody is waiting for as Vertex batch jobs at about half price. The connection's identity needs read and write access to the bucket. See [Batch routing](https://tessarac.com/docs/acog/optimize/batch-routing.md). |
| Cache lifetime | How long cached context is kept, where the model supports a choice. See [Caching](https://tessarac.com/docs/acog/optimize/caching.md#cache-lifetime). |

## Calling Vertex AI through ACOG

Call your models through the [Chat Completions](https://tessarac.com/docs/acog/api/chat-completions.md) or [Messages](https://tessarac.com/docs/acog/api/messages.md) APIs, or use Vertex AI's own API through the native route `https://gateway.acog.tessarac.com/vertex`. See [Native provider routes](https://tessarac.com/docs/acog/api/native-routes.md).

## Troubleshooting

| What you see | Usual cause | Fix |
|---|---|---|
| `google refused the credentials (invalid_target)` | The provider name is wrong, or the pool or provider is disabled | Copy the provider's full resource name from Google Cloud and check that it is enabled |
| `google refused the credentials (invalid_grant)` | The provider's issuer URL or attribute condition does not match your ACOG values | Compare them with the console |
| `403` from Vertex AI after a successful exchange | The identity lacks `roles/aiplatform.user`, or the model is not enabled in Model Garden | Grant the role, enable the model |
| `422 model_access_required` | The model is not enabled for the project | Enable it in Model Garden |
| `409 connection_not_usable` | A saved region or endpoint is not allowed on the hosted service | Edit the connection to use a Vertex location and a Google host |
| The model list says it is showing ACOG's catalog | The identity cannot list models | Optional: grant `roles/serviceusage.serviceUsageConsumer` |