docs(proxy): document Vertex AI backend setup, env vars, aliases, native passthrough (#2422)

## Description

Documents the Vertex AI proxy backend properly, fixing #2393. Following
the docs verbatim (`pip install "headroom-ai[proxy]"` + `headroom proxy
--backend vertex_ai`) currently fails with `vertexai import failed`, and
the LiteLLM-specific `VERTEXAI_PROJECT`/`VERTEXAI_LOCATION` env vars are
documented nowhere — risking requests silently resolving against the ADC
default quota project and billing the wrong GCP project.

All documented behavior was verified against source: alias normalization
in `headroom/providers/registry.py`
(`vertex`/`google-vertex`/`googlevertex` → `vertex_ai`), the
always-registered native publisher passthrough routes in
`headroom/providers/proxy_routes.py`, and `pyproject.toml` (no extra
pulls in `google-cloud-aiplatform`).

## Type of Change

- [ ] Bug fix
- [ ] New feature
- [x] Documentation update
- [ ] Refactor
- [ ] Other

## Changes Made

- `docs/content/docs/proxy.mdx`: new **Google Vertex AI** subsection
under Cloud providers — `google-cloud-aiplatform>=1.38` requirement (not
in any extra or Docker image), `VERTEXAI_PROJECT`/`VERTEXAI_LOCATION`
env vars with a warning about silent ADC quota-project fallback and
their distinction from the standard
`GOOGLE_CLOUD_PROJECT`/`GOOGLE_CLOUD_LOCATION` vars, backend name alias
equivalence (`vertex_ai` / `vertex` / `google-vertex` / `googlevertex` /
`litellm-vertex` / `litellm-vertex_ai`), and cross-links to the Claude
Code on Vertex page and the LiteLLM callback page.
- `docs/content/docs/proxy.mdx`: new **Native Vertex passthrough
routes** subsection documenting the unconditionally registered
`/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:*`
routes and the `publisher=google` (Gemini handler) vs
`publisher=anthropic` (LiteLLM-Vertex path) branching.
- `docs/content/docs/installation.mdx`: added `VERTEXAI_PROJECT` and
`VERTEXAI_LOCATION` rows to the LLM provider keys table, plus a pointer
to the new Vertex section for the SDK dependency.
- `docs/content/docs/litellm.mdx`: cross-reference callout
distinguishing the LiteLLM callback integration from the proxy's
`litellm-*` backends (issue gap #5).

## Testing

- [x] Docs build passes locally

**Test Output**

```
$ npm run build          # docs/ — same as CI validate-nextjs
✓ Static + SSG pages generated (exit code 0), /docs/proxy, /docs/installation, /docs/litellm prerendered

$ mkdocs build           # same as CI validate-mkdocs
INFO    -  Documentation built in 8.32 seconds
```

## Real Behavior Proof

- Environment: Windows 11, Node 20, npm 10, Python 3.13, mkdocs-material
(latest), branch `docs/2393-vertex-ai-backend` off `upstream/main`.
- Exact command / steps: `cd docs && npm ci && npm run build`; `mkdocs
build` from repo root; manually re-verified each documented claim
against `headroom/providers/registry.py` (alias normalization),
`headroom/providers/proxy_routes.py` (publisher passthrough routes), and
`pyproject.toml` `[project.optional-dependencies]` (no vertex SDK in any
extra).
- Observed result: Both docs builds succeed; new sections render with
valid internal anchors (`/docs/proxy#google-vertex-ai`,
`/docs/proxy#cloud-providers`, `/docs/claude-code-vertex`,
`/docs/litellm`).
- Not tested: Live end-to-end Vertex AI request through the proxy (no
GCP project available); error messages and env-var behavior are taken
from the issue reporter's verified reproduction on v0.32.0 and
cross-checked against LiteLLM's Vertex provider docs.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Parideboy 2026-07-19 20:45:10 +02:00 committed by GitHub
parent 9cba64d89e
commit 45a5a33b33
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 85 additions and 0 deletions

View file

@ -268,6 +268,12 @@ These variables configure Headroom at runtime. Set them in your shell, `.env` fi
| `ANTHROPIC_API_KEY` | Anthropic API key (used when proxying to Anthropic) |
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | AWS credentials for Bedrock backend |
| `GOOGLE_APPLICATION_CREDENTIALS` | Google Cloud credentials for Vertex AI backend |
| `VERTEXAI_PROJECT` | GCP project id for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_PROJECT`; set it explicitly to avoid silently billing your ADC default quota project) |
| `VERTEXAI_LOCATION` | GCP region for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_LOCATION`) |
The Vertex AI backend also requires `google-cloud-aiplatform>=1.38`, which is not
included in any extra or Docker image — see
[Google Vertex AI](/docs/proxy#google-vertex-ai) for setup details.
### Proxy configuration

View file

@ -5,6 +5,13 @@ description: Add Headroom compression to LiteLLM with a single callback. Works w
Headroom integrates with [LiteLLM](https://github.com/BerriAI/litellm) as a callback that compresses messages before they reach any provider. One line to enable, works with all 100+ LiteLLM-supported providers.
<Callout type="info">
Looking for the proxy's `--backend litellm-vertex` / `vertex_ai` / `bedrock` options
instead? Those make the Headroom **proxy** call cloud providers through LiteLLM — a
different mechanism from the callback documented here. See
[Cloud providers](/docs/proxy#cloud-providers).
</Callout>
## Installation
```bash

View file

@ -359,6 +359,78 @@ headroom proxy --backend azure
OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter
```
### Google Vertex AI
`--backend vertex_ai` delegates to [LiteLLM](https://docs.litellm.ai/docs/providers/vertex),
which brings two requirements that are easy to miss:
**1. Install the Vertex SDK.** `google-cloud-aiplatform` is not included in any
Headroom extra (`[proxy]`, `[all]`, …) or Docker image variant, so install it
alongside Headroom:
```bash
pip install "headroom-ai[proxy]" "google-cloud-aiplatform>=1.38"
```
Without it, the first Vertex request fails with
`litellm.BadRequestError: … vertexai import failed … No module named 'vertexai'`.
**2. Set the LiteLLM project/location variables.** LiteLLM reads the GCP project
and region from `VERTEXAI_PROJECT` / `VERTEXAI_LOCATION` — these are **not** the
standard Google Cloud variables (`GOOGLE_CLOUD_PROJECT` / `GOOGLE_CLOUD_LOCATION`)
used by `gcloud` and current Google SDKs:
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json # or use ADC
export VERTEXAI_PROJECT=<your-gcp-project>
export VERTEXAI_LOCATION=us-central1
headroom proxy --backend vertex_ai --region us-central1
```
<Callout type="warn">
If `VERTEXAI_PROJECT` is unset, requests do not fail loudly — they can silently
resolve against your Application Default Credentials' default quota project,
billing a different GCP project than you intended. Set it explicitly even if
`GOOGLE_CLOUD_PROJECT` is already exported.
</Callout>
**Backend name aliases.** `vertex_ai`, `vertex`, `google-vertex`, `googlevertex`,
`litellm-vertex`, and `litellm-vertex_ai` are all normalized to the same
LiteLLM-backed `vertex_ai` backend — CLI help text and older docs use these
spellings interchangeably.
Running Claude Code against Claude models on Vertex? See
[Claude Code on Vertex AI](/docs/claude-code-vertex) for the recommended
native Vertex-mode flow that reuses Claude Code's own GCP auth.
<Callout type="info" title="Not the same as the LiteLLM callback">
`--backend vertex_ai` runs Headroom **as a proxy that itself calls Vertex via
LiteLLM**. The [LiteLLM integration page](/docs/litellm) documents the inverse:
adding Headroom as a compression callback inside your own LiteLLM app. Despite
the shared name, they are different mechanisms.
</Callout>
### Native Vertex passthrough routes
Separately from `--backend vertex_ai`, the proxy always registers routes that
mirror Vertex's native REST shape verbatim — no backend flag needed:
```text
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:generateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamGenerateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:countTokens
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:rawPredict
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamRawPredict
```
Requests with `publisher=google` (Gemini models) go through Headroom's full
Gemini optimization handler; `publisher=anthropic` (Claude on Vertex) routes
through the same LiteLLM-Vertex path as `--backend vertex_ai`. Any client that
already speaks the native Vertex REST API can simply point its endpoint at the
proxy — this is the mechanism `headroom wrap claude` uses in
[Vertex mode](/docs/claude-code-vertex).
### Bedrock via a local gateway
`--backend bedrock` accepts **Anthropic** input (`/v1/messages`) and re-signs to AWS. Some setups are the other way around: the client already speaks **Bedrock** (e.g. Claude Code with `CLAUDE_CODE_USE_BEDROCK=1`, or any AWS SDK pointed at a custom endpoint), sending `POST /model/{id}/invoke` to a local gateway that re-signs and forwards to AWS (LiteLLM, LocalStack, a corporate Bedrock proxy).