mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
docs(proxy): document Vertex AI backend setup, env vars, aliases, native passthrough (#2422)
## Description Documents the Vertex AI proxy backend properly, fixing #2393. Following the docs verbatim (`pip install "headroom-ai[proxy]"` + `headroom proxy --backend vertex_ai`) currently fails with `vertexai import failed`, and the LiteLLM-specific `VERTEXAI_PROJECT`/`VERTEXAI_LOCATION` env vars are documented nowhere — risking requests silently resolving against the ADC default quota project and billing the wrong GCP project. All documented behavior was verified against source: alias normalization in `headroom/providers/registry.py` (`vertex`/`google-vertex`/`googlevertex` → `vertex_ai`), the always-registered native publisher passthrough routes in `headroom/providers/proxy_routes.py`, and `pyproject.toml` (no extra pulls in `google-cloud-aiplatform`). ## Type of Change - [ ] Bug fix - [ ] New feature - [x] Documentation update - [ ] Refactor - [ ] Other ## Changes Made - `docs/content/docs/proxy.mdx`: new **Google Vertex AI** subsection under Cloud providers — `google-cloud-aiplatform>=1.38` requirement (not in any extra or Docker image), `VERTEXAI_PROJECT`/`VERTEXAI_LOCATION` env vars with a warning about silent ADC quota-project fallback and their distinction from the standard `GOOGLE_CLOUD_PROJECT`/`GOOGLE_CLOUD_LOCATION` vars, backend name alias equivalence (`vertex_ai` / `vertex` / `google-vertex` / `googlevertex` / `litellm-vertex` / `litellm-vertex_ai`), and cross-links to the Claude Code on Vertex page and the LiteLLM callback page. - `docs/content/docs/proxy.mdx`: new **Native Vertex passthrough routes** subsection documenting the unconditionally registered `/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:*` routes and the `publisher=google` (Gemini handler) vs `publisher=anthropic` (LiteLLM-Vertex path) branching. - `docs/content/docs/installation.mdx`: added `VERTEXAI_PROJECT` and `VERTEXAI_LOCATION` rows to the LLM provider keys table, plus a pointer to the new Vertex section for the SDK dependency. - `docs/content/docs/litellm.mdx`: cross-reference callout distinguishing the LiteLLM callback integration from the proxy's `litellm-*` backends (issue gap #5). ## Testing - [x] Docs build passes locally **Test Output** ``` $ npm run build # docs/ — same as CI validate-nextjs ✓ Static + SSG pages generated (exit code 0), /docs/proxy, /docs/installation, /docs/litellm prerendered $ mkdocs build # same as CI validate-mkdocs INFO - Documentation built in 8.32 seconds ``` ## Real Behavior Proof - Environment: Windows 11, Node 20, npm 10, Python 3.13, mkdocs-material (latest), branch `docs/2393-vertex-ai-backend` off `upstream/main`. - Exact command / steps: `cd docs && npm ci && npm run build`; `mkdocs build` from repo root; manually re-verified each documented claim against `headroom/providers/registry.py` (alias normalization), `headroom/providers/proxy_routes.py` (publisher passthrough routes), and `pyproject.toml` `[project.optional-dependencies]` (no vertex SDK in any extra). - Observed result: Both docs builds succeed; new sections render with valid internal anchors (`/docs/proxy#google-vertex-ai`, `/docs/proxy#cloud-providers`, `/docs/claude-code-vertex`, `/docs/litellm`). - Not tested: Live end-to-end Vertex AI request through the proxy (no GCP project available); error messages and env-var behavior are taken from the issue reporter's verified reproduction on v0.32.0 and cross-checked against LiteLLM's Vertex provider docs. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
9cba64d89e
commit
45a5a33b33
3 changed files with 85 additions and 0 deletions
|
|
@ -268,6 +268,12 @@ These variables configure Headroom at runtime. Set them in your shell, `.env` fi
|
|||
| `ANTHROPIC_API_KEY` | Anthropic API key (used when proxying to Anthropic) |
|
||||
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | AWS credentials for Bedrock backend |
|
||||
| `GOOGLE_APPLICATION_CREDENTIALS` | Google Cloud credentials for Vertex AI backend |
|
||||
| `VERTEXAI_PROJECT` | GCP project id for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_PROJECT`; set it explicitly to avoid silently billing your ADC default quota project) |
|
||||
| `VERTEXAI_LOCATION` | GCP region for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_LOCATION`) |
|
||||
|
||||
The Vertex AI backend also requires `google-cloud-aiplatform>=1.38`, which is not
|
||||
included in any extra or Docker image — see
|
||||
[Google Vertex AI](/docs/proxy#google-vertex-ai) for setup details.
|
||||
|
||||
### Proxy configuration
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ description: Add Headroom compression to LiteLLM with a single callback. Works w
|
|||
|
||||
Headroom integrates with [LiteLLM](https://github.com/BerriAI/litellm) as a callback that compresses messages before they reach any provider. One line to enable, works with all 100+ LiteLLM-supported providers.
|
||||
|
||||
<Callout type="info">
|
||||
Looking for the proxy's `--backend litellm-vertex` / `vertex_ai` / `bedrock` options
|
||||
instead? Those make the Headroom **proxy** call cloud providers through LiteLLM — a
|
||||
different mechanism from the callback documented here. See
|
||||
[Cloud providers](/docs/proxy#cloud-providers).
|
||||
</Callout>
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -359,6 +359,78 @@ headroom proxy --backend azure
|
|||
OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter
|
||||
```
|
||||
|
||||
### Google Vertex AI
|
||||
|
||||
`--backend vertex_ai` delegates to [LiteLLM](https://docs.litellm.ai/docs/providers/vertex),
|
||||
which brings two requirements that are easy to miss:
|
||||
|
||||
**1. Install the Vertex SDK.** `google-cloud-aiplatform` is not included in any
|
||||
Headroom extra (`[proxy]`, `[all]`, …) or Docker image variant, so install it
|
||||
alongside Headroom:
|
||||
|
||||
```bash
|
||||
pip install "headroom-ai[proxy]" "google-cloud-aiplatform>=1.38"
|
||||
```
|
||||
|
||||
Without it, the first Vertex request fails with
|
||||
`litellm.BadRequestError: … vertexai import failed … No module named 'vertexai'`.
|
||||
|
||||
**2. Set the LiteLLM project/location variables.** LiteLLM reads the GCP project
|
||||
and region from `VERTEXAI_PROJECT` / `VERTEXAI_LOCATION` — these are **not** the
|
||||
standard Google Cloud variables (`GOOGLE_CLOUD_PROJECT` / `GOOGLE_CLOUD_LOCATION`)
|
||||
used by `gcloud` and current Google SDKs:
|
||||
|
||||
```bash
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json # or use ADC
|
||||
export VERTEXAI_PROJECT=<your-gcp-project>
|
||||
export VERTEXAI_LOCATION=us-central1
|
||||
|
||||
headroom proxy --backend vertex_ai --region us-central1
|
||||
```
|
||||
|
||||
<Callout type="warn">
|
||||
If `VERTEXAI_PROJECT` is unset, requests do not fail loudly — they can silently
|
||||
resolve against your Application Default Credentials' default quota project,
|
||||
billing a different GCP project than you intended. Set it explicitly even if
|
||||
`GOOGLE_CLOUD_PROJECT` is already exported.
|
||||
</Callout>
|
||||
|
||||
**Backend name aliases.** `vertex_ai`, `vertex`, `google-vertex`, `googlevertex`,
|
||||
`litellm-vertex`, and `litellm-vertex_ai` are all normalized to the same
|
||||
LiteLLM-backed `vertex_ai` backend — CLI help text and older docs use these
|
||||
spellings interchangeably.
|
||||
|
||||
Running Claude Code against Claude models on Vertex? See
|
||||
[Claude Code on Vertex AI](/docs/claude-code-vertex) for the recommended
|
||||
native Vertex-mode flow that reuses Claude Code's own GCP auth.
|
||||
|
||||
<Callout type="info" title="Not the same as the LiteLLM callback">
|
||||
`--backend vertex_ai` runs Headroom **as a proxy that itself calls Vertex via
|
||||
LiteLLM**. The [LiteLLM integration page](/docs/litellm) documents the inverse:
|
||||
adding Headroom as a compression callback inside your own LiteLLM app. Despite
|
||||
the shared name, they are different mechanisms.
|
||||
</Callout>
|
||||
|
||||
### Native Vertex passthrough routes
|
||||
|
||||
Separately from `--backend vertex_ai`, the proxy always registers routes that
|
||||
mirror Vertex's native REST shape verbatim — no backend flag needed:
|
||||
|
||||
```text
|
||||
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:generateContent
|
||||
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamGenerateContent
|
||||
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:countTokens
|
||||
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:rawPredict
|
||||
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamRawPredict
|
||||
```
|
||||
|
||||
Requests with `publisher=google` (Gemini models) go through Headroom's full
|
||||
Gemini optimization handler; `publisher=anthropic` (Claude on Vertex) routes
|
||||
through the same LiteLLM-Vertex path as `--backend vertex_ai`. Any client that
|
||||
already speaks the native Vertex REST API can simply point its endpoint at the
|
||||
proxy — this is the mechanism `headroom wrap claude` uses in
|
||||
[Vertex mode](/docs/claude-code-vertex).
|
||||
|
||||
### Bedrock via a local gateway
|
||||
|
||||
`--backend bedrock` accepts **Anthropic** input (`/v1/messages`) and re-signs to AWS. Some setups are the other way around: the client already speaks **Bedrock** (e.g. Claude Code with `CLAUDE_CODE_USE_BEDROCK=1`, or any AWS SDK pointed at a custom endpoint), sending `POST /model/{id}/invoke` to a local gateway that re-signs and forwards to AWS (LiteLLM, LocalStack, a corporate Bedrock proxy).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue